Package log is an important part of the application and having a consistent
logging mechanism and structure is mandatory. With several teams writing
different components that talk to each other, being able to read each others
logs could be the difference between finding bugs quickly or wasting hours.
With the log package in the standard library, we have the ability to create
custom loggers that can be configured to write to one or many devices. Since
we use syslog to send logging output to a central log repository, our logger
can be configured to just write to stdout. This not only simplifies things
for us, but will keep each log trace in correct sequence.
This package does not included logging levels. Everything needs to be logged
to help trace the code and find bugs. There is no such thing as over logging.
By the time you decide to change the logging level, it is always too late.
The question of performance comes up quite a bit. If the only performance
issue we see is coming from logging, we are doing very well. I have had these
opinions for a long time, but if you want more clarity on the subject listen
to this recent podcast:
Jon Gifford On Logging And Logging Infrastructure:
Robert Blumen talks to Jon Gifford of Loggly about logging and logging
infrastructure. Topics include logging defined, purposes of logging, uses of
logging in understanding the run-time behavior of programs, who produces logs,
who consumes logs and for what reasons, software as the consumer of logs,
log formats (structured versus free form), log meta-data, logging APIs,
logging as coding, logging and frameworks, the massive hairball of log file
management, modern logging infrastructure in which log records are stored and
indexed in a search engine, how searchable logs have transformed the uses of
log data, log data and analytics, leveraging the log database for statistical
insights, performance and resource issues of logging, are logs really different
than other data that systems record in databases, and how log visualization gives
users insights into their system. The show wraps up with a discussion of open
source logging platforms versus commercial SAAS providers.
There are two types of tracing lines we need to log. One is a trace line that
describes where the program is, what it is doing and any data associated with
that trace. The second is formatted data such as a JSON document or binary
dump of data. Each serve a different purpose but they both exists within the
same scope of space and time.
The format of each trace line needs to be consistent and helpful or else the
logging will just be noise and ultimately useless.
Here is a breakdown of each section and a sample value:
Here are examples of how trace lines would show in the log:
In the end, we want to see the flow of most functions starting and completing
so we can follow the code in the logs. We want to quickly see and filter errors,
which can be accomplished by using a capitalized version of the word ERROR.
The context is an important value. The context allows us to extract trace lines
for one context over others. Maybe in this case 8890 represents a user id.
When there is a need to dump formatted data into the logs, there are three
approaches. If the data can be represented as key/value pairs, you can write
each pair on their own line with the DATA tag:
When there is a single block of data to dump, then it can be written as a
single multi-line trace:
When special block formatting required, the Stringer interface can be
implemented to format data in custom ways:
The API for the log package is focused on initializing the logger and then
provides function abstractions for the different tags we have defined.