@serverless-guru/logger

Logger

Logger is an opinionated logger utility for Javascript. Its aim is to simplify log analysis with CloudWatchLogs Insight.

Key features

• Small footprint
• Enforces structured and consistent logs across all your Lambda functions
• Automatically masks sensible values
• Automatically compresses large payloads (>25Kb)
• Automatically ignores very large payloads (>60Kb)
• Supports CloudwatchLogs text and JSON format

Environment variables

• SG_LOGGER_LOG_EVENT: Log event, default: true
• SG_LOGGER_SKIP_MASK: Skip masking of sensible values, default: false
• SG_LOGGER_MAX_SIZE: Skip logging payload bigger than size (in bytes), default: 60000
• SG_LOGGER_NO_SKIP: Don't skip payloads bigger than SG_LOGGER_MAX_SIZE, default: false
• SG_LOGGER_COMPRESS_SIZE: Compress (gzip) payload bigger than size (in bytes), default: 25000
• SG_LOGGER_NO_COMPRESS: Don't compress logs bigger than SG_LOGGER_COMPRESS_SIZE, default: false
• SG_LOGGER_LOG_TS: Add timestamp (in ms) to the output object (useful when not using Cloudwatch Logs), default: false

Log schema

{
  "timestamp": 1729066777619,
  "service": "myService",
  "level": "INFO",
  "correlationId": "092f5cf0-d1c8-4a71-a8a0-3c86aeb1c212",
  "message": "my message",
  "context": {
    "handlerNamespace": "multiply",
    "factor": 2
  },
  "payload": {
    "key1": "value1",
    "key2": 3,
    "key3": {
        "key31": "value31"
    }
  }
}

Error schema

Without an Error object

logger.error('global error', {key1: 'value1'})

{
  "timestamp": 1729066777619,
  "service": "myService",
  "level": "ERROR",
  "correlationId": "3bfd61c4-8934-4ae9-b646-d57144094986",
  "message": "invalid factor",
  "context": {
    "handlerNamespace": "multiply",
    "factor": 2
  },
  "payload": {
    "key1": "value1"
  }
}

With an Error object

logger.error('global error', new RangeError('invalid factor', {
  cause: {
    factor: event.factor,
    limit: 10,
    reason: 'too big'
  }
}))

{
  "timestamp": 1729066777619,
  "service": "myService",
  "level": "ERROR",
  "correlationId": "3bfd61c4-8934-4ae9-b646-d57144094986",
  "message": "global error",
  "context": {
    "handlerNamespace": "multiply",
    "factor": 2
  },
  "error": {
      "name": "RangeError",
      "location": "/path/to/file.js:341",
      "message": "invalid factor",
      "stack": "RangeError: invalid factor\n    at main (/path/to/file.js:341:15)\n    at /path/to/file2.js:953:30\n    at new Promise (<anonymous>)\n    at AwsInvokeLocal.invokeLocalNodeJs (/path/to/file3.js:906:12)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)",
      "cause": {
          "factor": 12,
          "limit": 10,
          "reason": "too big"
      }
  }
}

Installation

npm i --save @serverless-guru/logger

Usage

The Logger instance can be re-used across modules, allowing to keep globally defined context keys.

helpers/logger.js

const { Logger } = require("@serverless-guru/logger");
const logger = new Logger("myService", "myFirstApplication");
const metricUnits = Logger.METRIC_UNITS;
module.exports = { logger, metricUnits };

helpers/math.js

const { logger } = require('./logger')

export const multiply = async (n, factor) => {
  const sleepMs = Math.floor(Math.random() * 1000 * factor)

  await delay(sleepMs)

  const result = n * factor

  logger.debug('Multiply', { n, duration: sleepMs, result })

  return result
}

const delay = (ms) => {
  return new Promise((resolve) => setTimeout(resolve, ms))
}

handlers/multiply.js

const { logger, metricUnits } = require("../helpers/logger.js");
const LOG_FORMAT = process.env.AWS_LAMBDA_LOG_FORMAT || 'Text'

const main = async (event, context) => {
  try {
    logger.setCorrelationId(context.awsRequestId)
    logger.addContextKey({
      handlerNamespace: 'multiply',
      logFormat: LOG_FORMAT,
    })
    logger.logInputEvent({ event });

    if (event.factor) {
      logger.addContextKey({ factor: event.factor })
      if (event.factor > 10) {
        const cause = { factor: event.factor, limit: 10, reason: 'too big' }
        logger.error('invalid factor', cause)
        throw new RangeError('invalid factor', { cause })
      }
    }

    const start = new Date().getTime()
    const promises = [1, 2, 3, 4, 5].map((n) => multiply(n, event.factor || 1))
    const result = await Promise.all(promises)
    const end = new Date().getTime()

    
    logger.info('Result', { result }, {}, ['factor'])

    logger.metric('multiply', {
      name: 'Duration',
      unit: metricUnits.Milliseconds,
      value: end - start,
      dimensions: [['LOG_FORMAT', LOG_FORMAT]],
    })
  } catch(error) {
    logger.error('global error', error)
  } finally {
    logger.clearLogContext()
  }
};

module.exports = { main };

The importance of CorrelationId

Why define a correlationId when we already have a requestId provided by AWS?.

The requestId is unique inside a single Lambda invocation. A correlationId can be passed to other services and allows to extract logs from multiple services invoked during a specific activity.

Let's consider the case of a Lambda behind an API Gateway. This function sends a message to SQS, which is then processed by another Lambda function invoking a remote API.

• The Web client generates a correlationId and passes it in the payload to API Gateway (API Gateway Logs are set to log Payloads in JSON)
• The first Lambda uses the setCorrelationId method to assign the correlationId from the payload to all log outputs
• The correlationId is part of the payload sent to SQS
• The second Lambda uses the setCorrelationId method to assign the correlationId from the SQS event to all log outputs
• The correlationId is added to the invocation payload of the remote API.

Using CloudWatchLogs insight, it is now possible to query simultaneously both Lambda LogGroups, API Gateway LogGroup with a single simple query:

fields @timestamp, @message
| filter correlationId="092f5cf0-d1c8-4a71-a8a0-3c86aeb1c212"
| limit 200

To get the logs of all events for the specific correlationId across multiple services.

Class methods

Constructor

const logger = new Logger(serviceName,applicationName,correlationId)

• serviceName [string, mandatory]: Added to each log output
• applicationName [string, mandatory]: Defines the Namespace for metrics
• correlationId [string, optional]: A new UUIDv4 is generated when not defined. Added to each log output.

setCorrelationId

Set a correlationId used across all log statements. Useful when the correlationId is received as payload to the Lambda function.

logger.setCorrelationId(correlationId)

• correlationId [string, mandatory]

getCorrelationId

Retrieves the current correlationId. Useful when the correlationId needs to be passed to API calls or other service integrations.

const correlationId = logger.getCorrelationId()

• correlationId [string, mandatory]

logInputEvent

Logs the object passed as argument when the environment variable LOG_EVENT is set to "true". Generally used to conditionally log the incoming event, but it can be used for any other payload too.

The message key will always be Input Event.

logger.logInputEvent(payload)

• payload [object]

Example

To conditionally log the incoming event, the Lambda context and the environment variables:

logger.logInputEvent({event, context, env: process.env})

addContextKey

Add keys to the context object. Keys added to the context are available in all log outputs under the top level context key. useful to automatically add values to all future logs.

logger.addContextKey(contextObject)

• contextObject: [object]

clearLogContext

Clears the all context keys. This needs to be invoked at the end of each Lambda invocation to avoid re-using context keys across subsequent invocation.

logger.clearLogContext()

log

Prints a log message.

logger.log(level, message, payload, context, sensitiveAttributes)

• level [string, mandatory]: one of info, debug, warn, error
• message [string, mandatory]: Assigned to the output message. It is good practice to keep it concise and describe the activity. Re-use the same message across multiple logs, identify the individual activities using context or payload values.
• payload [string, object]: The payload to log
• context [object]: Keys to add to the context of this log output
• sensitiveAttributes [array of string]: Additional attributes to mask in this log output

Shorthand

• logger.info(message, payload, context, sensitiveAttributes)
• logger.debug(message, payload, context, sensitiveAttributes)
• logger.warn(message, payload, context, sensitiveAttributes)
• logger.error(message, payload, context, sensitiveAttributes)

Default masked attributes

Any key, be it in the payload or the context, having one of this values will be masked in the output:

• password
• userid
• token
• secret
• key
• x-api-key
• bearer
• authorization

Masking can be disabled, by setting the environment variable LOG_MASK="false".

metric

This generates a log output in EMF format, creating a metric in Cloudwatch Metrics. The metrics will be available under the namespace defined by this.applicationName.

logger.metric(activity: string, meta: MetricMeta)

• activity [string, mandatory]: The default dimension
• meta [object]
  • name [string]: The name of the metric
  • value [number]: The value of the metric
  • unit [string]: The unit of the metric (see Logger.METRIC_UNITS)
  • dimensions [Array of String pairs]: Additional dimensions for the metric. [[name1, value1], [name2, value2]]

Note: To be able to use EMF, your log group needs to be sert to standard (default) and not infrequent access.

CloudWatchLogs logFormat

Lambda allows to use CloudWatchLogs Structured format (recommended), which not only stores the logs in JSON, but also allows to set the log level directly on the log group.

Configure with Serverless Framework

Serverless v3

Serverless V3 doesn't allow to set the format directly from the function. You need to configure it via Cloudformation resources by extending the definition of the function generated by the framework.

The logical key for a function in Cloudformation is the logical key of the function with the suffix LambaFunction.

service: myService
provider:
  name: aws
  runtime: nodejs20.x
  architecture: 'arm64'
functions:
  Multiply:
    handler: src/handlers/multiply.handler
    name: multiply
    environment:
      LOG_EVENT: 'true'
resources:
  resources:
    MultiplyLambdaFunction:
      Type: AWS::Lambda::Function
      Properties:
        LoggingConfig:
          LogFormat: JSON
          ApplicationLogLevel: WARN
          SystemLogLevel: INFO

Serverless v4

With Serverless v4, the logFormat can be directly defined in the framework definition, either globally under provider or per function.

service: myService
provider:
  name: aws
  runtime: nodejs20.x
  architecture: 'arm64'
  logs:
    lambda:
      logFormat: JSON
      applicationLogLevel: WARN
      systemLogLevel: INFO
functions:
  Multiply:
    handler: src/handlers/multiply.handler
    name: multiply
    environment:
      LOG_EVENT: 'true'
    logs:
      logFormat: JSON
      applicationLogLevel: WARN
      systemLogLevel: INFO