aws-logs-sink

aws-logs-sink

Stream logs to AWS CloudWatch Logs.

Either pipe your log lines to stdin from the CLI, or use this module programmatically in NodeJS––it exposes a NodeJS writable stream.

Can't I just use the AWS CLI/SDK?

Yes, but streaming logs to CloudWatch Logs is slightly less trivial than you might think, because each successive putLogEvents call must include the log stream's nextSequenceToken from the previous putLogEvents call. This module is a small wrapper around the JavaScript AWS SDK (V3) to make streaming logs easy.

CLI usage

Install globally:

npm install -g aws-logs-sink

Provide the log group name and stream name as arguments, and pipe to stdin:

echo "This is a test" | aws-logs-sink <log-group-name> <log-stream-name>

Of course a long running stream works too:

while true; do date; sleep 1; done | aws-logs-sink <log-group-name> <log-stream-name>

The log group and stream will be created if they don't exist. If the log stream already exists, its next sequence token will be queried, in order to be able to append to the stream.

CLI options:

Option	Description
-f, --flush-interval	Flush to CloudWatch every X seconds (default: 1)
--tee	Also print all input to stdout
--eol	Line termination character(s) (default: \n on Mac/Linux, \r\n on Windows)
--profile	AWS profile to use. Setting environment variable AWS_PROFILE works too
--region	AWS region to use. Setting environment variable AWS_REGION works too

Programmatic usage

Install locally:

npm install aws-logs-sink

import awsLogsSink from "aws-logs-sink";

// awsLogsSink is a function that returns a NodeJS writable stream:
const writable = awsLogsSink({
  logGroupName: "<log-group-name>",
  logStreamName: "<log-stream-name>",
  flushInterval: 1, // optional, see table above
  tee: false, // optional, see table above
  eol: "\n", // optional, see table above
  profile: "default", // optional, see table above
  region: "eu-west-1", // optional, see table above
  client: new CloudWatchLogsClient({}), // optional, will be created if not provided
});

writable.write("Hello, world!");

// Or more fancy with a pipeline:

import * as stream from "stream";

const readable = new stream.Readable({ read: () => {} });
readable.push("Hello, world! From pipeline");
readable.push(null);

stream.pipeline(readable, writable, (err) => {
  err ? console.error(err) : console.log("Done writing to CloudWatch");
});

Required AWS permissions

Permission	Required
logs:CreateLogGroup	Yes
logs:CreateLogStream	Yes
logs:PutLogEvents	Yes
logs:DescribeLogStreams	Only if writing to an existing log stream, to query its next sequence token