fluentd-node

fluent-logger for Node.js

fluent-logger implementation for Node.js. Built upon fluent-logger-node.

Install

$ npm install fluent-node

Prerequistes

The fluent daemon should be listening on a TCP port.

Simple configuration is following:

<source>
  @type forward
  port 24224
</source>

<match **.*>
  @type stdout
</match>

Usage

Send an event record to Fluentd

const FluentClient = require("fluent-node").FluentClient;
const logger = new FluentClient("tag_prefix", {
  socket: {
    host: "localhost",
    port: 24224,
    timeout: 3000, // 3 seconds
  }
});

The emit method has following signature

emit(data: Record<string, any>): Promise<void>;
emit(data: Record<string, any>, timestamp: number | Date | EventTime): Promise<void>;
emit(label: string, data: Record<string, any>): Promise<void>;
emit(label: string, data: Record<string, any>, timestamp: number | Date | EventTime): Promise<void>;

The returned Promise is resolved once the event is written to the socket, or rejected if an error occurs.

Acknowledgements

Fluentd provides explicit support for acknowledgements, which allow the client to be sure that the event reached its destination.

Enabling acknowledgements means that the promise returned by emit will be resolved once the client receives an explicit acknowledgement from the server.

const FluentClient = require("fluent-node").FluentClient;
const logger = new FluentClient("tag_prefix", {
  ack: {}
});

Event modes

Fluentd provides multiple message modes, Message, Forward, PackedForward(default), CompressedPackedForward. The Fluent client supports all of them.

const FluentClient = require("fluent-node").FluentClient;
const logger = new FluentClient("tag_prefix", {
  eventMode: "Message" | "Forward" | "PackedForward" | "CompressedPackedForward"
});

Disable automatic reconnect

const logger = new FluentClient("tag_prefix", {
  socket: {
    host: "localhost",
    port: 24224,
    timeout: 3000, // 3 seconds
    disableReconnect: true
  }
});

Shared key authentication

Logger configuration:

const logger = new FluentClient("tag_prefix", {
  socket: {
    host: "localhost",
    port: 24224,
    timeout: 3000, // 3 seconds
  }
  security: {
    clientHostname: "client.localdomain",
    sharedKey: "secure_communication_is_awesome"
  }
});

Server configuration:

<source>
  @type forward
  port 24224
  <security>
    self_hostname input.testing.local
    shared_key secure_communication_is_awesome
  </security>
</source>

<match dummy.*>
  @type stdout
</match>

See also the Fluentd examples.

TLS/SSL encryption

Logger configuration:

const logger = new FluentClient("tag_prefix", {
  socket: {
    host: "localhost",
    port: 24224,
    timeout: 3000, // 3 seconds
  }
  security: {
    clientHostname: "client.localdomain",
    sharedKey: "secure_communication_is_awesome"
  }
  tls: {
    ca: fs.readFileSync('/path/to/ca_cert.pem')
  }
});

Server configuration:

<source>
  @type forward
  port 24224
  <transport tls>
    ca_cert_path /path/to/ca_cert.pem
    ca_private_key_path /path/to/ca_key.pem
    ca_private_key_passphrase very_secret_passphrase
  </transport>
  <security>
    self_hostname input.testing.local
    shared_key secure_communication_is_awesome
  </security>
</source>

<match dummy.*>
  @type stdout
</match>

FYI: You can generate certificates using fluent-ca-generate command since Fluentd 1.1.0.

See also How to enable TLS/SSL encryption.

Mutual TLS Authentication

Logger configuration:

const logger = new FluentClient("tag_prefix", {
  socket: {
    host: "localhost",
    port: 24224,
    timeout: 3000, // 3 seconds
  }
  security: {
    clientHostname: "client.localdomain",
    sharedKey: "secure_communication_is_awesome"
  }
  tls: {
    ca: fs.readFileSync('/path/to/ca_cert.pem'),
    cert: fs.readFileSync('/path/to/client-cert.pem'),
    key: fs.readFileSync('/path/to/client-key.pem'),
    passphrase: 'very-secret'
  }
});

Server configuration:

<source>
  @type forward
  port 24224
  <transport tls>
    ca_path /path/to/ca-cert.pem
    cert_path /path/to/server-cert.pem
    private_key_path /path/to/server-key.pem
    private_key_passphrase very_secret_passphrase
    client_cert_auth true
  </transport>
  <security>
    self_hostname input.testing.local
    shared_key secure_communication_is_awesome
  </security>
</source>

<match dummy.*>
  @type stdout
</match>

EventTime support

We can also specify EventTime as timestamp.

const FluentClient = require("fluent-node").FluentClient;
const EventTime = require("fluent-node").EventTime;
const eventTime = new EventTime(1489547207, 745003500); // 2017-03-15 12:06:47 +0900
const logger = new FluentClient("tag_prefix", {
  socket: {
    host: "localhost",
    port: 24224,
    timeout: 3000, // 3 seconds
  }
});
logger.emit('tag', { message: 'This is a message' }, eventTime);

Handling errors

The Fluent client will manage errors internally, and reject promises on errors. If you'd like to access the non-user facing internal errors, you can do so by passing errorHandler

const FluentClient = require("fluent-node").FluentClient;
const logger = new FluentClient("tag_prefix", {
  onSocketError: (err: Error) => {
    console.log("error!", err)
  }
});

Retrying events

Sometimes it makes sense to resubmit events if their initial submission failed. You can do this by specifying eventRetry.

const FluentClient = require("fluent-node").FluentClient;
const logger = new FluentClient("tag_prefix", {
  eventRetry: {}
});

Server

fluent-node includes a fully functional forward server which can be used as a downstream Fluent sink.

const FluentServer = require("fluent-node").FluentServer;

const server = new FluentServer({ listenOptions: { port: 24224 }});

await server.listen();

Fluentd config:

<match pattern>
  @type forward
  send_timeout 60s
  recover_wait 10s
  hard_timeout 60s

  <server>
    name fluent_node 
    host 127.0.0.1
    port 24224
    weight 60
  </server>

  <secondary>
    @type file
    path /var/log/fluent/forward-failed
  </secondary>
</match>

License

Apache License, Version 2.0.

About NodeJS versions

This package is compatible with NodeJS versions >= 12.