researchable_loggable

Loggable

Loggable is Researchable's gem to standardize how we do logging on Ruby on Rails applications.

What it does

Loggable configures how your Rails applications outputs and formats logs, in a way that is easy to export and understand by the elastic stack. Specifically it does the following:

1. Configures logs to be printed to stdout, so the elastic agent can read them.
2. Formats logs using the ecs standard.
3. Uses lograge to improve how request logging is done.
4. Enables log-correlation for the elastic apm.

After adding this gem, your (production) logs will go from looking like this:

Started GET "/api/v1/profiles/my?hola=adiosa" for 172.19.0.1 at 2023-02-21 12:48:48 +0000
Processing by Api::V1::ProfilesController#my as */*
  Parameters: {"hola"=>"adiosa"}
Completed 401 Unauthorized in 11ms (ActiveRecord: 0.0ms | MongoDB: 0.0ms | Allocations: 3355)

to looking like this:

{"@timestamp":"2023-02-21T13:11:56.457Z","log.level":"INFO","message":"method=GET path=/api/v1/profiles/my format=*/* controller=Api::V1::ProfilesController action=my status=200 duration=11.54 view=0.00 db=0.00 unpermitted_params=[\"hola\"] params={\"hola\"=>\"adiosa\"}","ecs.version":"1.4.0","source.ip":"172.19.0.1","user.id":null,"http.code":"200"}

While slightly less human readable, this is meant to be used along with a log aggregator that will display it in a readable form. Development logs remain unaffected (see Usage for more on this).

How it works

Loggable is a Railtie that overrides the default rails logger configuration. Right from the initialization process, the logger is set to ::EcsLogging::Logger and the default log level is set to INFO (this can be overridden in application.rb or your environment configuration). It also configures lograge automatically. Note that the base_controller_class is set to 'ApplicationController'. All your controllers should extend from this class.

By default lograge and ::EcsLogging::Logger are not mutually compatible. The ECS formatter will try to render the output of lograge as the message, however some of the fields in the lograge output are metadata. To fix this, this gem uses a monkey patch on the ECS gem that flattens the lograge output onto the Ecs object.

Installation

Add this to your Gemfile:

gem 'researchable_loggable', '~> 1.0'

Additionally, if you want to be able to correlate logs with Elastic's APM traces, you may also add the elastic apm gem:

gem 'elastic-apm'

Then execute:

$ bundle install

If loggable detects the presence of this gem, it will inject transaction.id, span.id and trace.id to the log line.

Usage

For a brand new Rails application, adding this gem to the Gemfile should be all the configuration that is needed. This gem will automatically configure the Rails logger and lograge to follow the Researchable standard.

Parameters

This are the parameters that can be configured in the config.loggable namespace:

production_like. By default loggable will only modify how the logs look on production. This is because the format that this gem enforces is slightly less human readable (in favour of machine readability and compatibility with the Elastic Common Schema) than the default Rails one. For development, the more human readable format is preferred. However, if you want to override this and log in the production format while on development (e.g. because you are troubleshooting a problem with the logs or you are developing against this gem) then you may add the following to your config/environment/development.rb file:

config.loggable.production_like = true

current_user_method. This is the name of the method that will be called on the controller to get the current user. By default this is set to current_user. If you want to override this, you may add the following to your config/application.rb file:

config.loggable.current_user_method = :my_current_user_method

This method is used for ECS logging, if the method is not found, the user.id field will be set to nil.

No other configuration options are provided because this gem seeks to enforce consistency across all applications.

For applications that already had some level of log customization, some cleanup might be needed:

1. If lograge was added to the Gemfile already, it may be removed. Loggable relies on lograge internally, and will make it available to the Rails application.
2. Consequently, remove config/lograge.rb and all lograge related configuration.
3. Remove all existing log configuration, specially for production environments. Places to look for this configuration are config/application.rb and config/environment/production.rb. If you chose to enable production_like mode on your development environment, you might also want to cleanup config/environment/development.rb. Things to look for when cleaning up are statements like config.logger and config.log_formatter and anything with the word log or logger in it.

Development

This gem is a Railtie, and it's meant to be run under a Rails project. The easiest way to develop and test this gem is to add it to your Rails project locally by adding this to your Rails Gemfile:

gem 'loggable', path: "<your local path to the loggable gem you're developing against>"

If your Rails application is being executed by a docker container, then map the local gem repository to a directory in the container through a volume, and add the path of the volume to the Gemfile statement instead.

License

The gem is available as open source under the terms of the MIT License.