afex-logger

Overview

AFEX Logger contains implementation codes that can help you fast-track logging in your app codebase. Integrated with a centralized log server, your logs gets submitted via a background worker utilizing celery with redis as broker.

Requirements

This library requires:

• celery
• boto3

Installation

You can get the package using pip, as the following:

pip install afex-logger

Example

Below is a quick way to get started.

First, in your django (celery) project, add afex logger in the list of your installed apps in your django settings file as below:

INSTALLED_APPS = [
    ..., # your other installed apps
    "afex_logger"
]

You need a configuration provider class that extends afex_logger.util.ConfigProvider. The ConfigProvider provides an abstract method get_api_key which your app will hook onto to specify the right api key as retrieved from SSM.

The configuration provider class can be in any location within your app, this location is where you then need to specify within your app's setting file as:

LOG_CONFIG_PROVIDER="<location_to_your_configuration_provider_class>"

The configuration provider will assume test environment by checking the settings.DEBUG

An example configuration provider will look like:

from afex_logger.util import ConfigProvider
from django.conf import settings


class LogConfigProvider(ConfigProvider):
   
   def __init__(self):
      super().__init__(settings.DEBUG)

    def get_api_key(self):
        return settings.LOG_API_KEY

Usage

1. Logs Management

   1. Request Logs:

      This package has provided a middleware that automatically submits request logs to the server. To use this, just add afex_logger.http_interceptor.LogMiddleware to your list of middleware in your settings file.

   2. Other logs:

      You can submit other classes of logs (error, process and activity logs) by using the afex_logger.util.LogUtil class as follows

         from afex_logger.util import LogUtil
         
         LogUtil.submit_process_log(...) # for process logs
         LogUtil.submit_activity_log(...) # for activity logs
         LogUtil.submit_error_log(...) # for error logs
         LogUtil.submit_requests_log(...) # to manually submit logs
      

      The logs are aggregated and kept in local repository typically in cache memory. To send aggregated logs, a celery task have to be registered to execute the log aggregation and send to the log server.

      This can be achieved by simply adding the following in the celery.py file

         app.conf.beat_schedule = {
             ...
             'afex_logger.tasks.submit_log': {
                 'task': 'afex_logger.tasks.submit_log',
                 'schedule': crontab(minute="*/3")
             },
             ...
         }
      

      We also recommend that the log tasks be processed via a different queue. For example, adding the following inside your celery.py file

         app.conf.task_routes = {                                 
             ...,
             'afex_logger.tasks.aggregate_log': {'queue': 'logger_tasks'},
             'afex_logger.tasks.submit_log': {'queue': 'logger_tasks'},
             ...
         }
      

   3. Retrieving Logs:

      To retrieve any of the four classes of logs, use the same LogUtil class.

         from afex_logger.util import LogUtil
         
         parameters = {
            "page": 1,
            "page_size": 15,
            "keyword": "...", # this us tge field name to filter from
            "filter": "..." # this is the value to filter with
         }
         paged_process_logs, error_1 = LogUtil.fetch_process_logs(parameters) # fetching process logs
         paged_activity_logs, error_2 = LogUtil.fetch_activity_logs(parameters) # fetching activity logs
         paged_error_logs, error_3 = LogUtil.fetch_error_logs(parameters) # fetching error logs
         paged_request_logs, error_4 = LogUtil.fetch_request_logs(parameters) # fetching request logs
      

2. SSM API Key Management

   To retrieve API Keys from SSM, use afex_logger.ssm_service.SsmService

      
      from afex_logger.ssm_service import SsmService
      
      ssm_credentials = {...}
      secret_name = ...
      key_name = ...
      service = SsmService(**ssm_credentials)
   
      api_key, error = service.get_secret_value(secret_name, key_name)