gitlab.com/intalko/gosuite

Go Backend Commons

This package provides some subpackages that aims to ease the bootstraping process of a Go backend application. This ease the maintenance problem and help ensure similarity and maintainability standard between go backend applications developed that uses the library, as the majority of bootstraping process (like logging or database connection) has been handled.

This package is intented primarily for private usage. You may of course use this package, but it is not developed with general use case in mind.

Packages

Logging (logger)

The logger package provided by this package provides configuration and defaults for github.com/rs/zerolog:

• InitLogger accepts configuration parameter and initializes the logger. Ensure that InitLogger has been called such that the (default) logger can work correctly. Errored log outputs would not be used in the Logger, and this function should only be called once.
• the exported Logger is the main zerolog initiated by the package
• Trace, Debug, Info, Warn, Error, Fatal, and Panic, functions is a wrapper arrond the same function name of the Logger
• The default Logger currently supports writing to 5 log outputs:
  • Console writes to os.Stdout
  • DevConsole writes to console with nice formatting
  • Journald writes to the system journald
  • GcpLogging writes to Google Cloud Platform's Stackdriver / Cloud Logging
  • RotatedFile writes to log rotated file, with configurable formatting, (cron syntax) rotate frequency, min/max log files kept and max total log file size
• DefaultCronLogger adapts the Logger to format suitable for https://github.com/robfig/cron/v3
• NewCronLogger adapts a *zerolog.Logger to the cron logger. The DefaultCronLogger can be replaced with this function's return

Some exported functions that can be used beyond the default usage:

• InitRotatedfileWriterRotation: creates a rotated-file based on the RotatedFilewriterConfig to the WriterStructWithDefaultDestination. This function not be called without specifying one destination of log WriterStructWithDefaultDestination, as it initiates a rotating log scheduler by the main InitLogger function when RotatedFile is enabled.
• NewGcpLoggingWriter initiates a GCP Cloud Logging to io.Writer, suitable for log outputs such as those used by zerolog with severity map configured.

Postgres Database Connection and Migration (pgdb)

Provides database interfaces, with some defaults via

• pgx library: DbPool (or the standard library interface, StdDb)
• with connection initialization and its YAML or env configuration
• with migration: MigrateEntryPoint
• with Squirrel query builder: Qb

Connecting the exported default connections

The connection to the database should be initiated with a context.Context.

The exported DbPool and StdDb are respectively initiated using ConnectPgxPool and ConnectStdLib, respectively. Accessing the exported connection variables before successful initiation will cause failure/error/panic.

Over the two methods, querying via PgPool is better recommended for performance reasons.

Querying

Querying and executing SQL queris in application is most easily done via the exported Query, QueryRow and Exec. These are convenience wrapper arround DbPool.

If the usage of query builder is needed, the exported Qb is a great place to start to work with the squirrel query builder library. The completed/finished query can be executed with QbExec, QbQuery or QbQueryRow that executes it using DbPool.

Migration

The main migration functionality of this packages is providing a CLI capability to the application importing it via the exported function MigrateEntryPoint, that provides several key usages (the below examples assumes an application compiled into compiled executable which CLI entrypoint is set to migrate):

• compiled-executable migrate list
• compiled-executable migrate version
• compiled-executable migrate latest
• compiled-executable migrate up [steps]
• compiled-executable migrate down [steps]
• compiled-executable migrate force version
• compiled-executable migrate drop

Note:

• steps are the number of steps to migrate up or down. Optional and defaults to 1
• version are the version to be used when forcing the database into a particular version (after a failed migration, the database is "dirty" and needs to be be forced in order for migration to be able to run).

Although the helper convenience functions like MigrateGetInstance is available, the main purpose is to ease CLI migration.

(context-aware) Enhancements on pgx

By default, pgx takes a very decoupled approach on context and database. However, it would ease coding a lot if querying methods are "context-aware", that is:

• A "provider" to an interface with querying methods (Query, QueryRow and Exec - QueryExecer) is to be stored inside a context key.
• Callers to DB methods might and might not need to be inside a pgx.Tx (which is a QueryExecer), passing it optionally through function parameters is purely inconvenient.
• Querying methods search for the "provider" before falling back to the default pool (you might recognize this pattern as dependency injection).

The EnhancedDbPool aims to implement precisely this. Of course, all available methods from the underlying pgx continues to function the way they should.

Helper for main function (mainhelper)

This packages provides:

• CLI usage entrypoint quick helper that takes a map[string]func() to help declutter func main() {}
• Easy configuration loader (discussed in the Passing Configurations section)

Passing Configurations

The package mainhelper provides configuration loader functions:

• GetConfigFromYaml: from a config.yaml file
• GetConfigFromEnvironment: from environment variables, using env struct tags following github.com/sethvargo/go-envconfig
• GetConfig: from both sources

From config.yaml

The configuration types for each subpackages are decorated with yaml struct tags for relevant fields, allowing for full configuration from YAML. Applications consuming packages should has YAML based config aligned with the example YAML configuration.

The yaml being read can also be altered using --config yamlFileName.yaml for usage with alternate configuration source.

From environment variables

Following are the environment variables being read for the packages:

Logging

• Destinations: LOG_DESTINATIONS as text: "console,devconsole,gcplogging,rotate"
• GcpLoggingConfig.ProjectId: GCPLOGGING_PROJECTID
• GcpLoggingConfig.LogId: GCPLOGGING_LOGID
• GcpLoggingConfig.ServiceAccountFilePath: GOOGLE_APPLICATION_CREDENTIALS
• GcpLoggingConfig.ServiceAccountJSON: GCPLOGGING_SERVICEACCOUNTJSON
• GcpLoggingConfig.DefaultLabels: GCPLOGGING_DEFAULTLABELS
• RotatedFileConfig.FolderPath: LOGROTATE_FOLDER_PATH
• RotatedFileConfig.RotateCronFrequency: LOGROTATE_ROTATE_CRON_FREQUENCY
• RotatedFileConfig.FileNameFormat: LOGROTATE_FILE_NAME_FORMAT
• RotatedFileConfig.MaxLogFilesKept: LOGROTATE_MAX_LOG_FILES_KEPT
• RotatedFileConfig.MinLogFilesKept: LOGROTATE_MIN_LOG_FILES_KEPT
• RotatedFileConfig.MaxTotalLogFileSize: LOGROTATE_MAX_TOTAL_LOG_FILE_SIZE

Postgres / database

• Host: PG_HOST
• Port: PG_PORT
• Database: PG_DATABASE
• Username: PG_USERNAME
• Password: PG_PASSWORD
• SSL: PG_SSL
• MigrationStatementTimeoutSeconds: PG_MIGRATION_STATEMENT_TIMEOUT_SECONDS
• MigrationFromEmbedded: PG_MIGRATION_FROM_EMBEDDED
• MigrationDirectory: PG_MIGRATION_DIRECTORY