snakemake-logger-plugin-sqlite

Snakemake Logger Plugin: SQLite

Introduction

The Snakemake Logger Plugin: SQLite is a custom logger plugin for Snakemake that writes workflow execution logs to a SQLite database. This plugin enables detailed tracking of workflows, rules, jobs, and associated files, providing a structured and queryable format for analyzing workflow execution.

Usage

• Install via pip: pip install snakemake-logger-plugin-sqlite
• Run Snakemake with the --logger sqlite option to enable the SQLite logger.

Options

TODO

Design

The plugin integrates with Snakemake's logging system and processes log events using SQLAlchemy models. It uses event handlers to parse and store log records in the database. The database schema is designed to capture key entities such as workflows, rules, jobs, and files, along with their relationships.

• Log Handler:

  • The sqliteLogHandler class processes log records and delegates them to event handlers.
  • Manages database sessions and ensures transactional consistency.

• Event Handlers:

  • Specialized handlers process specific log events (e.g., workflow start, job info, errors).
  • Handlers parse log records and update the database models accordingly.

• Database Models:

  • SQLAlchemy models represent key entities such as workflows, rules, jobs, files, and errors.
  • Models capture attributes and relationships for comprehensive logging.

• Parsers:

  • Parsers extract structured data from log records for storage in the database.

Database Schema

The database schema is designed to capture the following entities:

• Workflow: Tracks workflow metadata (e.g., id, snakefile, status, started_at, end_time) and relationships to rules, jobs, and errors.
• Rule: Represents Snakemake rules with attributes like id, name, and relationships to jobs and errors.
• Job: Captures job execution details (e.g., id, status, started_at, end_time) and relationships to files.
• File: Represents input/output files with attributes like path and file_type.
• Error: Logs errors with details such as exception, traceback, and relationships to workflows and rules.

Database Versions and Migrations

TODO

Development

TODO

Contributing

Contributions are welcome! Please follow these steps:

• Fork the repository and create a new branch.
• Make your changes and ensure all tests pass.
• Submit a pull request with a detailed description of your changes.

License

This project is licensed under the MIT License. See the LICENSE file for details.