firepit

=============================== Firepit - STIX Columnar Storage

.. image:: https://img.shields.io/pypi/v/firepit.svg :target: https://pypi.python.org/pypi/firepit

.. image:: https://readthedocs.org/projects/firepit/badge/?version=latest :target: https://firepit.readthedocs.io/en/latest/?badge=latest :alt: Documentation Status

.. image:: https://github.com/opencybersecurityalliance/firepit/actions/workflows/testing.yml/badge.svg :target: https://github.com/opencybersecurityalliance/firepit :alt: Unit Test Status

.. image:: https://codecov.io/gh/opencybersecurityalliance/firepit/branch/develop/graph/badge.svg?token=Pu7pkqmE5W :target: https://codecov.io/gh/opencybersecurityalliance/firepit

Columnar storage for STIX 2.0 observations.

• Free software: Apache Software License 2.0
• Documentation: https://firepit.readthedocs.io.

Features

• Transforms STIX Observation SDOs to a columnar format
• Inserts those transformed observations into SQL (currently sqlite3 and PostgreSQL)

Motivation

STIX 2.0 JSON <https://docs.oasis-open.org/cti/stix/v2.0/stix-v2.0-part1-stix-core.html>_ is a graph-like data format. There aren't many popular tools for working with graph-like data, but there are numerous tools for working with data from SQL databases. Firepit attempts to make those tools usable with STIX data obtained from stix-shifter <https://github.com/opencybersecurityalliance/stix-shifter>_.

Firepit also supports STIX 2.1 <https://docs.oasis-open.org/cti/stix/v2.1/os/stix-v2.1-os.html>_

Firepit is primarily designed for use with the Kestrel Threat Hunting Language <https://github.com/opencybersecurityalliance/kestrel-lang>_.

Credits

This package was created with Cookiecutter_ and the audreyr/cookiecutter-pypackage_ project template.

.. _Cookiecutter: https://github.com/audreyr/cookiecutter .. _audreyr/cookiecutter-pypackage: https://github.com/audreyr/cookiecutter-pypackage

======= History

2.3.0 (2022-06-15)

• Added query.BinnedColumn so you can group by time buckets

2.2.0 (2022-06-08)

• Better STIX extension property support
  • Add a new __columns "private" table to store mapping from object path to column name
  • New path/prop metadata functions to supply metadata about STIX properties
• Improved STIX process "deterministic" id generation
  • Use a unique ID from extension properties, if found
  • Use related x-oca-asset hostname or ID if available

2.1.0 (2022-05-18)

• Add splint convert command to convert some logs files to STIX bundles

2.0.0 (2022-04-01)

• Use a "normalized" SQL database
• Initial STIX 2.1 support

1.3.0 (2021-10-04)

New assign_query API, minor query API improvements

• new way to create views via assign_query
• can now init a Query with a list instead of calling append
• Some SQL injection protection in query classes

1.2.0 (2021-08-18)

• Better support for grouped data

1.1.0 (2021-07-18)

• First stable release
• Concurrency fixes in cache()

1.0.0 (2021-05-18)

• First release on PyPI.