eflips-model

eflips-model

---

Part of the eFLIPS/simBA list of projects.

---

This repository contains both the reference specification, an SQLALchemy implementation and a usage example of the eFLIPS database.

Reference Specification

The reference specification is located (for now) in the huge PDF file schema.pdf. It is a UML diagram of the database schema. The diagram was created using OmniGraffle (proprietary).

SQLAlchemy Implementation

The SQLAlchemy implementation is located in the eflips/model directory. It is a Python package that contains the SQLAlchemy models and the database migration scripts. The package is structured as follows:

Installation

Releases of the package will be made available on https://pypi.org/. As such, it is installable with pip install eflips-model. However, it should be used by including it in other packages as a dependency. The versioning scheme is semantic versioning. This means that:

• patch releases (e.g. 1.0.0 to 1.0.1) are backwards compatible bug fixes, without schama changes
• minor releases (e.g. 1.0.0 to 1.1.0) are backwards compatible feature additions, with the schema changes being optional
• major releases (e.g. 1.0.0 to 2.0.0) are backwards incompatible changes, with the schema changes being mandatory

Supported database backends are

• PostgreSQL with the PostGIS extension (CREATE EXTENSION postgis;) and btree_gist (CREATE EXTENSION btree_gist;)

Usage

This package is not expected to be used directly. It is a dependency of the eflips-* packages.

This package utilizes GIS extensions through GeoAlchemy. However, we are not handling geometry on the python side in any special way. When developing a paclage that uses eflips-model, you will probably additionally need Shapely and pyProj, which are not pure python packages and require additional dependencies to be installed on the system.

Schema updates

The schema updates are handled by Alembic. The migration scripts are located in the eflips/model/migrations directory. To create a new migration script, execute the following commands in the root directory of the repository:

cd eflips/model
alembic revision --autogenerate -m "vx.y.z"
# Edit the migration script, as necessary

Creating a migration script is required for every change to the database schema, which should also correspond to a minor or major version change in the package version.

To apply the migration scripts, execute the following command in the root directory of the repository:

cd eflips/model
export DATABASE_URL=postgresql://user:pass@hostname:port/dbname # Change to your database URL
alembic upgrade head

Testing

We use pytest for testing. The tests are located in the tests directory. To run the tests, execute the following command in the root directory of the repository (after installing the dev dependencies):

---

NOTE: Be aware that the tests will clear the database specified in the DATABASE_URL environment variable. Make sure that you are not using a database that you want to keep.

---

# Change to your database URL
export DATABASE_URL=postgresql://user:pass@hostname:port/dbname 
pytest

Documentation

Documentation is available on Read the Docs.

To locally create the documentaiton from the docstrings in the code using sphinx-autoapi, you can create the documentation execute the following command in the root directory of the repository:

sphinx-build doc/ doc/_build -W

Development

We utilize the GitHub Flow branching structure. This means that the main branch is always deployable and that all development happens in feature branches. The feature branches are merged into main via pull requests. We utilize the semantic versioning scheme for versioning.

Dependencies are managed using poetry. To install the dependencies, execute the following command in the root directory of the repository:

poetry install

We use black for code formatting. You can use black . to format the code.

We use MyPy for static type checking. You can use mypy --strict --explicit-package-bases eflips/ to run MyPy on the code.

Please make sure that your poetry.lock and pyproject.toml files are consistent before committing. You can use poetry check to check this. This is also checked by pre-commit.

You can use pre-commit to ensure that MyPy, Black, and Poetry are run before committing. To install pre-commit, execute the following command in the root directory of the repository:

We recommend utilizing linters such as PyLint for static code analysis (but not doing everything it says blindly).

Usage Example

In examples a well-documented (german-language) Jupyter notebook can be found that explains how all pieces of the data structure fit together using the SQLAlchemy Implementation. See its README for details.

License

This project is licensed under the AGPLv3 license - see the LICENSE file for details.

Funding Notice

This code was developed as part of the project eBus2030+ funded by the Federal German Ministry for Digital and Transport (BMDV) under grant number 03EMF0402.