Because the Apache Python avro package is written in pure Python, it is
relatively slow. In one test case, it takes about 14 seconds to iterate through
a file of 10,000 records. By comparison, the JAVA avro SDK reads the same file in
1.9 seconds.
The fastavro library was written to offer performance comparable to the Java
library. With regular CPython, fastavro uses C extensions which allow it to
iterate the same 10,000 record file in 1.7 seconds. With PyPy, this drops to 1.5
seconds (to be fair, the JAVA benchmark is doing some extra JSON
encoding/decoding).
fastavro supports the following Python versions:
Documentation is available at http://fastavro.readthedocs.io/en/latest/
fastavro is available both on PyPI
pip install fastavro
and on conda-forge conda channel.
conda install -c conda-forge fastavro
Developer requirements can be installed with pip install -r developer_requirements.txt.
If those are installed, you can run the tests with ./run-tests.sh. If you have trouble
installing those dependencies, you can run docker build . to run the tests inside
a Docker container. This won't test on all versions of Python or on PyPy, so it's possible
to still get CI failures after making a pull request, but we can work through those errors
if/when they happen. .run-tests.sh only covers the Cython tests. In order to test the
pure Python implementation, comment out python setup.py build_ext --inplace
and re-run.
NOTE: Some tests might fail when running the tests locally. An example of this is this codec tests. If the supporting codec library is not available, the test will fail. These failures can be ignored since the tests will on pull requests and will be run in the correct environments with the correct dependencies set up.
We release both to PyPI and to conda-forge.
We assume you have twine installed and that you've created your own fork of fastavro-feedstock.
make tagmake publishSee the ChangeLog
FAQs
Fast read/write of AVRO files
We found that fastavro demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 2 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
PyPI confirms no security flaws were exploited in the Ultralytics supply chain attack and highlights improvements for safer package publishing.
Security News
Research
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
Research
Security News
Attackers used a malicious npm package typosquatting a popular ESLint plugin to steal sensitive data, execute commands, and exploit developer systems.