Huge News!Announcing our $40M Series B led by Abstract Ventures.Learn More
Socket
Sign inDemoInstall
Socket

pfio

Package Overview
Dependencies
Maintainers
4
Alerts
File Explorer

Advanced tools

Socket logo

Install Socket

Detect and block malicious and high-risk dependencies

Install

pfio

PFN IO library

  • 2.9.0
  • PyPI
  • Socket score

Maintainers
4

PFIO

Doc Status Badge

PFIO is an IO abstraction library developed by PFN, optimized for deep learning training with batteries included. It supports

  • Filesystem API abstraction with unified error semantics,
  • Explicit user-land caching system,
  • IO performance tracing and metrics stats, and
  • Fileset container utilities to save metadata.

Dependency

  • HDFS client and libhdfs for HDFS access
  • CPython >= 3.9

Installation and Document build

Installation

$ git clone https://github.com/pfnet/pfio.git
$ cd pfio
$ pip install .

Documentation

$ cd pfio/docs
$ make html
$ open build/html/index.html

How to use

Please refer to the official document for more information about the usage.

Release

Check the official document for latest release procedure.

Run tests locally:

$ pip install tox
$ tox

Bump version numbers in pfio/version.py .

Push and open a pull request to invoke CI. Once CI passed and the pull request merged, tag a release:

$ git tag -s X.Y.Z
$ git push --tags

Build:

$ rm -rf dist
$ pip3 install --user build
$ python3 -m build

Release to PyPI:

$ python3 -m pip install --user --upgrade twine
$ python3 -m twine upload --repository testpypi dist/*

Keywords

FAQs


Did you know?

Socket

Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.

Install

Related posts

SocketSocket SOC 2 Logo

Product

  • Package Alerts
  • Integrations
  • Docs
  • Pricing
  • FAQ
  • Roadmap
  • Changelog

Packages

npm

Stay in touch

Get open source security insights delivered straight into your inbox.


  • Terms
  • Privacy
  • Security

Made with ⚡️ by Socket Inc