Security News
Introducing the Socket Python SDK
The initial version of the Socket Python SDK is now on PyPI, enabling developers to more easily interact with the Socket REST API in Python projects.
conda-package-streaming
Advanced tools
An efficient library to read from new and old format .conda and .tar.bz2 conda packages.
An efficient library to read from new and old format .conda and .tar.bz2 conda packages.
Download conda metadata from packages without transferring entire file. Get
metadata from local .tar.bz2
packages without reading entire files.
Uses enhanced pip lazy_wheel
to fetch a file out of .conda
with no more than
3 range requests, but usually 2.
Uses tar = tarfile.open(fileobj=...)
to stream remote .tar.bz2
. Closes the
HTTP request once desired files have been seen.
The basic API yields (tarfile, member) tuples from conda files as tarfile is
needed to extract member. Note the .tar.bz2
format yields all members, not
just info/
, from stream_conda_info
/ stream_conda_component
, while the
.conda
format yields members from the requested inner archive — allowing the
caller to decide when to stop reading.
From a url,
from conda_package_streaming.url import stream_conda_info
# url = (ends with .conda or .tar.bz2)
for tar, member in stream_conda_info(url):
if member.name == "info/index.json":
index_json = json.load(tar.extractfile(member))
break
From s3,
client = boto3.client("s3")
from conda_package_streaming.s3 import stream_conda_info
# key = (ends with .conda or .tar.bz2)
for tar, member in stream_conda_info(client, bucket, key):
if member.name == "info/index.json":
index_json = json.load(tar.extractfile(member))
break
From a filename,
from conda_package_streaming import package_streaming
# filename = (ends with .conda or .tar.bz2)
for tar, member in package_streaming.stream_conda_info(filename):
if member.name == "info/index.json":
index_json = json.load(tar.extractfile(member))
break
From a file-like object,
from contextlib import closing
from conda_package_streaming.url import conda_reader_for_url
from conda_package_streaming.package_streaming import stream_conda_component
filename, conda = conda_reader_for_url(url)
# file object must be seekable for `.conda` format, but merely readable for `.tar.bz2`
with closing(conda):
for tar, member in stream_conda_component(filename, conda, component="info"):
if member.name == "info/index.json":
index_json = json.load(tar.extractfile(member))
break
If you need the entire package, download it first and use the file-based APIs. The URL-based APIs are more efficient if you only need to access package metadata.
Extract conda packages (both formats)
Easy to install from pypi or conda
Do the least amount of I/O possible (no temporary files, transfer partial packages)
Open files from the network / standard HTTP / s3
Continue using conda-package-handling to create .conda packages
Uses markdown, furo theme. Requires newer mdit-py-plugins.
pip install conda-package-streaming[docs]
One time:
sphinx-apidoc -o docs .
FAQs
An efficient library to read from new and old format .conda and .tar.bz2 conda packages.
We found that conda-package-streaming demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
The initial version of the Socket Python SDK is now on PyPI, enabling developers to more easily interact with the Socket REST API in Python projects.
Security News
Floating dependency ranges in npm can introduce instability and security risks into your project by allowing unverified or incompatible versions to be installed automatically, leading to unpredictable behavior and potential conflicts.
Security News
A new Rust RFC proposes "Trusted Publishing" for Crates.io, introducing short-lived access tokens via OIDC to improve security and reduce risks associated with long-lived API tokens.