Security News
Introducing the Socket Python SDK
The initial version of the Socket Python SDK is now on PyPI, enabling developers to more easily interact with the Socket REST API in Python projects.
When reading from many possibly large files in a fashion similar to random access, it is usually the fastest and most efficient to use memory maps.
Although memory maps have many advantages, they represent a very limited system resource as every map uses one file descriptor, whose amount is limited per process. On 32 bit systems, the amount of memory you can have mapped at a time is naturally limited to theoretical 4GB of memory, which may not be enough for some applications.
__del__()
destructor.Smmap wraps an interface around mmap and tracks the mapped files as well as the amount of clients who use it. If the system runs out of resources, or if a memory limit is reached, it will automatically unload unused maps to allow continued operation.
To allow processing large files even on 32 bit systems, it allows only portions of the file to be mapped. Once the user reads beyond the mapped region, smmap will automatically map the next required region, unloading unused regions using a LRU algorithm.
Although the library can be used most efficiently with its native interface, a Buffer implementation is provided to hide these details behind a simple string-like interface.
For performance critical 64 bit applications, a simplified version of memory mapping is provided which always maps the whole file, but still provides the benefit of unloading unused mappings on demand.
The package was tested on all of the previously mentioned configurations.
Its easiest to install smmap using the pip program:
$ pip install smmap
As the command will install smmap in your respective python distribution, you will most likely need root permissions to authorize the required changes.
If you have downloaded the source archive, the package can be installed by running the setup.py
script:
$ python setup.py install
It is advised to have a look at the Usage Guide for a brief introduction on the different database implementations.
The project is home on github at https://github.com/gitpython-developers/smmap .
The latest source can be cloned from github as well:
For support, please use the git-python mailing list:
Issues can be filed on github:
A link to the pypi page related to this repository:
smmap is licensed under the New BSD License.
FAQs
A pure Python implementation of a sliding window memory map manager
We found that smmap demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
The initial version of the Socket Python SDK is now on PyPI, enabling developers to more easily interact with the Socket REST API in Python projects.
Security News
Floating dependency ranges in npm can introduce instability and security risks into your project by allowing unverified or incompatible versions to be installed automatically, leading to unpredictable behavior and potential conflicts.
Security News
A new Rust RFC proposes "Trusted Publishing" for Crates.io, introducing short-lived access tokens via OIDC to improve security and reduce risks associated with long-lived API tokens.