
Security News
Open Source CAI Framework Handles Pen Testing Tasks up to 3,600× Faster Than Humans
CAI is a new open source AI framework that automates penetration testing tasks like scanning and exploitation up to 3,600× faster than humans.
persistant dictionaries and lists for python
This is ideal for lists or dictionaries which either need persistence, are too big to fit in memory or both.
There are existing alternatives like shelve
, which are very good too.
There main difference with pysos
is that:
shelve
on windows, but slightly slower than native dbms
on linux)shelve
which relies on an underlying dbm
implementation, which may vary from system to systempip install pysos
Dictionaries:
import pysos
db = pysos.Dict('somefile')
db['hello'] = 'persistence!'
Lists:
import pysos
db = pysos.List('somefile')
db.append('it is now saved in the file')
Just to give a ballpark figure, there is a mini benchmark included in test_benchmark.py
.
Here are the results on my laptop:
Writes: 28521 / second
Reads: 188502 / second
The test is just writing 100k small key/values, and reading them all too. It's just meant to give a rough idea.
It writes every time you set a value, but only the key/value pair. So the cost of adding an item stays constant. On the other hand, lots of updates / deletes / re-inserts would lead to data fragmentation in the file. This might deteriorate performance in the long run.
No. It's not thread safe. In practice, synchronization mechanisms are typically desired on a higher level anyway.
In the original version, there was a switch to choose between sync and async mode. However, it turned out to have only a relatively small impact on overall performance. Less than 25% on the hardware/OS/data I tested if I remember right. Since the benefits seem rather low, I removed the flag and the associated code altogether, in order to ensure safety by default. IMHO, it's preferable to loose a few microseconds rather than data upon a crash.
I experimented with that too. In my experience, with the hardware/OS/data I tested, it turned out to ...suck. Using memory mapped files lead to inconsistent and unpredictible performance, often much slower than direct file access.
FAQs
Simple Object Storage - Persistent dicts and lists for python.
We found that pysos demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
CAI is a new open source AI framework that automates penetration testing tasks like scanning and exploitation up to 3,600× faster than humans.
Security News
Deno 2.4 brings back bundling, improves dependency updates and telemetry, and makes the runtime more practical for real-world JavaScript projects.
Security News
CVEForecast.org uses machine learning to project a record-breaking surge in vulnerability disclosures in 2025.