pynosqldb

Absolutely! Here’s a full, detailed README for your PyNoSQL library, including installation, usage, JSON operations, indexing, compaction, and contribution instructions. I’ve structured it like a professional open-source project README.

PyNoSQL

PyNoSQL is a lightweight, JSON-based NoSQL database written in Python. It is designed for developers to learn, experiment, and practice NoSQL database concepts. PyNoSQL provides support for collections, indexes, write-ahead logging (WAL), compaction, and basic CRUD operations.

Features

• Collections: Organize data into collections (similar to MongoDB).
• Documents: Store JSON-compatible Python dictionaries.
• Indexes: Create B-tree indexes on fields for faster queries.
• Write-Ahead Logging (WAL): Ensures durability and crash recovery.
• Compaction: Remove stale records and tombstones to optimize storage.
• Querying: Find, update, and delete documents using JSON-like queries.
• Simple API: Lightweight and easy-to-use interface for experimentation.
• Fully in Python: No external database required.

Installation

git clone https://github.com/yourusername/pynosqldb.git
cd pynosqldb
python -m pip install -e .

The -e flag installs the library in editable mode, so you can modify it during development.

PyNoSQL requires Python 3.11+ and the msgpack library:

pip install msgpack

Usage

1. Creating or Opening a Database

from pynosql.database import Database

# Create a database
db = Database("mydb")
db.open()

2. Working with Collections

# Access or create a collection
users = db.collection("users")

3. Inserting Documents

a) Using Python dictionaries

user = {"name": "Alice", "age": 30, "email": "[e-mail]"}
user_id = users.insert(user)
print(f"Inserted document with _id: {user_id}")

b) From JSON string

import json

json_str = '{"name": "Bob", "age": 25, "email": "[e-mail]"}'
user = json.loads(json_str)
users.insert(user)

c) From JSON file

with open("users.json", "r") as f:
    data = json.load(f)

if isinstance(data, list):
    for user in data:
        users.insert(user)
else:
    users.insert(data)

4. Querying Documents

# Find all users older than 25
results = users.find({"age": 30})
for user in results:
    print(user)

5. Updating Documents

# Update documents matching a query
users.update({"name": "Alice"}, {"$set": {"age": 31}})

6. Deleting Documents

# Delete documents matching a query
users.delete({"name": "Bob"})

7. Index Management

# Create an index on the "age" field
db.create_index("users", "age")

# Indexed queries are faster
results = users.find({"age": 30})

8. Compaction

Over time, updates and deletions leave stale data and tombstones. Use compaction to optimize storage:

db.compact()

This copies only the latest version of each document into a new DB file and removes deleted records.

9. Closing the Database

Always close the database to flush data and indexes:

db.close()

Directory Structure

pynosqldb/
│
├── pynosql/
│   ├── __init__.py
│   ├── database.py
│   ├── collection.py
│   ├── storage/
│   │   ├── engine.py
│   │   ├── wal.py
│   │   ├── compaction.py
│   │   └── metadata.py
│   ├── query/
│   │   ├── matcher.py
│   ├── update/
│   │   ├── updater.py
│   ├── index/
│   │   ├── btree.py
│   │   ├── index_manager.py
│   ├── cli.py
│   ├── util.py
│   └── errors.py
│
├── tests/
├── setup.py
├── pyproject.toml
├── README.md
├── LICENSE
└── example.ipynb

License

MIT License. See LICENSE file for details.

This README now includes JSON insertion, updates, queries, deletion, indexing, compaction, and a clear example workflow.