an ORM/ODM for Google Cloud Datastore/MongoDB/redis, featuring a compatible interface with Peewee.
weedata
The weedata is a python ODM/ORM module for Google Cloud Datastore / MongoDB / Redis, featuring a compatible interface with Peewee.
The main priority of this library is to ensure compatibility with the Peewee API.
If you don't use advanced SQL features such as multi-table join queries and more, you can easily switch between SQL and NoSQL without modifying your application code.
I know using NoSQL as if it were SQL is not a very smart idea, but it can achieve maximum compatibility with various databases, so, just choose what you like.
Alright, if you're already familiar with Peewee, you basically don't need this documentation.
The majority of the content in this document is directly taken from the official Peewee documentation, with minimal alterations. Gratitude is expressed in advance for this.
If you have any questions during use, you can first check the peewee documentation, and then try in weedata to verify if weedata supports it or not.
Only Two Steps Needed:
from peewee import *
to
from weedata import *
db = SqliteDatabase(dbName)
to one of the following lines:
db = DatastoreClient(project="AppId")
db = MongoDbClient(dbName, "mongodb://localhost:27017/")
db = RedisDbClient("AppId", "redis://127.0.0.1:6379/")
weedata supports Google Cloud Datastore, MongoDB and Redis.
To use Google Cloud Datastore, you need to install google-cloud-datastore [optional, only install if need].
To use MongoDB, you need to install pymongo [optional, only install if need].
To use redis, you need to install Redis-py [optional, only install if need].
pip install google-cloud-datastore
pip install pymongo
pip install redis
pip install weedata
To use Google Cloud Datastore, firstly you need to create a project, set up authentication. You can refer to Firestore in Datastore mode documentation or backDatastore mode client libraries or Python Client for Google Cloud Datastore API for guidance.
API signature: DatastoreClient(project=None, namespace=None, credentials=None, \_http=None)
weedata uses pymongo as the MongoDB driver. After correctly installing the MongoDB service and pymongo, create a client following this API signature. The parameter 'project' corresponds to the MongoDB database name, and 'host' can be passed as complete database url.
MongoDbClient(project, dbUrl='mongodb://127.0.0.1:27017/')
weedata uses redis-py as the redis driver. After correctly installing the redis service and redis-py, create a client following this API signature.
The parameter 'project' corresponds to the redis key prefix, and 'host' can be passed as complete database url, you can also choose which db to be used by passing the parameter 'db', the range is [0-15].
RedisDbClient(project, dbUrl='redis://127.0.0.1:6379/0', key_sep=':')
Important Notice: Redis functions as an in-memory database. Although it includes disk persistence capabilities, this feature does not ensure 100% data integrity and presents a potential risk of data loss. Prior to implementing Redis as a storage database, it is crucial to grasp pertinent knowledge and configure it appropriately, including enabling RDB (Redis Database) or AOF (Append Only File) functionality. For example, you can add two lines in redis.conf:
appendonly yes
appendfsync always
or
appendonly yes
appendfsync everysec
weedata also provides a simple database implementation PickleDbClient using Python's pickle library, which can be used for testing purposes and in applications with limited resources and low data integrity requirements.
The parameter "dbName" is the filename of pickle file, if set to ":memory:", an in-memory database is created.
PickleDbClient(dbName, bakBeforeWrite=True)
from weedata import *
db = DatastoreClient(project="AppId")
db = MongoDbClient("AppId", "mongodb://localhost:27017/")
db = RedisDbClient("AppId")
class Person(Model):
class Meta:
database = db
name = CharField()
birthday = DateTimeField()
The best practice is to define a base class that connects to the database, and then have other models within your application inherit from it.
class MyBaseModel(Model):
class Meta:
database = db
class Person(MyBaseModel):
name = CharField()
birthday = DateTimeField()
class Message(MyBaseModel):
context = TextField()
read_count = IntegerField(default=0)
Or you can setup or change database connection dynamically by using bind method.
Person.bind(db)
Message.bind(db)
#or
db.bind([Person, Message])
Let's begin by populating the database with some people. We will use the save() and create() methods to add and update people's records.
from datetime import datetime
uncle_bob = Person(name='Bob', birthday=datetime(1960, 1, 15))
uncle_bob.save()
You can also add a person by calling the create() method, which returns a model instance. The insert_many() function is a convenient method for adding many data at once:
grandma = Person.create(name='grandma', birthday=datetime(1935, 3, 1))
Person.insert_many([{'name':'Herb', 'birthday':datetime(1950, 5, 5)}, {'name':'Adam', 'birthday':datetime(1990, 9, 1)}])
You can count the number of rows in any select query:
Tweet.select().count()
Tweet.select().where(Tweet.id > 50).count()
To update data, modify the model instance and call save() to persist the changes.
Here we will change Grandma's name and then save the changes in the database.
Or you can use an update statement that supports all standard arithmetic operators:
grandma.name = 'Grandma'
grandma.save()
Person.update({Person.name: 'Grandma L.'}).where(Person.name == 'Grandma').execute() #Changing to other name
Person.update(name='Grandma').where(Person.name == 'Grandma').execute() #Changing to other name
Person.update({Person.name: 'Dear. ' + Person.name}).where(Person.birthday > datetime(1950, 5, 5)).execute() #Adding a title of respect before someone's name
# update statement supports: +, -, *, /, //, %, **, <<, >>, &, |, ^
To delete one or many instances from database:
herb.delete_instance()
Person.delete().where(Person.birthday < datetime(1950, 5, 4)).execute()
To remove the whole collection(MongoDb)/kind(datastore), you can use:
Person.drop_table()
db.drop_tables([Person, Message])
Let's retrieve Grandma's record from the database. To get a single record from the database, use Select.get():
grandma = Person.get(name = 'Grandma')
grandma = Person.get(Person.name == 'Grandma')
grandma = Person.select().where(Person.name != 'Grandma').get()
grandma = Person.select().where(Person.name == 'Grandma').first()
grandma = Person.get_or_none(Person.name == 'Grandma')
grandma = Person.get_by_id('65bda09d6efd9b1130ffccb0')
grandma = Person.select().where(Person.id == '65bda09d6efd9b1130ffccb0').first()
grandma = Person.select(Person.name, Person.birthday).where(Person.name == 'Grandma').first()
The code lines above return an instance of the Model. If, in some situations, you need a dictionary, you can use dicts() to return a standard Python dictionary.
grandma.dicts()
grandma.dicts(only=[Person.name, Person.birthday])
grandma.dicts(exclude=[Person.birthday])
grandma.dicts(remove_id=True)
Let's list all the people in the database:
for person in Person.select():
print(person.name)
Let's make sure these are sorted alphabetically by adding an order_by() clause:
for person in Person.select().where(Person.birthday <= datetime(1960, 1, 15)).order_by(Person.name):
print(person.name)
for person in Person.select().order_by(Person.birthday.desc()):
print(person.name, person.birthday)
People whose birthday is between 1940 and 1960 (inclusive of both years):
d1940 = datetime(1940, 1, 1)
d1960 = datetime(1960, 12, 31)
query = Person.select().where((Person.birthday >= d1940) & (Person.birthday <= d1960))
for person in query:
print(person.name, person.birthday)
query = Person.select().where(Person.birthday.between(d1940, d1960))
query = Person.select().where(Person.birthday >= d1940).where(Person.birthday <= d1960)
query = Person.select().where((Person.birthday < d1940) | (Person.birthday > d1960))
query = Person.select().where(~((Person.birthday < d1940) | (Person.birthday > d1960)))
The following names of fields reserved by the model, should be avoided for your fields:
_key, _id, id
Parameters accepted by all field types and their default values:
unique = False – create a unique index on this column.index = None – create an index on this column. If you want to store content longer than 1500 bytes in the datastore, set the index to False.default = None – any value or callable to use as a default for uninitialized modelsenforce_type = False – determine if the new value is of a specific type.Other parameters accepted by Peewee can be passed, weedata simply ignores them in a straightforward manner.
weedata can provide default values for fields when objects are created. For example to have an IntegerField default to zero rather than None, you could declare the field with a default value:
class Message(Model):
context = TextField()
read_count = IntegerField(default=0)
In some instances it may make sense for the default value to be dynamic. A common scenario is using the current date and time. weedata allows you to specify a function in these cases, whose return value will be used when the object is created. Note we only provide the function, we do not actually call it:
class Message(Model):
context = TextField()
timestamp = DateTimeField(default=datetime.datetime.now)
In order not to pollute the model namespace, model-specific configuration is placed in a special class called Meta (a convention borrowed from the django framework):
db = MongoDbClient("AppId", "mongodb://localhost:27017/")
class Person(Model):
name = CharField()
birthday = DateTimeField()
class Meta:
database = db
Once the class is defined, you should not access ModelClass.Meta, but instead use ModelClass._meta.
The ModelOptions class implements several methods which may be of use for retrieving model metadata.
Person._meta.fields
Person._meta.client
Now, the ModelOptions accepts two parameters:
Model.bind() or db.bind() at run time.class Meta
database = db
primary_key = 'id_'
User.get_by_id('65bda09d6efd9b1130ffccb0')
User.get(User.username == 'Charlie')
User.get(username='Charlie')
User.select().where(User.username.in_(['Charlie', 'Adam'])).order_by(User.birthday.desc()).get()
You can filter for particular records using normal python operators. weedata supports a wide variety of query operators.
The following types of comparisons are supported by weedata:
| Comparison | Meaning |
|---|---|
| == | x equals y |
| != | x is not equal to y |
| < | x is less than y |
| <= | x is less than or equal to y |
| > | x is greater than y |
| >= | x is greater than or equal to y |
| .in_(list) | IN lookup |
| .not_in(list) | NOT IN lookup. |
| .between(v1, v2) | Between lookup |
| .startswith(prefix) | lookup using a string prefix |
| & | logical AND |
| | | logical OR |
| ~ | logical NOT (mongodb only) |
user = User.select().where(User.name == 'python').get()
user = User.select().where(User.name == 'python').first()
user = User.select().where(User.name.in_(['python', 'cobra'])).first()
user = User.select().where(User.name.not_in(['python', 'cobra'])).first()
users = User.select(User.name, User.score).where(User.name == 'python').execute()
users = User.select().where(User.birthdate.between(datetime.datetime(2024,1,1), datetime.datetime(2024,2,1))).execute()
user = User.select().where((User.name != 'python') & (User.name != 'cobra')).first()
user = User.select().where(User.name != 'python').where(User.name != 'cobra').first()
users = User.select().order_by(User.birthdate.desc(), User.score).limit(10).execute()
users = User.select().where((User.name == 'python') | (User.name == 'cobra')).execute()
users = list(User.get(User.age > 30))
Tweet.select().where(Tweet.content.startswith('de'))
User.update({User.score: User.att_days + (User.evaluation * 2)}).where(User.age < 10).execute()
User.replace(name='python', score=100, birthdate=datetime.datetime(2024,1,1)).execute()
FAQs
an ORM/ODM for Google Cloud Datastore/MongoDB/redis, featuring a compatible interface with Peewee.
We found that weedata demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
In this segment of the Risky Business podcast, Feross Aboukhadijeh and Patrick Gray discuss the challenges of tracking malware discovered in open source softare.
Research
Security News
A threat actor's playbook for exploiting the npm ecosystem was exposed on the dark web, detailing how to build a blockchain-powered botnet.
Security News
NVD’s backlog surpasses 20,000 CVEs as analysis slows and NIST announces new system updates to address ongoing delays.