umongo

====================== μMongo: sync/async ODM

.. image:: https://img.shields.io/pypi/v/umongo.svg :target: https://pypi.python.org/pypi/umongo :alt: Latest version

.. image:: https://img.shields.io/pypi/pyversions/umongo.svg :target: https://pypi.org/project/umongo/ :alt: Python versions

.. image:: https://img.shields.io/badge/marshmallow-3-blue.svg :target: https://marshmallow.readthedocs.io/en/latest/upgrading.html :alt: marshmallow 3 only

.. image:: https://img.shields.io/pypi/l/umongo.svg :target: https://umongo.readthedocs.io/en/latest/license.html :alt: License

.. image:: https://dev.azure.com/lafrech/umongo/_apis/build/status/Scille.umongo?branchName=master :target: https://dev.azure.com/lafrech/umongo/_build/latest?definitionId=1&branchName=master :alt: Build status

.. image:: https://readthedocs.org/projects/umongo/badge/ :target: http://umongo.readthedocs.io/ :alt: Documentation

μMongo is a Python MongoDB ODM. It inception comes from two needs: the lack of async ODM and the difficulty to do document (un)serialization with existing ODMs.

From this point, μMongo made a few design choices:

• Stay close to the standards MongoDB driver to keep the same API when possible: use find({"field": "value"}) like usual but retrieve your data nicely OO wrapped !
• Work with multiple drivers (PyMongo_, TxMongo_, motor_asyncio_ and mongomock_ for the moment)
• Tight integration with Marshmallow_ serialization library to easily dump and load your data with the outside world
• i18n integration to localize validation error messages
• Free software: MIT license
• Test with 90%+ coverage ;-)

.. _PyMongo: https://api.mongodb.org/python/current/ .. _TxMongo: https://txmongo.readthedocs.org/en/latest/ .. _motor_asyncio: https://motor.readthedocs.org/en/stable/ .. _mongomock: https://github.com/vmalloc/mongomock .. _Marshmallow: http://marshmallow.readthedocs.org

µMongo requires MongoDB 4.2+ and Python 3.7+.

Quick example

.. code-block:: python

import datetime as dt
from pymongo import MongoClient
from umongo import Document, fields, validate
from umongo.frameworks import PyMongoInstance

db = MongoClient().test
instance = PyMongoInstance(db)

@instance.register
class User(Document):
    email = fields.EmailField(required=True, unique=True)
    birthday = fields.DateTimeField(validate=validate.Range(min=dt.datetime(1900, 1, 1)))
    friends = fields.ListField(fields.ReferenceField("User"))

    class Meta:
        collection_name = "user"

# Make sure that unique indexes are created
User.ensure_indexes()

goku = User(email='[e-mail]', birthday=dt.datetime(1984, 11, 20))
goku.commit()
vegeta = User(email='[e-mail]', friends=[goku])
vegeta.commit()

vegeta.friends
# <object umongo.data_objects.List([<object umongo.dal.pymongo.PyMongoReference(document=User, pk=ObjectId('5717568613adf27be6363f78'))>])>
vegeta.dump()
# {id': '570ddb311d41c89cabceeddc', 'email': '[e-mail]', friends': ['570ddb2a1d41c89cabceeddb']}
User.find_one({"email": '[e-mail]'})
# <object Document __main__.User({'id': ObjectId('570ddb2a1d41c89cabceeddb'), 'friends': <object umongo.data_objects.List([])>,
#                                 'email': '[e-mail]', 'birthday': datetime.datetime(1984, 11, 20, 0, 0)})>

Get it now::

$ pip install umongo           # This installs umongo with pymongo
$ pip install my-mongo-driver  # Other MongoDB drivers must be installed manually

Or to get it along with the MongoDB driver you're planing to use::

$ pip install umongo[motor]
$ pip install umongo[txmongo]
$ pip install umongo[mongomock]

======= History

3.1.0 (2021-12-23)

Features:

• Add fields list to Document and EmbeddedDocument.__dir__() (see #367).

Bug fixes:

• Test database by comparing to None, not casting to bool to prevent an exception raised by pymongo >= 4 (see #366).

3.0.1 (2021-10-16)

Bug fixes:

• Fix ListField.insert: trigger set_modified, deserialize using inner field (see #364).

3.0.0 (2021-01-11)

Features:

• Fix internationalization of generated marshmallow fields for container fields (DictField, ListField, NestedField) (see #329).
• Don't pass field metadata as kwargs (deprecated in marshmallow 3.10.0) but as metadata argument (see #328).

Bug fixes:

• Fix IO validation of None for references, lists and embedded documents (see #330).
• Add _dict_io_validate to propagate IO validation through DictField (see #335).

Other changes:

• Backwards-incompatible: Require marshmallow>=3.10.0 (see #328).

3.0.0b14 (2020-12-08)

Features:

• Provide Instance subclasses for each framework to help users migrating a database from umongo 2 to umongo 3 (see #319).
• Backwards-incompatible: Postpone embedded document resolution in EmbeddedField to allow passing an embedded document as string before its registration. Unknown embedded document errors in EmbeddedField are now detected at runtime, not registration time. Also, indexes are now collected on first use rather than upon registration and should be accesses through Document.indexes cached property rather than Document.opts.indexes. (see #322)
• Backwards-incompatible: Make BaseSchema ordered. This fixes querying on embedded documents. Make BaseMarshmallowSchema ordered as well. (see #323)
• Backwards-incompatible: Make RemoveMissingSchema opt-out. By default, generated pure marshmallow schemas now skip missing values from Document instances rather that returning None. This can be changed by setting MA_BASE_SCHEMA_CLS. (see #325)

3.0.0b13 (2020-11-23)

Bug fixes:

• Fix a bug introduced in 3.0.0b12 preventing instance initialization with DB as parameter as in instance = PyMongoInstance(db). (see #318)

3.0.0b12 (2020-11-16)

Features:

• Backwards-incompatible: Rework Instance: merge BaseInstance, Instance and LazyLoaderInstance into a single abstract Instance class. Remove templates argument from Instance. Rename Instance.init to Instance.set_db. Don't republish concrete framework instances in umongo top module. (see #314)
• Add session context manager to PyMongoInstance and MotorAsyncIOInstance. This allows to use session related features (causally consistent reads, transactions) from umongo. (see #315)

3.0.0b11 (2020-11-06)

Features:

• Backwards-incompatible: Allow setting arbitrary attributes on Document and EmbeddedDocument instances. This change is part of a refactor meant to simplify set / get / delete operations on document objets and (marginally) improve performance. (see #272)
• Use structured information provided with DuplicateKeyError rather than parse the error message string (see #309).
• Add replace argument to commit method to force writing the whole document rather than updating (see #310).

Other changes:

• Support Python 3.9 (see #311).
• Backwards-incompatible: Drop motor<2.0.0 support (see #312).
• Backwards-incompatible: Drop MongoDB<4.2 support (see #313).

3.0.0b10 (2020-10-12)

Features:

• Allow passing Document and EmbeddedDocument in queries. (see #303)

3.0.0b9 (2020-10-05)

Features:

• Add support for motor 2.2 (see #294). (Picked from 2.3.0.)
• Backwards-incompatible: Add ExposeMissing context manager to return missing rather than None when dumping. Replace FromUmongoSchema with RemoveMissingSchema. This schema removes missing fields when dumping by using ExposeMissing internally. Make this feature opt-in by requiring the user to specify RemoveMissingSchema as MA_BASE_SCHEMA_CLS. (see #261)
• Backwards-incompatible: Remove mongo_world parameter from Schema.as_marshmallow_schema. Schemas generated by this method are meant to (de)serialize umongo objects, not dict straight from database. (see #299)
• Backwards-incompatible: Remove umongo.Schema. Schemas should inherit from umongo.abstract.BaseSchema. Expose RemoveMissingSchema as umongo.RemoveMissingSchema. (see #301)

Other changes:

• Backwards-incompatible: Drop Python 3.6 support (see #298).

3.0.0b8 (2020-07-22)

Features:

• Let Document inherit from EmbeddedDocument (see #266).

• Add MixinDocument allowing Document and EmbeddedDocument to inherit fields and pre/post methods from mixin objects (see #278).

• Backwards-incompatible: Remove as_attribute argument of BaseInstance.register method. Documents can not be accessed by name as instance attributes anymore. (see #290)

Bug fixes:

• Fix passing None to a field with _required_validate method (see #289).

3.0.0b7 (2020-05-08)

Features:

• Backwards-incompatible: Revert broken feature introduced in 3.0.0b6 allowing to get fields from mixin classes (see #273).

• Backwards-incompatible: Remove allow_inheritance option. Any Document or EmbeddedDocument may be subclassed (see #270).

• Backwards-incompatible: Field raises DocumentDefinitionError rather than RuntimeError when passed missing kwarg and Document.commit raises NotCreatedError when passed conditions for a document that is not in database (see #275).

3.0.0b6 (2020-05-04)

Features:

• Backwards-incompatible: abstract in EmbeddedDocument behaves consistently with Document. The _cls / cls field is only added on concrete embedded documents subclassing concrete embedded documents. And EmbeddedField only accepts concrete embedded documents. (see #86)

• Document and EmbeddedDocument may inherits from mixin classes. The mixin class should appear first (leftmost) in the bases: class MyDocument(MyMixin, Document). (see #188)

Other changes:

• Backwards-incompatible: marshmallow imports throughout the code are done as import marshmallow as ma. For convenience, missing and ValidationError can still be imported as umongo.missing and umongo.ValidationError.

3.0.0b5 (2020-04-30)

Features:

• Backwards-incompatible: Add MA_BASE_SCHEMA_CLS class attribute to Document and EmbeddedDocument to specify a base class to use in as_marshmallow_schema. Drop the check_unknown_fields, params and meta attributes of as_marshmallow_schema. Make mongo_world kwarg-only. The same effect can be achieved using base schema classes. This incidentally fixes broken as_marshmallow_schema cache feature. (see #263)
• Backwards-incompatible: Add TxMongoDocument.find_with_cursor and drop support for upstream deprecated find(cursor=True). (see #259).

Other changes:

• Backwards-incompatible: Require txmongo>=19.2.0 (see #259).

3.0.0b4 (2020-04-27)

Features:

• Backwards-incompatible: Remove partial load feature (see #256).
• Backwards-incompatible: Add Document.pk_field and remove BaseDataProxy.*_by_mongo_name methods (see #257).
• Backwards-incompatible: Raise AlreadyCreatedError when modifying pk of created document (see #258).

3.0.0b3 (2020-04-26)

Features:

• Backwards-incompatible: Replace ReferenceError with NoneReferenceError. Review the list of exceptions importable from root umongo module. (see #251)

Bug fixes:

• Don't modify data when calling set_by_mongo_name on a field that was not loaded in a partial load. (see #253)

Other changes:

• Backwards-incompatible: Drop Python 3.5 support (see #248).

3.0.0b2 (2020-04-18)

Features:

• Use fields for keys/values in DictField (see #245).

Bug fixes:

• Fix BaseField.repr (see #247).

3.0.0b1 (2020-03-29)

Features:

• Support marshmallow 3 (see #154).
• All field parameters beginning with "marshmallow_" are passed to the marshmallow schema, rather than only a given list of known parameters. (see #228)

Other changes:

• Backwards-incompatible: Drop support for marshmallow 2. See marshmallow upgrading guide for a comprehensive list of changes. (see #154)
• Backwards-incompatible: StrictDateTimeField is removed as marshmallow now provides NaiveDateTimeField and AwareDateTimeField. (see #154)
• Backwards-incompatible: default shall now be provided in deserialized form. (see #154)

2.3.0 (2020-09-06)

Features:

• Add support for motor 2.2 (see #294).

2.2.0 (2019-12-18)

Bug fixes:

• Fix find/find_one: pass filter as first positional argument (see #215).

Other changes:

• Support Python 3.8 (see #210).

2.1.1 (2019-10-04)

Bug fixes:

• Fix ObjectId bonus field: catch TypeError when deserializing (see #207).

2.1.0 (2019-06-19)

Features:

• Add support for motor 2.+ by adding a count_documents class method to the MotorAsyncIODocument class. count_documents attempts to transparently use the correct motor call signature depending on which version of the driver is installed. Note that the behavior of the cursor object returned by MotorAsyncIODocument.find strictly adheres to the interface provided by the underlying driver.

2.0.5 (2019-06-13)

Bug fixes:

• Ensure Reference and GenericReference fields round-trip (see #200).

2.0.4 (2019-05-28)

Bug fixes:

• Include modified BaseDataObject in BaseDataProxy.get_modified_fields and BaseDataProxy.get_modified_fields_by_mongo_name (see #195).
• Always return a boolean in List.is_modified (see #195).
• List: call set_modified when deleting an element using the del builtin (see #195).

2.0.3 (2019-04-10)

Bug fixes:

• Fix millisecond overflow when milliseconds round to 1s in StrictDateTimeField (see #189).

2.0.2 (2019-04-10)

Bug fixes:

• Fix millisecond overflow when milliseconds round to 1s in DateTimeField and LocalDateTimeField (see #189).

2.0.1 (2019-03-25)

Bug fixes:

• Fix deserialization of EmbeddedDocument containing fields overriding _deserialize_from_mongo (see #186).

2.0.0 (2019-03-18)

Features:

• Backwards-incompatible: missing attribute is no longer used in umongo fields, only default is used. marshmallow_missing and marshmallow_default attribute can be used to overwrite the value to use in the pure marshmallow field returned by as_marshmallow_field method (see #36 and #107).
• Backwards-incompatible: as_marshmallow_field does not pass load_from, dump_to and attribute to the pure marshmallow field anymore. It only passes validate, required, allow_none, dump_only, load_only and error_messages, as well as default and missing values inferred from umongo's default. Parameters prefixed with marshmallow_ in the umongo field are passed to the pure marshmallow field and override their non-prefixed counterpart. (see #170)
• Backwards-incompatible: DictField and ListField don't default to empty Dict/List. To keep old behaviour, pass dict/list as default. (see #105)
• Backwards-incompatible: Serialize empty Dict/List as empty rather than missing (see #105).
• Round datetimes to millisecond precision in DateTimeField, LocalDateTimeField and StrictDateTimeField to keep consistency between object and database representation (see #172 and #175).
• Add DateField (see #178).

Bug fixes:

• Fix passing a default value to a DictField/ListField as a raw Python dict/list (see #78).
• The default parameter of a Field is deserialized and validated (see #174).

Other changes:

• Support Python 3.7 (see #181).
• Backwards-incompatible: Drop Python 3.4 support (see #176) and only use async/await coroutine style in asyncio framework (see #179).

1.2.0 (2019-02-08)

• Add Schema cache to as_marshmallow_schema (see #165).
• Add DecimalField. This field only works on MongoDB 3.4+. (see #162)

1.1.0 (2019-01-14)

• Fix bug when filtering by id in a Document subclass find query (see #145).
• Fix getattr to allow copying and deepcopying Document and EmbeddedDocument (see #157).
• Add Document.clone() method (see #158).

1.0.0 (2018-11-29)

• Raise UnknownFieldInDBError when an unknown field is found in database and not using BaseNonStrictDataProxy (see #121)
• Fix (non fatal) crash in garbage collector when using WrappedCursor with mongomock
• Depend on pymongo 3.7+ (see #149)
• Pass as_marshmallow_schema params to nested schemas. Since this change, every field's as_marshmallow_schema method should expect unknown **kwargs (see #101).
• Pass params to container field in ListField.as_marshmallow_schema (see #150)
• Add meta kwarg to as_marshmallow_schema to pass a dict of attributes for the schema's Meta class (see #151)

0.15.0 (2017-08-15)

• Add strict option to (Embedded)DocumentOpts to allow loading of document with unknown fields from mongo (see #115)
• Fix fields serialization/deserialization when allow_none is True (see #69)
• Fix ReferenceFild assignment from another ReferenceField (see #110)
• Fix deletion of field proxied by a property (see #109)
• Fix StrictDateTime bonus field: _deserialize does not accept datetime.datetime instances (see #106)
• Add force_reload param to Reference.fetch (see #96)

0.14.0 (2017-03-03)

• Fix bug in mashmallow tag handling (see #90)
• Fix allow none in DataProxy.set (see #89)
• Support motor 1.1 (see #87)

0.13.0 (2017-01-02)

• Fix deserialization error with nested EmbeddedDocuments (see #84, #67)
• Add abstract and allow_inheritance options to EmbeddedDocument
• Remove buggy as_marshmallow_schema's parameter missing_accessor (see #73, #74)

0.12.0 (2016-11-11)

• Replace Document.opts.children by offspring and fix grand child inheritance issue (see #66)
• Fix dependency since release of motor 1.0 with breaking API

0.11.0 (2016-11-02)

• data_objects Dict and List inherit builtins dict and list
• Document&EmbeddedDocument store fields passed during initialization as modified (see #50)
• Required field inside embedded document are handled correctly (see #61)
• Document support marshmallow's pre/post processors

0.10.0 (2016-09-29)

• Add pre/post update/insert/delete hooks (see #22)
• Provide Umongo to Marshmallow schema/field conversion with schema.as_marshmallow_schema() and field.as_marshmallow_field() (see #34)
• List and Dict inherit from collections's UserList and UserDict instead of builtins types (needed due to metaprogramming conflict otherwise)
• DeleteError and UpdateError returns the driver result object instead of the raw error dict (except for motor which only has raw error dict)

0.9.0 (2016-06-11)

• Queries can now be expressed with the document's fields name instead of the name in database
• EmbeddedDocument also need to be registered by and instance before use

0.8.1 (2016-05-19)

• Replace Document.created by is_created (see #14)

0.8.0 (2016-05-18)

• Heavy rewrite of the project, lost of API breakage
• Documents are now first defined as templates then implemented inside an Instance
• DALs has been replaced by frameworks implementations of Builder
• Fix __getitem__ for Pymongo.Cursor wrapper
• Add conditions argument to Document.commit
• Add count method to txmongo

0.7.8 (2016-4-28)

• Fix setup.py style preventing release of version 0.7.7

0.7.7 (2016-4-28)

• Fix await error with Reference.fetch
• Pymongo is now only installed with extra flavours of umongo

0.7.6 (2016-4-28)

• Use extras_require to install driver along with umongo

0.7.5 (2016-4-23)

• Fixing await (Python >= 3.5) support for motor-asyncio

0.7.4 (2016-4-21)

• Fix missing package in setup.py

0.7.3 (2016-4-21)

• Fix setup.py style preventing from release

0.7.2 (2016-4-21)

• Fix crash when generating indexes on EmbeddedDocument

0.7.1 (2016-4-21)

• Fix setup.py not to install tests package
• Pass status to Beta

0.7.0 (2016-4-21)

• Add i18n support
• Add MongoMock support
• Documentation has been a lot extended

0.6.1 (2016-4-13)

• Add <dal>_lazy_loader to configure Document's lazy_collection

0.6.0 (2016-4-12)

• Heavy improvements everywhere !

0.1.0 (2016-1-22)

• First release on PyPI.