xwebdb
Advanced tools
Readme
pronounced: Cross Web Database
Documentation work in progess
A browser-based NoSQL database backed and persisted by IndexedDB. |
Tailored for serverless applications, offline-first, seamlessly syncing when connected. |
Exceptionally fast, outperforming other databases with similar features in terms of performance. |
Lightweight, below 50KB (minified, 14KB gzipped) |
Mongo-like (MQL) query language (a subset, almost identical) |
Strongly-typed query language. Leveraging typescript for a very strict queries even on deep fields |
Live queries (reactive queries) are supported, i.e. can be used directly as a state-manager for react/angular/svelte/vue …etc. |
Built-in object modelling, query caching, data encoding and a lot more. |
| Feature | LocalForage | PouchDB | Dexie.js | XWebDB |
|---|---|---|---|---|
| Minified Size | 29KB | 142KB | 80KB | 48KB |
| Performance^ | good | good | good | fastest |
| Query Language | Key/value | Map/Reduce | Mongo-like | Mongo-like |
| Sync | no sync | CouchDB sync | paid/server | serverless services (free) |
| Live Queries | unsupported | unsupported | supported | supported |
| Aggregation | unsupported | unsupported | unsupported | supported |
XWebDB has a pretty good performance. It has the fastest insert, bulk-insert, update, delete, and read times even with large databases.
The performance of XWebDB can be attributed to:
However, it is important to note that achieving such high-performance requires maintaining a complete copy of the database in memory. While this approach may seem unconventional, it poses no significant issues for the intended use cases of this database, particularly given today's standards. The memory footprint for storing 10,000 2KB documents is nearly 20MB, which is considered manageable.
O(1)O(log n)O(log n)[!WARNING] While XWebDB may appear to be a promising choice for your next project compared to other databases, it is essential to carefully weigh your decision. Other solutions have undergone rigorous testing, have been battle-tested, and enjoy robust support from larger communities. This is not to discourage you from using XWebDB; in fact, I am currently using it in multiple projects myself. However, it's important to acknowledge that XWebDB is a relatively new project. With time, it is expected to mature and improve. I hope that in the future, this cautionary section can be removed from the documentation. Until then, it is advisable to thoroughly consider your options before making a final decision.
Install using npm
npm install xwebdb
Alternatively, you can include the pre-built and minified file in your HTML:
<script src="https://unpkg.com/xwebdb/dist/xwebdb.min.js"></script>
To create a database, you need to instantiate the Database class with a configuration object. The only required property is ref, which specifies the reference name of the database.
import { Database } from "xwebdb";
// Database creation and configuration
let db = new Database({
ref: "my-database", // Specify the reference name for the database
});
[!WARNING] The above example is oversimplified and shouldn't be used in a real-world application as it doesn't specify the model to be used in the database for object mapping.
For more advanced configurations, please refer to the Configuration section below.
You can define a document model by extending the Doc class and specifying its properties and methods:
import { Database, Doc } from "xwebdb";
// Define a document class
class Person extends Doc {
firstName: string = ""; // Define a firstName property
lastName: string = ""; // Define a lastName property
get fullName() {
return this.firstName + " " + this.lastName; // Define a computed property fullName
}
}
// Create a database with the specified model
let db = new Database<Person>({
ref: "my-database1",
model: Person, // Specify the document model
});
For more advanced object mapping, please refer to the Object mapping section below.
Once you have a database instance, you can perform various operations on it, such as creating, finding, updating, and deleting documents. Here are some examples:
// Creating a document
db.insert(Person.new({ firstName: "Ali" }));
// Create a new document with firstName "Ali"
// defining an index
db.createIndex({ fieldName: "firstName", unique: false, sparse: true });
// create an index on "firstName" field
// Finding documents
db.find({ firstName: "Ali" });
// Find documents with firstName "Ali"
// Using operators in queries
db.find({ firstName: { $eq: "Ali" } });
// Find documents with firstName equal to "Ali"
// Use aggregation method chaining
(await db.aggregate({ firstName: $eq }))
.$skip(1)
.$limit(5)
.$sort({ lastName: -1 })
.$addFields((doc) => ({ firstLetter: doc.firstName.charAt(0) }));
// Updating documents
db.update({ firstName: "Ali" }, { $set: { firstName: "Dina" } });
// Update firstName from "Ali" to "Dina"
// Deleting documents
db.delete({ firstName: { $eq: "Ali" } });
// Delete documents with firstName "Ali"
// reload database from persistence layer
db.reload();
// synchronizing database (an RSA must have been configured)
db.sync();
Those operations are explained extensively in their respective sections below (check: Inserting, Indexing, Reading, Counting, Aggregation, Updating, Upserting, Deleting, Reloading, and Synchronization).
By leveraging Live Queries, you can perform queries that not only return initial results but also establish a continuous connection to the queried data. This connection ensures that any changes made to the data in the database are automatically reflected in the query results in real-time, without the need for manual refreshing or re-querying. It also means that any changes made to the query resulting object will be persisted to the database.
// Perform a live query
let res1 = await db.live({ firstName: "Ali" });
// same results as above
let res2 = await db.live({ firstName: "Ali" });
// Get regular non-live result
let res3 = await db.find({ firstName: "Ali" });
res1[0].firstName = "Mario";
// Update the firstName property
// The above line updates the database and 'res1'
// it is equivalent to:
// db.update({ firstName: 'Ali' }, { $set: { firstName: 'Mario' } });
// 'res2' will have the updated value automatically
// 'res3' will retain the old value
// since it was obtained using 'find' instead of 'live'
db.insert(Model.new({ firstName: "Ali" }));
// when the operation above adds a new document
// the res1 & res2 will be updated automatically
// and will have the new document
// won't get automatic updates from DB
db.live({ firstName: "Dina" }, { fromDB: false });
// won't set automatic updates to DB
db.live({ firstName: "Dina" }, { toDB: false });
// killing:
// kill live query turning it to regular query
res1.kill();
// won't get automatic updates from DB anymore
res1.kill("fromDB");
// will not reflect changes to DB anymore
res1.kill("toDB");
With live queries, you can build dynamic applications that respond to data changes in real-time. Live queries enable you to use XWebDB directly as a state manager in your front-end framework (react, angular, vue, svelte, solid ...etc). This is discussed extensively in Live queries section.
import { Database, Doc } from "xwebdb";
// Model/Schema
class Person extends Doc {
firstName: string = "";
lastName: string = "";
get fullName() {
return this.firstName + " " + this.lastName;
}
}
// Database creation and configuration
let db = new Database<Person>({
ref: "my-database",
// Define a reference to be used as a database name for IndexedDB
model: Person,
// Define model for object mapping
timestampData: true,
// Include "createdAt" and "updatedAt" fields in documents
stripDefaults: true,
// Remove default values from the IndexedDB and remote database
corruptAlertThreshold: 0.2,
// Set tolerance level for data corruption
deferPersistence: 500,
// Resolve promises before persisting operations to IndexedDB
indexes: ["firstName"],
// Define non-unique indexes
cacheLimit: 1000,
// Set cache limit to avoid overwhelming memory
encode: (obj) => JSON.stringify(obj),
// Implement encryption for data persistence
decode: (str) => JSON.parse(str),
// Implement decryption for data retrieval
});
ref:string (Required, no default value)model:a class that extends Doc (Defaults to Doc)import { Doc } from "xwebdb";
class Person extends Doc {
firstName: string = "default name";
// Default value for 'firstName'
age: number = 25;
// Default value for 'age'
}
// Create a document using .new
Person.new({ firstName: "Ali" });
// The above returns a document
// with the default value for 'age'
// and "Ali" as the value for 'firstName'
timestampData:boolean (Defaults to false)stripDefaults:boolean (Defaults to false)corruptAlertThreshold:number (Defaults to 0)deferPersistence:false | number (Defaults to false)indexes:Array<string> (Defaults to an empty array)db.ensureIndex However, it offers less options. For example, the indexes created using this approach will not be unique by default. If you require unique indexes, you would need to recreate them using db.ensureIndex and explicitly define them as unique (check ensureIndex below for more information).cacheLimit:number (Defaults to 1000)encode:(input:string)=>string (Defaults to undefined)decode:(input:string)=>string (Defaults to undefined)import { Database } from "xwebdb";
function encrypt() {
/* encrpytion code */
}
function decrypt() {
/* decrpytion code */
}
let db = new Database({
ref: "database",
encode: (input: string) => encrpyt(input),
decode: (input: string) => decrypt(input),
});
Object mapping is mechanism by which you define a structure for your data using JavaScript classes.
import { Doc } from "xwebdb";
class Person extends Doc {
firstName: string = "";
lastName: string = "";
birth: number = 20;
// getters
get fullName() {
return this.firstName + " " + this.lastName;
}
get age() {
new Date().getFullYear() - this.birth;
}
// alias
name = fullname;
// helper method
setBirthByAge(age: number) {
this.birth = new Date().getFullYear() - age;
}
}
From the above example you can see the following advantages when defining your model:
The model class extends Doc, which is mandatory because:
_id field will be added automatically. XWebDB, by default uses UUID generator that is even faster than the native crypto.randomUUID().stripDefaults options is set to true on database instantiation.Having your model as a class allows for more creativity and flexibility, the following example implements a basic level of hierarchy in model definition, since two models share similar type of values:
import { Doc } from "xwebdb";
class Person extends Doc {
// overwrites the default _id generator
_id: string = crypto.randomUUID();
firstName: string = "";
lastName: string = "";
get fullName() {
return this.firstName + " " + this.lastName;
}
}
class Doctor extends Person {
speciality: string = "";
}
class Patient extends Person {
illness: string = "";
}
let doctorsDB = new Database<Doctor>({
model: Doctor,
ref: "doctors",
});
let patientsDB = new Database<Patient>({
model: Patient,
ref: "patients",
});
You can explore more advanced concepts such as OOP, modularity, dependency injection, decorators, mixins, and more.
Submodels (Child models/sub-documents) are also supported in object mapping using SubDoc class and mapSubModel function.
import { Doc, SubDoc, mapSubModel } from "xwebdb";
/**
* Toy is a sub-document of a sub-document of a document
* Sub document definintion must extend "SubDoc"
*/
class Toy extends SubDoc {
name: string = "";
price: number = 0;
get priceInUSD() {
return this.price * 1.4;
}
}
/**
* Child is a sub-document of a document
* Sub document definintion must extend "SubDoc"
*/
class Child extends SubDoc {
name: string;
age: number = 0;
toys: Toy[] = mapSubModel(Toy, []);
favoriteToy: Toy = mapSubModel(Toy, Toy.new({}));
get numberOfToys() {
return this.toys.length;
}
}
class Parent extends Doc {
name: string = "";
age: number = 9;
male: boolean = false;
mainChild: Child = mapSubModel(Child, Child.new({}));
children: Child[] = mapSubModel(Child, []);
get female() {
return !this.male;
}
}
From the above example you can see that mapSubModel takes two arguments:
When trying to insert/create a new document use the .new() method.
db.insert(Parent.new());
// inserts a new "Parent" document.
// fields/properties of the document will all be the default values.
// to define properties other than the defaults
// you can pass them as a plain JS object.
db.insert(
Parent.new({
name: "Ali",
age: 31,
male: true,
mainChild: Child.new({
name: "Kiko",
}),
// properties that are not
// mentioned in this object
// will be the defaults defined
// in the class above
})
);
// Note that the .new() method
// doesn't actually insert a new document.
// it merely returns a document in preparation for insertion.
When persisting data, only the actual fields (neither getters nor methods) will be persisted. Using the stripDefaults option on database instantiation will also remove the default values from the persisted data (StripDefaults).
class Child extends Doc {
age: number = 9;
oldToys: Toy[] = mapSubModel(Toy, []);
newToys: Toy[] = mapSubModel(Toy, []);
get numberOfToys() {
return this.oldToys.length + this.newToys.length;
}
get toyEachYear() {
return this.numberOfToys / this.age > 1;
}
}
let childrenDB = new Database<Child>({ ref: "children", model: Child });
// simple query
childrenDB.find({ toyEachYear: { $gt: 2 } });
// if you wouldn't use the computed property
// your query will be very complex
// having to use many operators
// like: $or, $size, $gt and maybe even more.
// all fields have default values
db.insert(Parent.new());
// all fields have default values except 'age'
db.insert(Parent.new({ age: 30 }));
$setOnInsert (more on upserting in the examples below).To insert documents in the database use the insert method (or alias: create), which can either take single document or an array of documents. Remember to always use the Model.new() when inserting document.
import { Database, Doc } from "xwebdb";
class Person extends Doc {
name: string = "";
age: number = 0;
}
let db = new Database<Person>({ ref: "database", model: Person });
// insert an empty document (with default values)
db.insert(Person.new({}));
// insert a document with fields
db.insert(
Person.new({
name: "ali",
age: 12,
})
);
db.insert(
Person.new({
name: "ali",
// age field will have the default value
})
);
// inserting multiple documents at once
db.insert([Person.new({ name: "ali" }), Person.new({ name: "dina" })]);
The insert method will return a promise that resolves to the number of inserted documents and an array of the inserted documents. The last line of the above example will return a promise that resolves to the following:
{
"number": 2,
"docs": [
{
"_id": "ad9436a8-ef8f-4f4c-b051-aa7c7d26a20e",
"name": "ali",
"age": 0
},
{
"_id": "38ae1bbd-60a7-4980-bbe9-fce3ffaec51c",
"name": "dina",
"age": 0
}
]
}
Reading from the database using read (alias: find):
db.read({ name: "ali" });
db.find({ name: "ali" }); // alias
The read (or find) method takes two arguments:
skip, limit, sort, and project the matched documents.Here's a more elaborate example:
db.read(
{ name: "ali" }, // query
{
skip: 2, // skip the first two matches
limit: 10, // limit matches to 10 results
project: {
// pick-type projection
// result will only have
// the following props
name: 1,
age: 1,
// omit-type projection
// result will not have
// the following props
address: 0,
email: 0,
// NOTE: you can either use
// pick-type or omit-type projection
// except for _id which is by default
// always returned and which you can choose to omit
},
sort: {
name: 1, // ascending sort by name
age: -1, // descending sort by age
// single or multiple sort elements
// can be specified at the same time
},
}
);
The query API (first argument of read) closely resembles MongoDB MQL. You can query documents based on field equality or utilize a range of comparison operators such as $lt, $lte, $gt, $gte, $in, $nin, $ne, and $eq. Additionally, logical operators like $or, $and, $not, and $where are available for more complex querying capabilities.
{name:"Ali"}{age:{$gt:10}}{$and:[{age:10},{name:"Ali"}].To specify equality conditions in a query filter document, you can use { <FieldName> : <Value> } expressions. This allows you to find documents that match specific field values. Here are some examples:
// Select all documents where the name is "ali"
db.find({ name: "ali" });
// Select all documents where the age is exactly 27
// and the height is exactly 180
db.find({
age: 27,
height: 180,
});
In these examples, the filter field is used to specify the equality conditions for the query. You can provide multiple field-value pairs to further refine the query.
However, like MongoDB, when dealing with deeply nested objects, simple field-level equality may not work as expected. Consider the following example:
// Suppose you have the following document:
{
item: "Box",
dimensions: {
height: 30,
width: 20,
weight: 100
}
}
// The following queries won't match:
db.find({
dimensions: { height: 30 }
});
db.find({
dimensions: { height: 30, width: 20 }
});
// The following query will match:
db.find({
dimensions: { height: 30, width: 20, weight: 100 }
});
In the case of deeply nested objects, using field-level equality alone will not work. To query deeply nested documents, you need to use the $deep operator. The $deep operator allows you to specify nested fields and their values in a query. More information about the $deep operator can be found below.
Syntax: { <fieldName>: { <operator>: <specification> } }
$eqSpecifies equality condition. The $eq operator matches documents where the value of a field equals the specified value. It is equivalent to { <FieldName> : <Value> }.
Any field type{ <fieldName> : { $eq: <value> } }// Example
db.find({ name: { $eq: "ali" } });
// same as:
db.find({ name: "ali" });
$ne$ne selects the documents where the value of the field is not equal to the specified value. This includes documents that do not contain the field.
Any field type{ <fieldName> : { $ne: <value> } }// selecting all documents where "name"
// does not equal "ali"
db.find({ name: { $ne: "ali" } });
$gtselects those documents where the value of the field is greater than (i.e. >) the specified value.
number & Date fields{ <fieldName> : { $gt: <value> } }// applied on a number field
db.find({ year: { $gt: 9 } });
// applied on a date field
db.find({
createdAt: { $gt: new Date(1588134729462) },
});
$ltselects those documents where the value of the field is less than (i.e. <) the specified value.
number & Date fields{ <fieldName> : { $lt: <value> } }// applied on a number field
db.find({ year: { $lt: 9 } });
// applied on a date field
db.find({
createdAt: { $lt: new Date(1588134729462) },
});
$gteselects those documents where the value of the field is greater than or equal to (i.e. >=) the specified value.
number & Date fields{ <fieldName> : { $gte: <value> } }// applied on a number field
db.find({ year: { $gte: 9 } });
// applied on a date field
db.find({
createdAt: { $gte: new Date(1588134729462) },
});
$lteselects those documents where the value of the field is less than or equal to (i.e. <=) the specified value.
number & Date fields{ <fieldName> : { $lte: <value> } }// applied on a number field
db.find({ year: { $lte: 9 } });
// applied on a date field
db.find({
createdAt: { $lte: new Date(1588134729462) },
});
$inThe $in operator selects the documents where the value of a field equals any value in the specified array.
Any field type{ <fieldName> : { $in: [<value1>, <value2>, ... etc] } }// find documents where the "name"
// field is one of the specified
// in the array
db.find({ name: { $in: ["ali", "john", "dina"] } });
$ninThe $nin operator (opposite of $in) selects the documents where the value of a field doesn't equals any value in the specified array.
Any field type{ <fieldName> : { $nin: [<value1>, <value2>, ... etc] } }// find documents where the "name"
// field is one of the specified
// in the array
db.find({ name: { $nin: ["ali", "john", "dina"] } });
$existsWhen <boolean> is passed and is true, $exists matches the documents that contain the field, including documents where the field value is null. If <boolean> is false, the query returns only the documents that do not contain the field.
Any field type{ <fieldName> : { $exists: <boolean> } }// select documents where the "name"
// field is defined, even if it is null
db.find({ name: { $exists: true } });
// select documents where the "name"
// field is not defined
db.find({ name: { $exists: false } });
$type$type selects documents where the value of the field is an instance of the specified type. Type specification can be one of the following:
"string""number""boolean""undefined""array""null""date""object"Any field type{ <fieldName> : { $type: <spec> } }// find documents where the "name" field
// is a string.
db.find({ name: { $type: "string" } });
$modSelect documents where the value of a field divided by a divisor has the specified remainder (i.e. perform a modulo operation to select documents).
number & Date fields{ <fieldName> : { $mod: [divisor, remainder] } }// select documents where the "years" field
// is an even number
db.find({
years: {
$mod: [2, 0],
},
});
// select documents where the "years" field
// is an odd number
db.find({
years: {
$mod: [2, 1],
},
});
$regexSelects documents which tests true for a given regular expression.
string fields{ <fieldName> : { $regex: <RegExp> } }// select documents where the "name"
// field starts with either "a" or "A".
db.find({ name: { $regex: /^a/i } });
$allThe $all operator selects the documents where the value of a field is an array that contains all the specified elements.
array fields{ <fieldName> : { $all: [<value1>, <value2>,...etc] } }// select documents where the "tags" field
// is an array that has "music" & "art"
db.find({ tags: { $all: ["music", "art"] } });
$elemMatchThe $elemMatch operator matches documents that contain an array field with at least one element that matches all the specified query criteria.
array fields{{<fieldName>:{$elemMatch:{<query1>,<query2>,...etc}}}// select documents where the "price" field
// is an array field that has an element
// matching the following criteria
// has an even number
// less than 8
// and greater than 0
db.find({
price: {
$elemMatch: {
$mod: [2, 0],
$lt: 8,
$gt: 0,
},
},
});
$sizeThe $size operator matches any array with the number of elements (length of the array) specified by the argument.
array fields{ <fieldName> : { $size: number } }// select documents where the "tags"
// field is an array that has 10 elements.
db.find({ tags: { $size: 10 } });
The array fields has the operators $all, $elemMatch and $size specific for them. However, all the operators mentioned earlier can also be applied to arrays, and they will return true if any element in the array matches the specified condition.
Here is a summary of how the operators work when applied to arrays:
$eq: Matches an array if it contains an element equal to the value specified by the operator.$ne: Matches an array if it contains an element different from the value specified by the operator.$gt: Matches an array if it contains a number greater than the value specified by the operator.$lt: Matches an array if it contains a number less than the value specified by the operator.$gte: Matches an array if it contains a number greater than or equal to the value specified by the operator.$lte: Matches an array if it contains a number less than or equal to the value specified by the operator.$in: Matches an array if it contains any of the values specified by the operator.$nin: Matches an array if it contains none of the values specified by the operator.$mod: Matches an array if it contains a number that, when divided by the divisor specified by the operator, yields the remainder specified by the operator.$regex: Matches an array if it contains a string that matches the regular expression specified by the operator.$exists: Matches any given array.$type: Matches an array if the array itself is of the type "array" as specified by the operator.These operators provide flexibility for querying and filtering arrays based on various conditions.
All the above operators can be negated using the $not operator.
// find all documents
// where they have "tags" that is not of length 10
db.find({ tags: { $not: { $size: 10 } } });
// similar to $ne
db.find({ name: { $not: { $eq: "ali" } } });
// find documents where the "name" field
// is a not a string
db.find({ name: { $not: { $type: "string" } } });
// select documents where the "tags"
// field is an array that doesn't have "music" & "art"
db.find({ tags: { $not: { $all: ["music", "art"] } } });
// select documents where the "years" field
// is an even number
db.find({
years: {
$not: {
$mod: [2, 1],
},
},
});
// ...etc
$and$and performs a logical AND operation on an array of two or more expressions (e.g. <field level query 1>, <field level query 2> , etc.) and selects the documents that satisfy all the expressions in the array. The $and operator uses short-circuit evaluation. If the first expression (e.g. <field level query 1>) evaluates to false, XWebDB will not evaluate the remaining expressions.
Syntax
{
$and: [
<query1>,
<query2>,
<query3>,
...etc
]
}
/**
* Select a document where the name
* isn't equal to "ali" and the property exists
*/
db.find({
$and: [{ $name: { $ne: "ali" } }, { $name: { $exists: true } }],
});
$nor$nor performs a logical NOR operation on an array of one or more query expression and selects the documents that fail all the query expressions in the array.
Syntax
{
$nor: [
<query1>,
<query2>,
<query3>,
...etc
]
}
/**
* Select a document where the "name" is not "alex"
* and the age is not 13
*/
db.find({
$nor: [{ $name: "alex" }, { $age: 13 }],
});
$orThe $or operator performs a logical OR operation on an array of two or more expressions and selects the documents that satisfy at least one of the expressions.
Syntax
{
$or: [
<query1>,
<query2>,
<query3>,
...etc
]
}
/**
* Select a document where the "name" is not "ali"
* or the age is not 13
*/
db.find({
$or: [{ name: "ali" }, { $age: 13 }],
});
$whereMatches the documents that when evaluated by the given function would return true.
[!WARNING] The
$whereprovides greatest flexibility, but requires that the database processes the JavaScript expression or function for each document in the collection. It's highly advisable to avoid using the$whereoperator and instead use indexed getters as explained in the object mapping section of this documentation.
Syntax
{
$where: (this: Model) => boolean;
}
/**
* Select a document where the "name"
* is 5 characters long and ends with "x"
*/
db.find({
$where: function () {
// DO NOT use arrow function here
return this.name.length === 5 && this.name.endsWith("x");
},
});
$deepThe $deep operator is the only operator in XWebDB that doesn't exist in MongoDB. It has been introduced as an alternative to the dot notation to match deep fields in sub-documents.
Take the following document for example:
{
item: "box",
dimensions: {
height: 100,
width: 50
}
}
The following queries will behave as follows:
db.find({ dimensions: { height: 100 } });
// will not match, because field-level literal equality
// requires the query object to exactly equal the document object
db.find({ $deep: { dimensions: { height: 100 } } });
// the above query will match the document
The reason that the $deep operator has been added is to keep strict typing even when querying deeply nested objects. Since it is not possible (in typescript) to define strict typings for the dot notation.
Syntax
{
$deep: <query>
}
Basic example:
// document
{
item: "box",
dimensions: {
height: 100,
width: 50
}
}
db.find({ $deep: { dimensions: { height: 100 } } });
You can specify multiple deep fields:
//documents
{
name: {
first: "ali",
last: "saleem",
}
}
{
name: {
first: "john",
last: "cena",
}
}
db.find({
$deep: {
name: {
first: { $in: ["ali", "john"] },
last: { $in: ["saleem", "cena"] },
},
}
});
You can use the $deep operator even in array elements by defining their index:
// document:
{
name: "ali",
children: [
{
name: "keko",
age: 2,
},
{
name: "kika",
age: 1,
}
]
}
db.find({
$deep: {
children: {
0: {
age: { $gt : 1 }
}
}
}
})
To count the number of documents matching a specific query use the count method.
Count method always returns a promise that resolves to a number.
// count number of documents
// that has "ali" in the "name" field
db.count({ name: "ali" });
// count number of documents
// that has age greater than 20
db.count({ age: { $gt: 20 } });
The query API in the count method is the same as the query API in read method, which in turn very similar to MongoDB query language.
Aggregation is process of combining multiple rows of data into a single value or summary. It involves performing mathematical or statistical operations on a set of data to generate meaningful insights or summaries. Aggregation is commonly used to calculate various metrics, such as sums, averages, counts, maximums, minimums, or other statistical calculations, over a group of rows.
Aggregation In XWebDB uses the method chaining syntax, and the following aggregation methods are supported: $sort, $limit, $skip, $project, $match, $addFields, $group, $unwind
let aggregate = await db.aggregate(
//optional starting query
{
name: "ali",
}
);
aggregate
.$sort({
age: -1, // sorting descending
city: 1, // sorting ascending
})
.$skip(1)
.$limit(50)
.$project({ children: 1 })
.$addFields((doc) => ({
// adds fields to each document
// based on calculation done on it
numberOfChildren: doc.children.length,
}))
// filter the aggregate based on a specific query
// this can take the same syntax of `db.find`
.$match({ numberOfChildren: { $gt: 1 } })
// deconstruct an array field within a document
// creating a new document for each element in the array
.$unwind("children")
// adds a field
.$addFields((doc) => ({ numberOfBrothers: doc.numberOfChildren - 1 }))
// group documents together based on a specified key
// and perform various transformations on the grouped data
// using a reducer function
.$group({
_id: "children",
reducer: (group) => ({
kidsNamed: group[0].children
count: group.length
}),
});
You can use the aggregation methods in any order and as much as you want, until you get the target meaningful data.
The current implementation of aggregation in XWebDB is a starting point, especially the $group method.
Planned updates on next versions:
$sum, $avg, $max, and $min)To update documents matching a specific query use the update method.
db.update(
// first argument is the find query
// matching target documents
// to be updated
{
name: "ali",
},
// second argument is the update
// must be supplied with one of the update operators
{
$set: {
age: 32,
},
},
// third argument is a boolean
// whether to update multiple documents
// or to update the first match only
// defaults to "false"
// i.e. update first match only
false
);
The first argument of the update method takes a query with the same syntax as the find & count methods as explained above. The second argument must be supplied with the update operators.
[!INFO] Although it is possible in MongoDB to use the direct field updates (no operator,
{age:5}), this is not supported in XWebDB to enforce a more strict type declaration.
The insert method will return a promise that resolves to the number of updated documents, a boolean of whether the update was an upsert or not, and an array of the updated documents. the following is an example:
{
"number": 2,
"upsert": false,
"docs": [
{
"_id": "ad9436a8-ef8f-4f4c-b051-aa7c7d26a20e",
"name": "ali",
"age": 0
},
{
"_id": "38ae1bbd-60a7-4980-bbe9-fce3ffaec51c",
"name": "dina",
"age": 0
}
]
}
The update operators are:
$setSets the value of a field in a document to a specified value.
any field type{
$set: {
<fieldName1>: <value1>,
<fieldName2>: <value2>,
...etc
}
}
/**
* update the name "ali" to "dina"
*/
db.update(
{
name: "ali",
},
{
$set: {
name: "dina",
},
}
);
$unsetSets the value of a field in a document to undefined.
any field type{
$unset: {
<fieldName1>: "",
<fieldName2>: "",
...etc
}
}
/**
* setting the name "ali" to undefined
*/
db.update(
{
name: "ali",
},
{
$unset: {
name: "",
},
}
);
$currentDateSets the value of a field in a document to a Date object or a timestamp number representing the current date.
Date & number fields{
$currentDate: {
// date object (applies to Date fields)
<fieldName1>: true,
// date object (alternative syntax)
<fieldName2>: { $type: "date" },
// timestamp (applies to number fields)
<fieldName3>: { $type: "timestamp" },
...etc
}
}
/**
* setting the name "ali" to undefined
*/
db.update(
{
name: "ali",
},
{
$currentDate: {
lastLogin: true,
lastLoginTimestamp: {
$type: "timestamp",
},
},
}
);
$renameRenames a property name to a different one while keeping the value the same.
any field type{
$rename: {
<fieldName1>: <newName1>,
<fieldName2>: <newName2>,
...etc
}
}
/**
* setting the property name "name" to "firstName"
*/
db.update(
{
name: "ali",
},
{
$rename: {
name: "firstName",
},
}
);
$incincrements the value of a field in a document to by a specific value.
number fields{
$inc: {
<fieldName1>: <number>,
<fieldName2>: <number>,
...etc
}
}
/**
* increment the age field by 2, and the months by 24
*/
db.update(
{
name: "ali",
},
{
$inc: {
age: 2,
months: 24,
},
}
);
You can also pass a negative value to decrement
/**
* decrement the age field by 2, and the months by 24
*/
db.update(
{
name: "ali",
},
{
$inc: {
age: -2,
months: -24,
},
}
);
$mulmultiplies the value of a field in a document to by a specific value.
number fields{
$mul: {
<fieldName1>: <number>,
<fieldName2>: <number>,
...etc
}
}
/**
* multiplies the age field by 2, and the months by 24
*/
db.update(
{
name: "ali",
},
{
$mul: {
age: 2,
months: 24,
},
}
);
You can also use the $mul operator to do a division mathematical operation:
/**
* divides the age field by 2, and the months by 24
*/
db.update(
{
name: "ali",
},
{
$mul: {
age: 1 / 2,
months: 1 / 24,
},
}
);
$maxUpdates the field value to a new value only if the specified value is greater than the existing value.
number & Date fields{
$max: {
<fieldName1>: <number|Date>,
<fieldName2>: <number|Date>,
...etc
}
}
/**
* sets the age field to 10 if it's less than 10
*/
db.update(
{
name: "ali",
},
{
$max: {
age: 10,
},
}
);
$minUpdates the field value to a new value only if the specified value is less than the existing value.
number & Date fields{
$min: {
<fieldName1>: <number|Date>,
<fieldName2>: <number|Date>,
...etc
}
}
/**
* sets the age field to 10 if it's greater than 10
*/
db.update(
{
name: "ali",
},
{
$min: {
age: 10,
},
}
);
$addToSetAdds elements to an array only if they do not already exist in it.
Array fields{
$addToSet: {
// adds a single value
<fieldName1>: <value>,
// adds multiple values
<fieldName2>: {
$each: [
<value1>,
<value2>,
<value3>,
... etc
]
}
}
}
/**
* Update a document where "name" is "ali"
* by adding the skill "javascript"
* and adding two projects
*/
db.update(
{ name: "ali" },
{
$addToSet: {
skills: "javascript",
projects: {
$each: ["projectA.js", "projectB.js"],
},
},
}
);
$popremoves the first or last element of an array. Pass $pop a value of -1 to remove the first element of an array and 1 to remove the last element in an array.
Array fields{
$pop: {
// removes first element
<fieldName1>: -1,
// removes last element
<fieldName2>: 1
}
}
/**
* Update a document where name is "ali"
* removing last element of "skills"
* and removes first element of "projects"
*/
db.update(
{ name: "ali" },
{
$pop: {
skills: 1,
projects: -1,
},
}
);
$pullRemoves all array elements that match a specified query or value.
Array fields{
$pull: {
// remove by value
<fieldName1>: <value>,
// or remove by query
<fieldName2>: <query>
}
}
/**
* Update a document where name is "ali"
* removing skill "javascript"
* and removing any project that ends with .js
*/
db.update(
{ name: "ali" },
{
$pop: {
projects: {
$regex: /\.js$/i,
},
skills: "javascript",
},
}
);
$pullAllThe $pullAll operator removes all instances of the specified values from an existing array.
Unlike the $pull operator that removes elements by specifying a query, $pullAll removes elements that match the listed values.
Array fields{
$pullAll: {
<fieldName1>: [<value1>, <value2> ... etc],
<fieldName2>: [<value1>, <value2> ... etc],
... etc
}
}
/**
* Update a document where name is "ali"
* removing skills: "javascript" and "php"
* and removes "new", "x-client"
*/
db.update(
{ name: "ali" },
{
$pullAll: {
skills: ["php", "javascript"],
},
}
);
$pushThe $push operator appends a specified value to an array.
Array fields{
$push: {
<fieldName1>: <value>,
// or
<fieldName2>: {
// multiple fields
$each: [<value1>, <value2>, ...etc]
// with modifiers
// discussed below
$slice: <number>,
$position: <number>,
$sort: <sort-specification>
},
... etc
}
}
/**
* Update a document where name is "ali"
* removing skills: "javascript" and "php"
* and removes "new", "x-client"
*/
db.update(
{ name: "ali" },
{
$push: {
skills: "javascript",
projects: {
$each: ["projectA.js", "projectB.js"],
},
},
}
);
$push with $each modifierModifies the $push operators to append multiple items for array updates. It can also be used with $addToSet operator.
db.update(
{ name: "ali" },
{
$push: {
projects: {
$each: ["projectA.js", "projectB.js"],
},
},
}
);
$push with $position modifier$position modifier must be supplied with a number. It specifies the location in the array at which the $push operator insert elements. Without the $positionmodifier, the$push operator inserts elements to the end of the array.
Must be used with $each modifier.
db.update(
{ name: "ali" },
{
$push: {
projects: {
$each: ["projectA.js", "projectB.js"],
$position: 1,
// the array items above will be pushed
// starting from position 1
},
},
}
);
If the $position modifier is supplied with a number larger than the length of the array the items will be pushed at the end without leaving empty or null slots.
$push with $slice modifierModifies the $push operator to limit the size of updated arrays.
Must be used with $each modifier. Otherwise it will throw. You can pass an empty array ([ ]) to the $each modifier such that only the $slice modifier has an effect.
/**
* Update a document where name is "ali"
* by appending "a" & "b" to "tags"
* unless the size of the array goes
* over 6 elements
*/
db.update(
{ name: "ali" },
{
$push: {
tags: {
$each: ["a", "b"],
$slice: 6,
},
},
}
);
$push with $sort modifierhe $sort modifier orders the elements of an array during a $push operation. Pass 1 to sort ascending and -1 to sort descending.
Must be used with $each modifier. Otherwise it will throw. You can pass an empty array ([]) to the $each modifier such that only the $sort modifier has an effect.
Here's an elaborate example explaining all the modifiers that can be used with $push operator:
/**
* Update a document where name is "john"
* by appending a new child to "children"
* at the start of the array
* then sorting the children by
* the "age" / ascending
* and the "name" / descending
* then allow only 10 children in the array
* by removing the last elements that goes
* over 10
*/
db.update(
{ name: "john" },
{
$push: {
children: {
$each: [{ name: "tim", age: 3 }],
$position: 0,
$sort: {
age: 1,
name: -1,
},
$slice: 10,
},
},
}
);
[!INFO] You can use the
$sliceand$sortmodifiers with empty$eacharray, so they have an effect without actually pushing items to the array, i.e. only sorting (for$sortmodifier) or only slicing (for$slicemodifier).
Upserting is a combination of updating & inserting. If the document you want to update already exists, it will be updated with new information. But if the document doesn't exist, a completely new document will be created with the specified information.
To do upsert operations use the upsert method. The upsert method has the same API as the update method but requires the $setOnInsert operator.
In short, The upsert method behaves exactly the same as the update method, with one exception: if no target document matched the query in the first argument, a new document will be created and the contents of the new document will be based on $setOnInsert operator.
import { Database, Doc } from "xwebdb";
class Person extends Doc {
name: string = "";
age: number = 0;
}
let db = new Database<Person>({
ref: "database",
model: Person,
});
db.upsert(
// for documents matching the following query:
{
name: "ali",
},
{
// update the name to "dina"
$set: {
name: "dina",
},
// if no documents matches
// create a new document with the name "dina"
$setOnInsert: Person.new({
name: "dina",
}),
}
);
The upsert method will return a promise that resolves to the number of updated (or inserted) documents, a boolean of whether the update was an upsert or not, and an array of the updated (or inserted) documents. the following is an example:
{
"number": 1,
"upsert": true,
"docs": [
{
"_id": "ad9436a8-ef8f-4f4c-b051-aa7c7d26a20e",
"name": "ali",
"age": 0
}
]
}
To delete documents matching a specific query use the delete (alias: remove) method.
The delete method takes two arguments, the first one is the query that the target documents to be deleted needs to match, and the second one is a boolean of whether to delete multiple documents or the first one matching the query.
// delete a document with "name" field that equals "ali"
db.delete(
// query
{
name: {
$eq: "ali",
},
},
// multi-delete:
// true: delete all documents matching
// false: delete the first match only
false // default: false
);
// delete all documents with "age" field greater than 45
db.delete(
// query
{
age: {
$gte: 45,
},
},
// multi-delete:
// true: delete all documents matching
// false: delete the first match only
true // default: false
);
Database indexing is a powerful technique that optimizes the performance of database queries by creating a separate data structure known as an index. When an index is created for a specific field, it acts as a reference point for the database engine, enabling fast retrieval of data when querying that field. By eliminating the need for full table scans, indexing significantly improves search and retrieval operations, especially when dealing with large amounts of data. However, it's important to note that indexing does introduce additional overhead during write operations. Therefore, thoughtful consideration should be given to the selection and design of indexes to strike a balance between read and write performance.
In short, creating an index for a specific field is beneficial when you expect that field to be frequently used in queries. XWebDB provides complete support for indexes on any field in the database. By default, all databases have an index on the _id field, and you may add additional indexes to support important queries and operations.
To create an index use the method createIndex, It takes an object as an argument with the following properties:
fieldName (required): Specifies the name of the field that you want to index. You can use dot notation to index a field within a nested document.
unique (optional, defaults to false): This option enforces field uniqueness. When a unique index is created, attempting to index two documents with the same field value will result in an error.
sparse (optional, defaults to false): When set to true, this option ensures that documents without a defined value for the indexed field are not indexed. This can be useful in scenarios where you want to allow multiple documents without the field, while still enforcing uniqueness.
db.createIndex({
// field name to be indexed
fieldName: "email",
// enforce uniqueness for this field
unique: true,
// don't index documents with 'undefined' value for this field
// when this option is set to "true"
// even if the index unique, multiple documents that has
// the field value as "undefined" can be inserted in the
// database and not cause a rejection.
sparse: true,
});
db.createIndex({
// index on a field that is in a sub-document
fieldName: "address.city",
unique: false,
sparse: true,
});
To remove an index, use the removeIndex method
db.removeIndex("email");
The createIndex and removeIndex methods return a promise that resolves to an object specifying the affected index:
{
"affectedIndex": "email"
}
To create indexes on database initialization use the indexes configuration parameter:
import { Database, Doc } from "xwebdb";
class Person extends Doc {
name: string = "";
email: string = "";
age: number = 0;
}
let db = new Database<Person>({
ref: "myDB",
model: Person,
indexes: ["email"],
});
However, the indexes created on the database initialization are non-unique, to make them unique you have to recreate them using the createIndex method.
[!NOTE]
createIndexcan be called when you want, even after some data was inserted, though it's best to call it at application startup.
When you initialize a database using new Database, the existing documents stored in the persistence layer (i.e. IndexedDB) will be read asynchronously. The database object will have loaded property, which represents a promise. This promise will be resolved once all the previous documents have finished loading. In other words, you can use the load property to determine when the database has finished loading the existing data and is ready for use.
import { Database, Doc } from "xwebdb";
class Person extends Doc {
name: string = "";
email: string = "";
age: number = 0;
}
let db = new Database<Person>({
ref: "myDB",
model: Person,
indexes: ["email"],
});
db.loaded.then(() => {
// do whatever
});
However, all database operations (such as inserting, reading, updating, and deleting) will wait for database initial loading if it hasn't occurred. So you can safely perform such operations without checking the loaded property.
If for any reason you want to reload the database from IndexedDB you can use the reload method:
db.reload();
Reasons that you may want to reload a database:
XWebDB synchronization empowers seamless data replication and consistency across multiple instances, following a multi-master model. Synchronization can be backed by various remote cloud databases like Cloudflare KV, S3 cloud file system, CosmosSB, DynamoDB, and more using synchronization adapters. Additionally you can write your own synchronization adapter to add support any remote Cloud database.
Synchronization protocol in XWebDB utilizes a unique revision ID methodology, ensuring a reliable and conflict-aware data replication. Each instance generates a revision ID for every data modification, enabling effective comparison during synchronization.
Conflict resolution in XWebDB follows the "latest-win" principle, prioritizing the most recent modification to automatically resolve conflicts. While this is an over-simplification of the real-world conflicts, its sufficient for the expected use-cases, avoiding needless complexity and manual intervention.
To synchronize data, the database must be instantiated with a sync adapter:
import { Database } from "xwebdb";
import { kvAdapter } from "xwebdb-kvadapter";
const db = new Database({
sync: {
// define remote sync adapter
syncToRemote: kvAdapter("YOUR_ENDPOINT", "YOUR_TOKEN"),
// define an interval at which the database will
// automatically sync with the remote database
// defaults to "0" (will not sync on interval) only manually
syncInterval: 500,
},
/// rest of database configuration
/// ref: ...
/// model: ...
/// ...etc: ...
});
When a database is configured for synchronization, you can use the sync method to manually synchronize the database with remote database. The method will wait for any database operation, or sync operation to complete before starting the sync progress.
db.sync();
However, if you want a more forceful synchronization (i.e. doesn't wait for other operation or checkpoints matching) you can use the method forceSync.
db.forceSync();
Both methods will return a promise that resolves to an object indicating the number of documents sent, number of documents received, and whether the sync found a difference or not.
{
// number of documents sent
"sent": 2,
// number of documents received
"received": 3,
// a number indicating whether a sync operation
// found a difference or not
// 1: documents have been sent or received
// -1: remote and local databases have the same checkpoint
// i.e. no further checks or comparisons were made
// 0: checkpoints were unequal for the local and remote DB
// but the documents were the same
// when this occurs checkpoints will be automatically unified
"diff": 1
}
Finally, to check whether a sync operation is currently in progress or not, use the syncInProgress property, this will be true if there's a sync currently in progress, or false otherwise.
The sync adapter must be a function that when called with a name it returns a class that implements remoteStore interface:
interface remoteStore {
name: string;
clear: () => Promise<boolean>;
del: (key: string) => Promise<boolean>;
set: (key: string, value: string) => Promise<boolean>;
get: (key: string) => Promise<string>;
delBulk: (keys: string[]) => Promise<boolean[]>;
setBulk: (couples: [string, string][]) => Promise<boolean[]>;
getBulk: (keys: string[]) => Promise<(string | undefined)[]>;
keys: () => Promise<string[]>;
}
To go through authentication definition you can write a function that returns the adapter after having the required authentication information. Here's an example that you can use as a starting point:
import { remoteStore } from "xwebdb";
function createAdapter(APIToken: string, endpoint: string) {
return function (name: string) {
return new Store(APIToken, endpoint, name);
};
}
class Store implements remoteStore {
APIToken: string = "";
endpoint: string = "";
name: string = "";
constructor(APIToken: string, endpoint: string, name: string) {
this.APIToken = APIToken;
this.endpoint = endpoint;
this.name = name;
}
async clear() {
// deleting all data
// returns a boolean:
/// true for a successful operation
/// false for a failed operation
}
async del(key: string) {
// delete a specific document by key
// returns a boolean:
/// true for a successful operation
/// false for a failed operation
}
async set(key: string, value: string) {
// set a document (update/insert)
// returns a boolean:
/// true for a successful operation
/// false for a failed operation
}
async get(key: string) {
// gets a document by key
// returns the document as a serialized string
}
async delBulk(keys: string[]) {
// deletes multiple documents
// returns an array of booleans:
/// true for a successful operation
/// false for a failed operation
}
async setBulk(couples: [string, string][]) {
// sets a number of documents
// the argument is an array of couples
// where the first element of each couple is the key
// and the second element of each couple is the value
// returns an array of booleans:
/// true for a successful operation
/// false for a failed operation
}
async getBulk(keys: string[]) {
// returns an array of documents
// specified by the argument "keys" array
// returns an array of serialized (stringified) documents
// at the same order of the passed keys
}
async keys() {
// returns all the keys in the database
// as an array of strings
}
}
Live queries features enable you to query a set of documents as an observable array. Meaning, that the query value will change once the database has been updated and any modification you do on the query result will also be reflected on the database. Think of it like a two-way data-binding but for databases.
To use live queries use the live method, The live method takes two arguments:
find, delete, and update).skip, limit, sort, and project the matched documents, and also to define the direction of the live data, meaning if it's a one way binding, either reflect changes from the database to the query result or reflect modification on the query result to the database, or it's both.db.live(
{
// query
name: "ali",
},
{
// automatically update the query result
// once the database updates
toDB: true, // default: true
// automatically update the database
// if the query result object gets modified
fromDB: true, // default: true
// limiting, skipping, sorting & projecting
// same as `db.find()`
limit: 10,
skip: 1,
project: {
_id: -1,
name: 1,
},
sort: {
name: -1,
},
}
);
The live method will return a promise that resolves to an object like this:
{
// actual observable array
observable: [
{
_id: "1",
name: "ali"
},
{
_id: "2",
name: "dina"
}
],
// call a callback function when the array changes (i.e. creates an observer)
observe(callback),
// removes a specific (or all) observers (i.e. callbacks)
unobserve(callback?),
// do something to the array without notifying the observers
silently(modifier),
// stop data binding in a specific direction or both directions
kill(direction)
}
To add an observer to the resulting query use the observe method:
let liveResult = await db.live({ name: "ali" });
console.log(liveResult.observable); // will print the query result
liveResult.observe((changes) => {
console.log(changes);
});
When the query result gets modified (whether by a database update or query result direct modification) the observe callback will be called with argument of array of changes:
db.insert([
{
_id: "1",
name: "ali",
},
{
_id: "2",
name: "dina",
},
]);
// get everything in the database as a live query
let live = db.live({});
// observe the live query
live.observe((change) => {
console.log(change);
});
// modify the query result
live.observable[1].name = "lilly";
In the above example, the database will update, setting the name of the second document to lilly instead of dina. In addition, since we're observing the query result and printing the changes, the following object will be printed to the console:
[
{
// the object that has been modified
// this can be a document or a sub-document
// depending on the depth of the modification
object: {
_id: "1",
name: "lilly",
},
// the old value
oldValue: "dina",
// the new value
// after the update
value: "lilly",
// the path of the modification
path: [0, "name"],
// a snapshot of the live query result
snapshot: [
{
_id: "1",
name: "ali",
},
{
_id: "2",
name: "lilly",
},
],
// change type
// this can be "insert", "update", "delete"
// "reverse", or "shuffle"
type: "update",
},
];
Observers, like the one above, can be stopped using the unobserve method.
// get everything in the database as a live query
let live = db.live({});
function myObserver1() {
// observer 1 code
}
function myObserver2() {
// observer 2 code
}
// observe the live query
live.observe(myObserver1);
live.observe(myObserver2);
// you can pass the observer to stop it
live.unobserve(myObserver1); // stop observer 1
// or pass an array of observers to stop them
live.unobserve([myObserver1, myObserver2]); // stop observer 1 & 2
// or don't pass anything to stop all observers
live.unobserve(); // stop all observers
// this will also stop reflecting changes to the database
If you don't want to reflect specific modification to the database, you can use the method silently this will apply your modification without notifying (calling) any observer (even the ones you write):
let live = db.live({});
live.silently((observable) => {
// the document will be added without notifying
// any observer
// it even won't be added to the database
// it will only be added to `live.observable`
observable.push({
_id: "3",
name: "John",
});
});
If you'd like to stop reflecting changes to the database use the method kill("toDB")
let live = db.live({});
live.kill("toDB");
// modifications to live.observable
// will not be reflected to the database now
// equivalent to:
let live = db.live({}, { toDB: false });
If you'd like to stop reflecting DB updates to the live query result use kill("fromDB")
let live = db.live({});
live.kill("fromDB");
// updates on the database will not cause the live.observable
// to update
// equivalent to:
let live = db.live({}, { fromDB: false });
If you'd like to stop reflecting updates on both sides on both sides use kill() without passing any argument, and this will turn the live query to a regular find query.
let live = db.live({});
live.kill();
// this will kill the live query, turning it to a regular query
// however, you can still register observers yourself
// equivalent to
let live = db.live({}, { fromDB: false, toDB: false });
Live queries can be very useful when you use XWebDB as a state manager for your front-end framework.
In the following example, I've used the hook useLiveQuery to observe the live query and used stateChange to tell react the state of the application has changed:
Hook:
import * as React from "react";
import { Doc, ObservableArray } from "xwebdb";
export function useLiveQuery<D extends Doc>(req: Promise<ObservableArray<D[]>>) {
const [, stateChange] = React.useState(0);
const [v, data] = React.useState([]);
const stateChangeCallback = React.useCallback(() => {
stateChange((prev) => prev + 1);
}, []);
const observerFn = React.useCallback(() => {
stateChangeCallback();
}, [stateChangeCallback]);
React.useEffect(() => {
req.then((res) => {
res.observe(observerFn);
data(res.observable);
});
return () => {
req.then((res) => {
res.unobserve(observerFn);
});
};
}, [req, observerFn]);
return v;
}
Usage:
export default function App() {
const res = useLiveQuery(db.live());
return (
<div className="App">
<ol>
{res.map((x) => (
<li className="document" key={x._id}>
{JSON.stringify(x, null, " ")}
</li>
))}
</ol>
<br />
<button onClick={() => db.insert({ _id: Math.random().toString() })}>
Insert document in DB
</button>
<p>
The button above will insert a document in the DB and this insertion will be
reflected automatically to the state of the component since it is using the
"live" method.
</p>
</div>
);
}
If you prefer to use react class class components, in the following example I wrote function that enables you to use live queries directly in the class components:
function useLiveQuery<D extends Doc, G extends ObservableArray<D[]>>(
input: Promise<G>,
component: React.Component
): D[] {
let placeholder: D[] = [];
let signature = Math.random().toString();
placeholder[signature] = true;
input.then((res) => {
let unobservers: (() => void)[] = [];
Object.keys(component).forEach((key) => {
if (component[key] && component[key][signature]) {
component[key] = res.observable;
component.setState({});
const observer = () => component.setState({});
res.observe(observer);
unobservers.push(() => res.unobserve(observer));
}
});
const oCWU = component.componentWillUnmount || (() => {});
component.componentWillUnmount = () => {
unobservers.forEach((u) => u());
oCWU.call(component);
};
});
return placeholder;
}
Usage:
class App extends React.Component {
data = useLiveQuery(db.live(), this);
render() {
return (
<div>
Documents:
{this.data.length}
<br />
{this.data.map((x) => (
<div key={x._id}>
<pre>{JSON.stringify(x, null, " ")}</pre>
</div>
))}
<button onClick={() => db.insert({ _id: Math.random().toString() })}>
Insert Document
</button>
</div>
);
}
}
A similar function can be used in angular, however, it needs to access ChangeDetectorRef to tell angular that a change has occurred, it also uses the changeDetectorRef to know whether the component has been destroyed or not:
async function ngLiveQuery<G>(input: Promise<ObservableArray<G[]>>, cd: ChangeDetectorRef) {
let res = await input;
const observer = () => {
if ((cd as any).destroyed) {
res.unobserve(observer);
} else {
cd.detectChanges();
}
};
res.observe(observer);
return res.observable;
}
Usage:
@Component({
selector: "my-app",
standalone: true,
imports: [CommonModule],
template: `
number: {{ ((items$ | async) || []).length }}
<div class="document" *ngFor="let item of items$ | async">
<pre>{{ JSON.stringify(item, null, " ") }}</pre>
</div>
<br />
<button (click)="insert()">Insert document in DB</button>
<p>
The button above will insert a document in the DB and this insertion will be
reflected automatically to the state of the component since it is using the "live"
method.
</p>
`,
})
export class App {
constructor(private cd: ChangeDetectorRef) {}
items$ = ngLiveQuery(db.live(), this.cd);
JSON = JSON;
insert() {
db.insert({} as any);
}
}
A similar implementation can be written for vue framework:
export function useLiveQuery(input, instance) {
let placeholder = [];
let signature = Math.random().toString();
placeholder[signature] = true;
let unobserver = [];
input
.then((res) => {
Object.keys(instance.$.data).forEach((key) => {
if (instance.$.data[key] && instance.$.data[key][signature]) {
instance.$.data[key] = res.observable;
const observer = () => {
instance.$.data[key] = null;
instance.$.data[key] = res.observable;
};
res.observe(observer);
unobserver.push(() => res.unobserve(observer));
}
});
})
.catch((e) => console.log(e.toString()));
instance.$.um = [
function () {
unobserver.forEach((u) => u());
},
];
return placeholder;
}
Usage:
export default {
data() {
return {
name: "hello world!",
documents: useLiveQuery(db.live({}), this),
};
},
methods: {
insert() {
db.insert({});
},
},
};
A similar function can be written for svelte as well:
import { onMount, onDestroy } from "svelte";
import { writable } from "svelte/store";
import { ObservableArray } from "xwebdb";
export function useLiveQuery<G>(input: Promise<ObservableArray<G[]>>) {
const items = writable([]);
const unobservers = [];
onMount(() => {
input.then((res) => {
items.set(res.observable);
const observer = () => items.set(res.observable);
res.observe(observer);
unobservers.push(() => res.unobserve(observer));
});
});
onDestroy(() => {
unobservers.forEach((u) => u());
});
return { items };
}
And used like this:
<script>
import { db } from "../db";
import { useLiveQuery } from "../uselivequery";
const { items } = useLiveQuery(db.live());
</script>
<main>
Documents: {#each $items as item, index (item._id)}
<div key="{item._id}">{item._id}</div>
{/each}
</main>
A similar hook can also be implemented for solid:
export function useLiveQuery<G>(input: Promise<ObservableArray<G[]>>) {
const [items, setItems] = createSignal<G[]>([]);
const unobservers: (() => void)[] = [];
input.then((res) => {
setItems(res.observable);
const observer = () => {
setItems([]);
setItems(res.observable);
};
res.observe(observer);
unobservers.push(() => res.unobserve(observer));
});
onCleanup(() => {
unobservers.forEach((u) => u());
});
return items;
}
And used in your components:
const App = () => {
const items = useLiveQuery(db.live());
return (
<ol>
<For each={items()}>
{(item, index) => (
<li data-index={index()}>{JSON.stringify(item, null, " ")}</li>
)}
</For>
<button onClick={() => db.insert({} as any)}>Add Document</button>
</ol>
);
};
By now, after you have read the examples above, you should be able to write other implementations to support other frameworks, the basics are the same:
Working with deeply nested documents in XWebDB is different from MongoDB. While MongoDB uses the dot notation like this:
{
"children.0.born": 2021
}
This would break the strict typing in XWebDB, instead XWebDB uses the $deep operator where the sub-document would be referenced like this:
{
"$deep": {
"children": {
"0": {
"born": 2021
}
}
}
}
In the following examples, let's suppose that we have a database of parents (Parent), each parent have multiple children (Child) as sub-documents.
import { Database, Doc, SubDoc } from "xwebdb";
class Parent extends Doc {
name: string = "";
born: number = 0;
children: Child[] = xwebdb.mapSubModel(Child, []);
}
class Child extends SubDoc {
name: string = "";
born: number = 0;
}
let db = new Database<Parent>({
ref: "parents-database",
model: Parent,
});
$deep querying// get parents who's first child was born later than 2021
db.find({
$deep: {
children: {
0: {
born: {
$gt: 2021,
},
},
},
},
});
$deep updating// update parent's first child name to "Joseph"
db.update(
{ name: "Ali" },
{
$set: {
$deep: {
children: {
0: {
name: "Joseph",
},
},
},
},
}
);
$deep projecting// get all parents in database but emit first child birth year
db.find(
{},
{
project: {
$deep: {
children: {
0: {
born: -1,
},
},
},
},
}
);
$deep sorting// get all parents in database sorting them by first child name ascending
db.find(
{},
{
sort: {
$deep: {
children: {
0: {
name: 1,
},
},
},
},
}
);
nedb, I've used his logic when writing query runner similar in behavior and API to MongoDB.object-observer, that helped me a lot when writing observable live queries.idb, since I've used much of his code when writing the persistence layer specific to IndexedDB.easy-promise-queue, a simple promise queue, that has been used as a task-queue for database operations.The MIT License (MIT)
Copyright (c) 2023 Ali Saleem
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
todo
FAQs
Strongly-typed, NoSQL, fast, light-weight, synching, Mongo-like database with built-in ODM.
The npm package xwebdb receives a total of 0 weekly downloads. As such, xwebdb popularity was classified as not popular.
We found that xwebdb demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
As cyber threats become more autonomous, AI-powered defenses are crucial for businesses to stay ahead of attackers who can exploit software vulnerabilities at scale.
Security News
UnitedHealth Group disclosed that the ransomware attack on Change Healthcare compromised protected health information for millions in the U.S., with estimated costs to the company expected to reach $1 billion.
Security News
GitHub is susceptible to a CDN flaw that allows attackers to host malware on any public repository.