RESToff
Synchronize client/server data using your existing RESTful APIs.
- Under MIT license.
- Database agnostic and schema-less.
- Reconciliation
- Automatic reconciliation of offline changes.
- In
clientOnly
mode, RESToff automatically provides a complete RESTful offline service for your app. - Reconciliation works without soft deletes or last modified date columns.
- Note: reconciliation is much faster with soft delete and last modified date columns available.
- Conflicts are resolved by default by saving both versions. Setting concurrency to 'overwrite' changes
this behavior to optimistic, last-one-wins strategy where merge conflicts are overwritten
- Supports synchronous RESTful calls.
- Simplifies source code and improves client responsiveness.
- Supports resources in the following formats:
- Works in following frameworks
- Standalone - RESToff can be used without a framework.
- Angular - RESToff wrapped in an Angular 1 provider.
- Limitations and Expectations:
- TODO
- Tests to verify order of pending changes
- edit record A, edit A again: 2nd edit also contains 1st edit
- Next features
- Support rolling back a pending change
- Replace concurrency option with a plugin model
- This also will allow support for custom merge conflict resolution
- support non-standard get/put/post.
- Example: a request GET actually does a delete.
- support put/post updates where resource is changed on the server.
- requires better mockable REST api backend for testing.
- support nested resources (example: /users/45/addresses).
- support non-standard restful api: ability to map a user.
RestOff Usage
Examples:
var roff = restoff({
rootUri : "http://api.example.com/"
});
roff.get("users").then(function(user) {
console.log(rest.dbRepo.read("users"));
});
roff.delete("users/301378d5").then(function(deletedUserId) {
console.log("User " + deletedUserId + " deleted");
});
var aUser = {
"id" : "ffa454",
"first_name": "Happy",
"last_name": "User"
}
roff.post("users", aUser).then(function(user){
console.log("Posted user %O.", user);
});
aUser.last_name = "put";
roff.put("users/ffa454", aUser).then(function(user){
console.log("Put user %O.", user);
});
Let's synchronize subsets of resources on the backend service using the user id:
var roff = restoff({
"rootUri" : "http://api.example.com/"
});
function synchronize(roff, userId) {
roff.get("users/" + userId);
roff.get("books?ownedby="+userId);
roff.get("addresses?userid="+userId);
...
roff.forcedOffline = true;
roff.get("users/" + userId).then(function(user) {
});
}
Resoff Reconciliation in Detail
Row | Action | 1) Server Only Change | 2) Client Only Change | 3) Changes in Both |
---|
A | Insert (Post) | Get and Overwrite on Client | Post and Overwrite on Server | Same Primary Key. Apply B3. |
B | Update (Post/Put) | Get and Overwrite on Client | Post and Overwrite on Server | Brent Reconciliation |
C | Delete | Delete from Client | Delete on Server | Nothing to Do: Both deleted |
D | Delete with Update on client | | | Honor Delete: ignore client update |
- A1, A2, B1, B2, C1, C2, C3: No potential merge conflicts
- A3: This is the case where the same primary key was generated on each client. In this case we will simply view it as a reconciliation of changes because:
- You should be using UUIDs for keys and the chance of collisions is very low. If you are using incrementing IDs, then you will need an algorithm to generate these IDs uniquely on the client. See the
generateId
function.
- D3: The client/server deleted a record that was updated on the server/client. In this case, we honor the delete.
- B3: Using Brent reconciliation which means we assign a new resource id (primaryKey) to the resource and post it.
- This does not work with hierarchical data. The intent is to allow a user to take advantage of the existing User Interface of an Application to deal with the merge conflict.
- TODO: We will be providing a call back that lets the developer provide a custom merge conflict screen.
RESToff Options
rootUri [Required]
rootUri
is appended to RESTful calls when an incomplete uri
is provided. rootUri
is also used, under the hood, to determine the name of the repository.
Examples:
var roff = restoff({
"rootUri" : "http://api.example.com/",
});
roff.get("users").then(function(result){
}
roff.get("http://another.example.com/users").then(function(result){
}
persistenceDisabled [Optional]
When true, storage of RESTful results on the client are disabled.
- Effectivly bypass the core feature of RESToff causing it to act like a standard http library.
- Useful for debugging.
Example configuration:
var roff = restoff({
rootUri : "http://api.example.com/",
persistenceDisabled : true
});
clientOnly [Optional]
Creates a backend RESTful service on your client.
- Allows your RESTful API to run 100% on the client.
- Note: By design, the
pending
repository runs in clientOnly
mode meaning the pending RESTful endpoint never hits the backend service.
var rest = restoff({
"rootUri" : "http://api.example.com/",
"clientOnly" : true
});
rest.put("users", { Name : "Happy User"}).then(function(result) {
rest.get("users").then(function(result)) {
});
});
onCallPending(pendingAction, uri) [Optional]
onCallPending
is the function called by RESToff before a pending call is executed. Provided is all information about the pending action and the uri record.
// TODO: Provide the json format for pendingAction and the uri record.
var rest = restoff({
rootUri: "http://api.example.com/",
onCallPending: function(pendingAction, uri) {
console.log("Pending will execute %O %O", pendingAction, uri);
}
});
dbService.primaryKeyName or options {primaryKeyName:"name"} [Required: Default id]
primaryKeyName
is the name of the primary key field for a given repository.
- RESToff requires every repository to have a single primary key
- the primary key can't be based on a composite key
- The default value is 'id'.
You can globally set the primaryKeyName
or on a per RESTful call basis:
var roff = restlib.restoff({
rootUri: "api.example.com",
dbService: {
primaryKeyName : "ID",
},
});
roff.get("users").then(function(result) {
console.log("Primary key name used was ID.");
});
roff.get("users", {primaryKeyName:"USER_ID"}).then(function(result) {
console.log("Primary key name used was USER_ID.");
});
forcedOffline [Optional]
Forces the application to run in offline mode.
- Useful to see how an application behaves when it is offline.
- Example: Reload all information, go offline, and see how the application behaves.
isOnline
will return false when forcedOffline
is true.
var roff = restoff({
rootUri: "http://api.example.com/"
});
synchronize(roff, "user456");
roff.forceOffline();
if (!roff.isOnline) {
console.log ("We are offline!");
}
NOTE: There is a slight difference between clientOnly
mode and forcedoffline
mode.
- In
forcedOffline
mode, RESToff will store any put/post/delete actions in the pending
repository. - In
clientOnly
mode, Retsoff will not store any put/post/delete actions in the pending
repository.
generateId [Optional]
generateId
is the function called by RESToff to generate a primary key. Bey default, restOff uses RestOff.prototype._guidGenerate()
but you can define your own.
var rest = restoff({
rootUri: "http://api.example.com/",
generateId: function() { return Math.floor((1 + Math.random()) * 0x10000); }
});
onReconciliation(completedAction) [Optional]
onReconciliation
is the function called by RESToff after reconciliation of a given resource is complete.
var rest = restoff({
rootUri: "http://api.example.com/",
onReconciliation: function(completedAction) {
console.log("The following completed action was reconciled and saved to the server %O.", completedAction);
}
});
Note: Placed features will provide a more robust reconciliation process allowing the developer to provide their own custom reconciliation process. Currently, RESToff always applies Brent Reconciliation.
concurrency [Optional]
The concurrency
option defaults to null and conflicts are resolved through Brent reconciliation (keep both versions)
Setting concurrency = 'overwrite'
will result in conflicts resolved by being overwritten by the local version
pendingUri and pendingRepoName
When offline, RESToff places any put/post/delete RESTful operations in a pending
repository. RESToff uses it's own persistence engine meaning it calls itself using get/post/delete.
Use pendingUri
and pendingRepoName
to configure the URI and repository name of the pending
repository.
- Note: In
clientOnly
mode, no changes are recorded in the pending
repository.
RESToff Methods
autoQueryParamSet(name, value)
A parameter of name
with value
will be added/appended to EVERY RESTful api call. Useful for adding parameters such as an access token.
Example usage:
var roff = restoff({
rootUri : "http://api.example.com/"
});
roff.autoQueryParamSet("access_token", "rj5aabcea");
roff.get("users").then(function(user)) {
...
}
autoQueryParamGet(name)
Returns the value of the query parameter with the provided name
.
Example usage:
var roff = restoff({
rootUri : "http://api.example.com/"
});
roff.autoQueryParamSet("access_token", "rj5aabcea");
var paramValue = roff.autoQueryParamGet("access_token");
TODO: Add autoQueryParameters as an option.
A header of name
with value
will be added to the header of every RESTful api call. Useful for adding parameters such as an access token.
Example usage:
var roff = restoff({
rootUri : "http://api.example.com/"
});
roff.autoHeaderParamSet("access_token", "rj5aabcea");
Returns the value of the header parameter with the provided name
.
Example usage:
var roff = restoff({
rootUri : "http://api.example.com/"
});
roff.autoHeaderParamSet("access_token", "rj5aabcea");
var headerValue = roff.autoHeaderParamGet("access_token");
TODO: Add autoHeaderParams as an option.
clear(repoName, [forced])
Clears the cache of the given repository name if the repository exists.
- Does not clear the repository if there are pending changes.
- Optional parameter
forced
- Pass a value of true
to force clearing a repository even if it has pending changes.
- Does not create a repository named
repoName
to the repository if it doesn't exist.
Example:
var roff = restoff({
rootUri : "http://api.example.com/"
});
roff.clear("users", true);
console.log("User repository, even if it had pending, was cleared ")
clearAll([forced])
Clears the cache of all repositories.
- Does not clear any repositories if even one of them have pending changes.
- Optional parameter
forced
- Pass a value of true
to force clearing all repositories even if there are any pending changes.
Example:
var roff = restoff({
rootUri : "http://api.example.com/"
});
roff.clearAll(true);
console.log("User repository, even if it had pending, was cleared ");
delete(uri, [options]), deleteRepo(uri, [options])
delete(uri, [options])
asynchronously deletes a resource from a remote server and in the local repository.deleteRepo(uri, [options])
synchronously deletes a resource from in the local repository.
Note:
- A 404 (not found) is "ignored" and the resource is still considered removed from the local repository.
- If
delete
is called on a non-existent repository, an empty repository is created.
Example usage:
var roff = restoff();
return roff.delete("http://test.development.com:4050/users/553fdf")
.then(function(result){
});
roff.forcedOffline = true;
var result = roff.deleteRepo("http://test.development.com:4050/users/553fdf");
get(uri, [options]), getRepo(uri, [options])
get(uri, [options])
asynchronously retrieves a resource from a remote server. Uses the local repository when offline.getRepo(uri, [options])
synchronously retrieves a resource from the local repository.
Example usage:
var roff = restoff();
return roff.get("http://test.development.com:4050/testsweb/testdata/users")
.then(function(result){
});
roff.forcedOffline = true;
var result = roff.getRepo("http://test.development.com:4050/testsweb/testdata/users");
post(uri, resource, [options]), postRepo(uri, resource, [options])
-
post(uri, resource, [options])
asynchronously posts a resource to a remote server and in the local repository adding the resource if it doesn't exist or overwriting the existing resource.
-
postRepo(uri, resource, [options])
synchronously posts a resource in the local repository adding the resource if it doesn't exist or overwriting the existing resource.
-
When online, post(...)
calls will happen immediately.
-
With postRepo(...)
, updates will happen when a get(...)
is executed on that resource.
Example usage:
var roff = restoff();
var newUser = {
"id" : "ffa454",
"first_name": "Happy",
"last_name": "User"
}
return roff.post("http://test.development.com:4050/users", newUser)
.then(function(result){
});
roff.forcedOffline = true;
var result = roff.postRepo("http://test.development.com:4050/users", newUser)
put(uri, resource, [options]), putRepo(uri, resource, [options])
put(uri, resource, [options])
asynchronously puts a known resource on a remote server and in the local repository updating the resource id provided in the uri.putRepo(uri, resource, [options])
synchronously puts a known resource in the local repository updating the resource id provided in the uri.
Example usage:
var roff = restoff({
"rootUri" : "http://test.development.com:4050/"
});
var existingUser = {
"id" : "ffa454",
"first_name": "Happy",
"last_name": "User"
}
return roff.post("users/ffa454", existingUser)
.then(function(result){
});
roff.forcedOffline = true;
var result = roff.postRepo("users/ffa454", existingUser)
uriFromClient(uri)
restOff may add additional query parameters when restOff.get(uri) is called. Use this method to see the final uri sent to the backend server.
Example usage:
var roff = restoff().autoQueryParamSet("access_token", "rj5aabcea");
var actualUri = roff.uriFromClient("http://test.development.com:4050/emailaddresses");
expect(actualUri)).to.equal("http://test.development.com:4050/emailaddresses?access_token=rj5aabcea");
FAQ
- What is the difference between
offlineOnly
and forcedOffline
?
forcedOffline
will line up any pending changes that are then synchronized when the client comes back online.offlineOnly
does not log any "pending changes" because the repository will never be synchronzied with a backend service.
Developoment Setup
Want to help out? Here is what you need to do to get started.
// TODO: Finish this part of documentation
Using In Your Projects
Change directory to your node project.
$ npm install --save restoff
Setup
$ npm install
$ npm install -g json-server // for testing
Update Node Module Dependencies
$ npm outdated
Node Modules Used
Continuous Rebuild and Testing
See ./dist for files we build.
$ gulp
Test
$ gulp webtests
Test Server
Read documentation in gulpfile.js to see how to setup automated web testing.
$ gulp webserver
Publish and Push New Version
First time?
$ npm adduser # Need to do one time
-
Verifty package.json version is newer than one in npm (visit https://www.npmjs.com/package/restoff)
-
Verify no pending changes
$ git status
$ npm publish ./
$ git tag -a 0.1.4 -m "v0.1.4" // 0.1.4 is an example
$ git push origin --tags
- Increment version in package.json and update check-build test. Re-run tests.
Development Setup
- Install LiveReload Chrome extension.
Hosts File
Add to your /etc/hosts file:
127.0.0.1 test.development.com
Initial install
$ npm install
(NPM install stuff ommitted)
$ mocha tests
3 passing (or something like this)
Start the test suite
$ gulp
(gulp stuff happens, look for this line...)
Server started http://test.development.com:4050
... then open the test suite in your browser
http://test.development.com:4050
RESToff Angular
RESToff is wraped in an angular provider.
Example Usage:
angular.module("fakeRoot", ["restoff"])
.config(["restoffProvider", function (restoffProvider) {
restoffProvider.setConfig({
rootUri:"http://localhost/"
});
}]);
Note that we "hard code" the configuration, but you could also get the configuration from another service.
Development Issues
Live Reload Isn't Reloading
- In chrome, navigate to
chrome://extensions/
then find the LiveReload extension and check Allow access to file URLs. - Click on livereload icon in chrome browser: small circle in center should become solid.
- Is more than one instance of gulp running?
Hide HTTP Network Messages During Testing
- In chrome, right mouse click and inspect.
- From the console tab, click on filter (icon next to Top)
- Check "Hide network messages"