Huge News!Announcing our $40M Series B led by Abstract Ventures.Learn More

worker-pouch

Advanced tools

Socket logo

Install Socket

Detect and block malicious and high-risk dependencies

Install

worker-pouch

PouchDB over a web worker


Version published
Maintainers
1
Created

worker-pouch Build Status

// This pouch is powered by Workers!
var db = new PouchDB('mydb', {adapter: 'worker'});

Adapter plugin to use PouchDB over Web Workers or Service Workers. Transparently proxies all PouchDB API requests to the Worker, so that the most expensive database operations are run in a separate thread.

Basically, worker-pouch allows you use the PouchDB API like you normally would, but your UI will suffer fewer hiccups, because any blocking operations (such as IndexedDB or checksumming) are run inside of the worker.

The worker-pouch adapter passes the full PouchDB test suite. It requires PouchDB 5.0.0+.

Topics

Install

$ npm install worker-pouch

Usage

This plugin has two modes:

  • Easy Mode, supporting Chrome and Firefox, with a fallback for other browsers, and
  • Custom Mode, potentially supporting more browsers, and allowing you to use your own Web Worker or Service Worker.

In Easy Mode, you don't need to set up the worker yourself, because the script is loaded in a Blob URL. Whereas in Custom Mode, you must manage the Web Worker or Service Worker yourself.

Easy Mode

You can do Easy Mode either with prebuilt JavaScript or via Browserify/Webpack.

Setup via prebuilt JavaScript

The client JS file is available at node_modules/worker-pouch/dist/pouchdb.worker-pouch.js. Or you can just download it from Github above (in which case, it will be available as window.workerPouch).

Then include it in your HTML, after PouchDB:

<script src="pouchdb.js"></script>
<script src="pouchdb.worker-pouch.js"></script>

Then you can create a worker-powered PouchDB using:

var db = new PouchDB('mydb', {adapter: 'worker'});
Setup via Browserify/Webpack/etc.

The same rules apply, but you have to notify PouchDB of the new adapter:

var PouchDB = require('pouchdb');
PouchDB.adapter('worker', require('worker-pouch'));
Detecting browser support

Unfortunately, creating workers via Blob URLs is not supported in all browsers. In particular, IE, Edge, Safari, and iOS are not supported. Luckily, Firefox and Chrome are the browsers that benefit the most from web workers. There is also an API to detect browser support.

In Easy Mode, you will need to use the isSupportedBrowser() API if you would like to support browsers other than Firefox and Chrome.

Custom Mode

In this mode, you manage the Web Worker or Service Worker yourself, and you register the two endpoints so that worker-pouch can communicate with the "backend" and "frontend."

Since this doesn't require Blob URLs, and because you can use custom PouchDB objects, you can potentially support more browsers this way. It's much more flexible.

This mode only supports bundling via Browserify/Webpack/etc. There is no prebuilt option.

To use, you'll need this code on the client side:

// client-side code
var PouchDB = require('pouchdb');
PouchDB.adapter('worker', require('worker-pouch/client'));

var worker = new Worker('worker.js'); // or Service Worker

var db = new PouchDB('mydb', {
  adapter: 'worker',
  worker: function () { return worker; }
});

Note that you create the PouchDB object passing in both adapter: 'worker' and worker, which points to a function that returns the worker. (Once this PouchDB bug is resolved, you'll be able to directly pass in the worker object instead.)

Then you include this code on the worker side:

// worker-side code
var registerWorkerPouch = require('worker-pouch/worker');
var PouchDB = require('pouchdb');

// attach to global `self` object
registerWorkerPouch(self, PouchDB);

If you would like to customize how PouchDB is created inside of the worker, then you can also pass in a custom PouchDB factory function, which is a function that takes an options object (e.g. {name: 'mydb', auto_compaction: true}) and returns a PouchDB object.

This is useful in cases where PouchDB's IndexedDB adapter doesn't work inside of a worker (such as Safari), so for instance you can have the pouchCreator function return an in-memory PouchDB object.

Here's an example:

var PouchDB = require('pouchdb');
require('pouchdb/extras/memory');
function pouchCreator(opts) {
  opts.adapter = 'memory'; // force in-memory mode
  return new PouchDB(opts);
}

var registerWorkerPouch = require('worker-pouch/worker');
registerWorkerPouch(self, pouchCreator);

The PouchDB worker code will listen for messages from the client side, but should ignore any non-worker-pouch messages, so you are free to still use worker.postMessage() as desired.

Performance benefits of worker-pouch

These numbers were recorded using this site. The test involved inserting 10000 PouchDB documents, and was run on a 2013 MacBook Air. Browser data was deleted between each test.

Time (ms)Blocked the DOM?
Chrome 48
   put() - normal50070No
   put() - worker56993No
   bulkDocs() - normal2740Yes
   bulkDocs() - worker3454No
Firefox 43
   put() - normal39595No
   put() - worker41425No
   bulkDocs() - normal1027Yes
   bulkDocs() - worker1130No

Basic takeaway: put()s avoid DOM-blocking (due to using many smaller transactions), but are much slower than bulkDocs(). With worker-pouch, though, you can get nearly all the speed benefit of bulkDocs() without blocking the DOM.

(Note that by "blocked the DOM," I mean froze the animated GIF for a significant amount of time - at least a half-second. A single dropped frame was not penalized. Try the test yourself, and you'll see the difference is pretty stark.)

Fallback for unsupported browsers

In Easy Mode, this plugin doesn't support all browsers. So it provides a special API to dianogose whether or not the current browser supports worker-pouch. Here's how you can use it:

var workerPouch = require('worker-pouch');

workerPouch.isSupportedBrowser().then(function (supported) {
  var db;
  if (supported) {
    db = new PouchDB('mydb', {adapter: 'worker'});
  } else { // fall back to a normal PouchDB
	db = new PouchDB('mydb');
  }  
}).catch(console.log.bind(console)); // shouldn't throw an error

The isSupportedBrowser() API returns a Promise for a boolean, which will be true if the browser is supported and false otherwise.

If you are using this method to return the PouchDB object itself from a Promise, be sure to wrap it in an object, to avoid "circular promise" errors:

var workerPouch = require('worker-pouch');

workerPouch.isSupportedBrowser().then(function (supported) {
  if (supported) {
    return {db: new PouchDB('mydb', {adapter: 'worker'})};
  } else { // fall back to a normal PouchDB
	return {db: new PouchDB('mydb')};
  }
}).then(function (dbWrapper) {
  var db = dbWrapper.db; // now I have a PouchDB
}).catch(console.log.bind(console)); // shouldn't throw an error

Debugging

worker-pouch uses debug for logging. So in the browser, you can enable debugging by using PouchDB's logger:

PouchDB.debug.enable('pouchdb:worker:*');

FAQs

Wait, doesn't PouchDB already work in a Web Worker?

Yes, you can use pure PouchDB inside of a Web Worker. But the point of this plugin is to let you use PouchDB from outside a Web Worker, and then have it transparently proxy to another PouchDB that is isolated in a Web Worker.

What browsers are supported?

Only those browsers that 1) allow blob URLs for Web Worker scripts, and 2) allow IndexedDB inside of a Web Worker. Today, that means Chrome and Firefox.

Can I use it with other plugins?

Not right now, although map/reduce is supported.

Don't I pay a heavy cost of structured cloning due to worker messages?

Yes, but apparently this cost is less than that of IndexedDB, because the DOM is significanty less blocked when using worker-pouch. Another thing to keep in mind is that PouchDB's internal document representation in IndexedDB is more complex than the PouchDB documents you insert. So you clone a small PouchDB object to send it to the worker, and then inside the worker it's exploded into a more complex IndexedDB object. IndexedDB itself has to clone as well, but the more complex cloning is done inside the worker.

Does replication occur inside the worker?

It's a bit subtle. The answer is yes, if you do this:

var local = new PouchDB('local', {adapter: 'worker'});
local.replicate.to('http://example.com/db');

However, the answer is no if you do:

var local = new PouchDB('local', {adapter: 'worker'});
var remote = new PouchDB('http://example.com/db');
local.replicate.to(remote);

The reason is that when you create a remote PouchDB using new PouchDB('http://example.com/db'), then that runs inside the UI thread. However, when you .replicate.to('http://example.com/db'), then that string is passed ver-batim to the worker thread, where worker-pouch becomes responsible for creating the remote PouchDB. Hence replication will occur inside of the worker thread.

In general, if you are very concerned about performance implications of what runs inside of the woker vs what runs outside of the worker, you are encouraged to not use worker-pouch and to instead just run PouchDB inside a worker and handle message-passing yourself (might I recommend promise-worker?). This is the only way to really ensure that all PouchDB operations are isolated to the worker.

Building

npm install
npm run build

Your plugin is now located at dist/pouchdb.worker-pouch.js and dist/pouchdb.worker-pouch.min.js and is ready for distribution.

Testing

In the browser

Run npm run dev and then point your favorite browser to http://127.0.0.1:8000/test/index.html.

The query param ?grep=mysearch will search for tests matching mysearch.

Automated browser tests

You can run e.g.

CLIENT=selenium:firefox npm test
CLIENT=selenium:phantomjs npm test

This will run the tests automatically and the process will exit with a 0 or a 1 when it's done. Firefox uses IndexedDB, and PhantomJS uses WebSQL.

Running the custom-api tests

Run:

npm run test-custom

Or to debug:

npm run test-custom-local

FAQs

Package last updated on 23 May 2016

Did you know?

Socket

Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.

Install

Related posts