qoper8-ww - npm Package Compare versions

Comparing version 2.4.0 to 2.5.0

package.json

 {
   "name": "qoper8-ww",
-  "version": "2.4.0",
+  "version": "2.5.0",
   "description": "Queue-Based WebWorker Pool Management Systems",
@@ -5,0 +5,0 @@ "main": "/src/QOper8.js",

README.md

@@ -55,11 +55,11 @@ # QOper8: Queue-based WebWorker Pool Manager
 
-      const {QOper8} = await import('https://cdn.jsdelivr.net/gh/robtweed/QOper8/src/qoper8.min.js');
+      const {QOper8} = await import('https://cdn.jsdelivr.net/gh/robtweed/QOper8/src/QOper8.min.js');
 
 ### Clone from Github
 
-Alternatively, clone or copy the file [*/src/qoper8.min.js*](/src/qoper8.min.js)
+Alternatively, clone or copy the file [*/src/QOper8.min.js*](/src/QOper8.min.js)
 to an appropriate directory on your web server and load it directly from there, eg:
 
 
-      const {QOper8} = await import('/path/to/qoper8.min.js');
+      const {QOper8} = await import('/path/to/QOper8.min.js');
 
@@ -89,8 +89,10 @@ ### From NPM
 
+- *disableLogging*: if set to *true*, QOper8's externally-accessible read/write *logging* property is deactivated, thereby preventing the risk of unauthorised message "snooping* using the browser's JavaScript console in a production system.  If not specified, it defaults to *false*.
 
+
 You can optionally modify the parameters used by QOper8 for monitoring and shutting down inactive WebWorker processes, by using the following *options* properties:
 
-- *inactivityCheckInterval*: how frequently (in seconds) a WebWorker checks itself for inactivity.  If not specified, a value of 60 (seconds) is used
+- *workerInactivityCheckInterval*: how frequently (in seconds) a WebWorker checks itself for inactivity.  If not specified, a value of 60 (seconds) is used
 
-- *inactivityLimit*: the length of time (in minutes) a WebWorker process can remain inactive until QOper8 shuts it down.  If not specified, the maximum inactivity duration is 20 minutes.
+- *workerInactivityLimit*: the length of time (in minutes) a WebWorker process can remain inactive until QOper8 shuts it down.  If not specified, the maximum inactivity duration is 20 minutes.
 
@@ -171,3 +173,3 @@
 
-So, as you can see, everything related to the WebWorker processes and the message flow between the main process and the WebWorker processes is handled automatically for you by QOper8.  As far as you are concerned, there are just two steps:
+So, as you can see, everything related to the WebWorker processes and the message flow between the main process and the WebWorker processes is handled automatically for you by QOper8.  As far as you are concerned, there are just three steps:
 
@@ -178,3 +180,5 @@ - you ceeate a Message Handler script file for each of your required message *type*s
 
+- you await the response object returned from the WebWorker by your message handler
 
+
 ## The Message Handler Method Script
@@ -250,3 +254,3 @@
 
-        const {QOper8} = await import('../js/qoper8.min.js');
+        const {QOper8} = await import('../js/QOper8.min.js');
 
@@ -302,2 +306,47 @@ // Start/Configure an instance of QOper8:
 
+## How Many WebWorkers Should I Use?
+
+It's entirely up to you.  Each WebWorker in your pool will be able to invoke your type-specific message handler, and each will run identically.  There's a few things to note:
+
+- Having more than one WebWorker will allow a busy workload of queued messages to be shared amongst the WebWorker pool;
+
+- if you have more than one WebWorker, you have no control over which WebWorker handles each message you add to the QOper8 queue.  This should normally not matter to you, but you need to be aware;
+
+- A QOper8 WebWorker process only handles a single message at a time.  The WebWorker is not available again until it invokes the *finished()* method within your handler.
+
+- You'll find that overall throughput will initially increase as you add more WebWorkers to your pool, but you'll then find that throughput will start to decrease as you further increase the pool.  It will depend on a number of factors, such as the type of browser and the number of CPU cores available on the machine running the browser.  Typically optimal throughput is achieved with 3-4 WebWorkers.
+
+- If you use just a single WebWorker, your queued messages will be handled individually, one at a time, in strict chronological sequence.  This can be advantageous for certain kinds of activity where you need strict control over the serialisation of activities.  The downside is that the overall throughput will be typically less than if you had a larger WebWorker pool.
+
+
+## Benchmarking QOper8 Throughput
+
+To get an idea of the throughput performance of QOper8 on different browsers, try out our simple 
+[QOper8 Benchmarking Application](https://robtweed.github.io/QOper8/benchmark).  
+
+This application allows you to specify the WebWorker Pool Size, and you then set up the parameters for generating a stream of identical messages that will be handled by a simple almost "do-nothing" message handler.  
+
+You specify the total number of messages you want to generate, eg 50,000, but rather than the application simply adding the whole lot to the QOper8 queue in one go, you define how to generate batches of messages that get added to the queue.  So you define:
+
+- the batch size, eg 1000 messages at a time
+- the delay time between batches, eg 200ms
+
+This avoids the performance overheads of the browser's JavaScript run-time handling a potentially massive array which could potententially adversely affect the performance throughput.  
+
+The trick is to create a balance of batch size and delay to maintain a sustainably-sized queue.  The application reports its work and results to the browser's JavaScript console, and will tell you if the queue increases with each message batch, or if the queue is exhausted between batches.
+
+Keep tweaking the delay time:
+
+- increase it if the queue keeps expanding with each new batch
+- decrease it if the queue is getting exhausted at each batch
+
+At the end of each run, the application will display, in the JavaScript console:
+
+- the total time taken
+- the throughtput rate (messages handled per second)
+- the number of messages handled by each of the WebWorkers in the pool you specified.
+
+The results can be pretty interesting, particularly comparing throughput for different browsers on the same hardware platform.  For example, you will probably find that Firefox is significantly faster than Safari.
+
+
 ## Optionally Packaging Your Message Handler Code
@@ -331,3 +380,3 @@
 
-        let url = QOper8.createUrl(handlerCode);
+        let url = qoper8.createUrl(handlerCode);
 
@@ -347,3 +396,3 @@ And we can now use this URL in the *handlersByMessageType* Map, eg:
 
-        const {QOper8} = await import('../js/qoper8.min.js');
+        const {QOper8} = await import('../js/QOper8.min.js');
 
@@ -367,3 +416,3 @@ // Start/Configure an instance of QOper8, without specifying a handlersByMessageType Map:
         `;
-        let url = QOper8.createUrl(handlerCode);
+        let url = qoper8.createUrl(handlerCode);
 
@@ -419,3 +468,22 @@ // Now add this Blob URL to your QOper8 instance's handlersByMessageType Map:
 
+- qoper8.getMessageCount(): Returns the total number of messages successfully handled by QOper8
 
+- qoper8.getQueueLength(): Returns the current queue length.  Under most circumstances this should usually return zero.
+
+- qoper8.stop(): Controllably shuts down all WebWorkers in the pool and prevents any further messages being added to the queue.  Any messages currently in the queue will remain there and will not be processed.
+
+- qoper8.start(): Can be used after a *stop()* to resume QOper8's ability to add messages to its queue and to process them.  QOper8 will automatically start up new WebWorker(s).
+
+- qoper8.createUrl(code): Creates a *blobURL* (see earlier for explanation).
+
+### Properties:
+
+- qoper8.name: returns *QOper8*
+
+- qoper8.build: returns the build number, eg 2.5
+
+- qoper8.buildDate: returns the date the build was created
+
+- qoper8.logging: read/write property, defaults to *false*.  Set it to *true* to see a trace of QOper8 foreground and WebWorker activity in the JavaScript console.  Set to false for production systems to avoid any overheads and to prevent message snooping. To prevent unauthorised logging, use the *disableLogging* property at startup (this cannot be subsequently modified by a user or third-party script)
+
 ## Events
@@ -457,2 +525,4 @@
 - *workerTerminated*: emitted whenever QOper8 shuts down a WebWorker
+- *stop*: emitted whenever QOper8 is stopped using the *stop()* API
+- *start*: emitted whenever QOper8 is re-started using the *start()* API
 
@@ -459,0 +529,0 @@ You can provide your own custom handlers for these events by using the *on()* method within your main module.

src/QOper8.js

@@ -26,3 +26,3 @@ /*
 
-7 August 2022
+15 August 2022
 
@@ -33,3 +33,3 @@ */
 
-let workerCode = `let QWorker=class{constructor(){this.listeners=new Map,this.logging=!1;let e=new Map,r=!1,t=!1,i=this,o=!1,s=!1,n=!1,a=6e4,l=18e4,g=new Map,p=!1,d=Date.now(),h=function(){i.log("Worker "+r+" sending request to shut down");clearInterval(QOper8Worker.timer),postMessage({qoper8:{shutdown:!0}}),i.emit("shutdown_signal_sent")},c=function(e){(e=e||{}).qoper8||(e.qoper8={}),e.qoper8.finished=!0,postMessage(e),i.emit("finished",e),o=!1,s&&h()};this.onMessage=function(f){let m;if(d=Date.now(),o=!0,f.qoper8&&f.qoper8.init&&void 0!==f.qoper8.id)return t?(m="QOper8 Worker "+r+" has already been initialised",i.emit("error",m),c({error:m,originalMessage:f})):(r=f.qoper8.id,n=f.qoper8.uuid,f.qoper8.workerInactivityCheckTime&&(a=f.qoper8.workerInactivityCheckTime),f.qoper8.workerInactivityLimit&&(l=f.qoper8.workerInactivityLimit),f.qoper8.handlersByMessageType&&(g=f.qoper8.handlersByMessageType),i.logging=f.qoper8.logging,p=setInterval(function(){let e=Date.now()-d;i.log("Worker "+r+" inactive for "+e),i.log("Inactivity limit: "+l),e>l&&(o?(i.log("Worker "+r+" flagged for termination"),s=!0):h())},a),i.log("new worker "+r+" started..."),i.emit("started",{id:r}),t=!0,c());if(!f.qoper8||!f.qoper8.uuid)return m="Invalid message sent to QOper8 Worker "+r,i.emit("error",m),c({error:m,originalMessage:f});if(f.qoper8.uuid!==n)return m="Invalid UUID on message sent to QOper8 Worker "+r,i.emit("error",m),c({error:m,originalMessage:f});let u=JSON.parse(JSON.stringify(f));if(delete f.qoper8.uuid,delete u.qoper8,i.log("Message received by worker "+r+": "+JSON.stringify(u,null,2)),i.emit("received",{message:u}),!f.type&&!f.handlerUrl)return m="No type or handler specified in message sent to worker "+r,i.emit("error",m),c({error:m,originalMessage:u});if(!f.type||!g.has(f.type))return m="No handler for messages of type "+f.type,i.log(m),i.emit("error",m),c({error:m,originalMessage:u});if(!e.has(f.type)){let t=g.get(f.type);i.log("fetching "+t);try{importScripts(t);let o=self.handler;e.set(f.type,o),i.emit("handler_imported",{handlerUrl:t})}catch(e){return m="Unable to load Handler Url "+t,i.log(m),i.log(JSON.stringify(e,null,2)),i.emit("error",m),c({error:m,originalMessage:u,workerId:r})}}e.get(f.type).call(i,f,c)}}log(e){this.logging&&console.log(Date.now()+": "+e)}on(e,r){this.listeners.has(e)||this.listeners.set(e,r)}off(e){this.listeners.has(e)&&this.listeners.delete(e)}emit(e,r){if(this.listeners.has(e)){this.listeners.get(e).call(this,r)}}},QOper8Worker=new QWorker;onmessage=async function(e){QOper8Worker.onMessage(e.data)};`;
+let workerCode = `let QWorker=class{constructor(){this.logging=!1;let e=new Map,r=new Map,t=!1,o=!1,i=this,n=!1,s=!1,a=!1,l=6e4,g=18e4,p=new Map,d=!1,f=Date.now(),h=0,u=function(){i.log("Worker "+t+" sending request to shut down");clearInterval(QOper8Worker.timer),postMessage({qoper8:{shutdown:!0}}),i.emit("shutdown_signal_sent")},c=function(e){(e=e||{}).qoper8||(e.qoper8={}),e.qoper8.finished=!0,postMessage(e),i.emit("finished",e),n=!1,s&&u()};this.getMessageCount=function(){return h},this.on=function(r,t){e.has(r)||e.set(r,t)},this.off=function(r){e.has(r)&&e.delete(r)},this.emit=function(r,t){if(e.has(r)){e.get(r).call(this,t)}},this.onMessage=function(e){let m;if(f=Date.now(),n=!0,e.qoper8&&e.qoper8.init&&void 0!==e.qoper8.id)return o?(m="QOper8 Worker "+t+" has already been initialised",i.emit("error",m),c({error:m,originalMessage:e})):(t=e.qoper8.id,a=e.qoper8.uuid,e.qoper8.workerInactivityCheckInterval&&(l=e.qoper8.workerInactivityCheckInterval),e.qoper8.workerInactivityLimit&&(g=e.qoper8.workerInactivityLimit),e.qoper8.handlersByMessageType&&(p=e.qoper8.handlersByMessageType),i.logging=e.qoper8.logging,d=setInterval(function(){let e=Date.now()-f;i.log("Worker "+t+" inactive for "+e),i.log("Inactivity limit: "+g),e>g&&(n?(i.log("Worker "+t+" flagged for termination"),s=!0):u())},l),i.log("new worker "+t+" started..."),i.emit("started",{id:t}),o=!0,c());if(!o)return m="QOper8 Worker "+t+" has not been initialised",i.emit("error",m),c({error:m,originalMessage:e});if(!e.qoper8||!e.qoper8.uuid)return m="Invalid message sent to QOper8 Worker "+t,i.emit("error",m),c({error:m,originalMessage:e});if(e.qoper8.uuid!==a)return m="Invalid UUID on message sent to QOper8 Worker "+t,i.emit("error",m),c({error:m,originalMessage:e});let y=JSON.parse(JSON.stringify(e));if(delete e.qoper8.uuid,delete y.qoper8,i.log("Message received by worker "+t+": "+JSON.stringify(y,null,2)),i.emit("received",{message:y}),"qoper8_terminate"!==e.type){if(!e.type&&!e.handlerUrl)return m="No type or handler specified in message sent to worker "+t,i.emit("error",m),c({error:m,originalMessage:y});if(!e.type||!p.has(e.type))return m="No handler for messages of type "+e.type,i.log(m),i.emit("error",m),c({error:m,originalMessage:y});if(!r.has(e.type)){let o=p.get(e.type);i.log("fetching "+o);try{importScripts(o);let n=self.handler;r.set(e.type,n),i.emit("handler_imported",{handlerUrl:o})}catch(e){return m="Unable to load Handler Url "+o,i.log(m),i.log(JSON.stringify(e,null,2)),i.emit("error",m),c({error:m,originalMessage:y,workerId:t})}}h++,r.get(e.type).call(i,e,c)}else u()}}log(e){this.logging&&console.log(Date.now()+": "+e)}},QOper8Worker=new QWorker;onmessage=async function(e){QOper8Worker.onMessage(e.data)};`;
 
@@ -43,2 +43,16 @@ // ******* QOper8 *****************
 
+    function uuidv4() {
+      if (window.location.protocol === 'https:') {
+        return ([1e7]+-1e3+-4e3+-8e3+-1e11).replace(/[018]/g, c =>
+          (c ^ crypto.getRandomValues(new Uint8Array(1))[0] & 15 >> c / 4).toString(16)
+        );
+      }
+      else {
+        return 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, function(c) {
+          let r = Math.random() * 16 | 0, v = c == 'x' ? r : (r & 0x3 | 0x8);
+          return v.toString(16);
+        });
+      }
+    }
+
     if (obj.workerInactivityCheckInterval) obj.workerInactivityCheckInterval = obj.workerInactivityCheckInterval * 1000;
@@ -48,12 +62,13 @@ if (obj.workerInactivityLimit) obj.workerInactivityLimit = obj.workerInactivityLimit * 60000;
     this.name = 'QOper8';
-    this.build = '2.4';
-    this.buildDate = '7 August 2022';
+    this.build = '2.5';
+    this.buildDate = '15 August 2022';
     this.logging = obj.logging || false;
-    this.poolSize = obj.poolSize || 1;
-    this.worker = {
-      inactivityCheckInterval: obj.workerInactivityCheckInterval || 60000,
-      inactivityLimit: obj.workerInactivityLimit || 20 * 60000
-    }
+    let poolSize = +obj.poolSize || 1;
+    let maxPoolSize = obj.maxPoolSize || 32;
+    if (poolSize > maxPoolSize) poolSize = maxPoolSize;
+    let loggingDisabled = obj.disabled || false;
+    let inactivityCheckInterval = obj.workerInactivityCheckInterval || 60000;
+    let inactivityLimit = obj.workerInactivityLimit || (20 * 60000);
     this.handlersByMessageType = obj.handlersByMessageType || new Map();
-    this.listeners = new Map();
+    let listeners = new Map();
 
@@ -66,18 +81,39 @@ let uuid = uuidv4();
     let nextWorkerId = 0;
+    let stopped = false;
+    let noOfMessages = 0;
     let q = this;
 
-    function uuidv4() {
-      if (window.location.protocol === 'https:') {
-        return ([1e7]+-1e3+-4e3+-8e3+-1e11).replace(/[018]/g, c =>
-          (c ^ crypto.getRandomValues(new Uint8Array(1))[0] & 15 >> c / 4).toString(16)
-        );
+    this.log = function(message) {
+      if (!loggingDisabled && this.logging) {
+        console.log(Date.now() + ': ' + message);
       }
-      else {
-        return 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, function(c) {
-          let r = Math.random() * 16 | 0, v = c == 'x' ? r : (r & 0x3 | 0x8);
-          return v.toString(16);
-        });
+    };
+
+    this.getMessageCount = function() {
+      return noOfMessages;
+    };
+
+    this.on = function(type, callback) {
+      if (!listeners.has(type)) {
+        listeners.set(type, callback);
       }
-    }
+    };
 
+    this.off = function(type) {
+      if (listeners.has(type)) {
+        listeners.delete(type);
+      }
+    };
+
+    this.emit = function(type, data) {
+      if (listeners.has(type)) {
+        let handler =  listeners.get(type);
+        handler.call(q, data);
+      }
+    };
+
+    this.getQueueLength = function() {
+      return queue.length;
+    };
+
     function processQueue() {
@@ -98,3 +134,3 @@ q.log('try processing queue: length ' + queue.length);
         q.log('no available workers');
-        if (workers.size < q.poolSize) {
+        if (workers.size < poolSize) {
           q.log('starting new worker');
@@ -166,3 +202,3 @@ startWorker();
           let callback = callbacks.get(worker.id);
-          if (callback) callback(res);
+          if (callback) callback(res, worker.id);
           callbacks.delete(worker.id);
@@ -174,3 +210,4 @@ }
             isAvailable.set(+worker.id, true);
-            processQueue();
+            q.emit('worker' + worker.id + 'Available');
+            if (!stopped) processQueue();
           }
@@ -181,2 +218,3 @@ else if (res.qoper8.shutdown) {
             q.emit('workerTerminated', worker.id);
+            q.emit('worker' + worker.id + 'Terminated');
             worker.terminate();
@@ -193,4 +231,4 @@ }
           handlersByMessageType: q.handlersByMessageType,
-          workerInactivityCheckTime: q.worker.inactivityCheckTime,
-          workerInactivityLimit: q.worker.inactivityLimit,
+          workerInactivityCheckInterval: inactivityCheckInterval,
+          workerInactivityLimit: inactivityLimit,
           logging: q.logging
@@ -204,3 +242,12 @@ }
 
-    this.addToQueue = function(obj) {
+    function addToQueue(obj) {
+      if (stopped) {
+        if (obj.qoper8 && obj.qoper8.callback) {
+          obj.qoper8.callback({
+            error: 'QOper8 has been stopped'
+          });
+        }
+        return;
+      }
+      noOfMessages++;
       queue.push(obj);
@@ -211,14 +258,55 @@ q.emit('addedToQueue', obj);
 
-  }
+    this.message = function(obj, callback) {
+      if (!obj.qoper8) obj.qoper8 = {};
+      obj.qoper8.callback =  callback || false
+      addToQueue(obj);
+    }
 
-  log(message) {
-    if (this.logging) {
-      console.log(Date.now() + ': ' + message);
+    function isNowAvailable(id) {
+      return new Promise((resolve) => {
+        q.on('worker' + id + 'Available', function() {
+          q.off('worker' + id + 'Available');
+          resolve();
+        });
+      });
+    };
+
+    function isStopped(id) {
+      return new Promise((resolve) => {
+        q.on('worker' + id + 'Terminated', function() {
+          q.off('worker' + id + 'Terminated');
+          resolve();
+        });
+      });
+    };
+
+    this.stop = async function() {
+      stopped = true;
+      for (const [id, worker] of workers) {
+        if (isAvailable.get(+id)) {
+          q.log('Web Worker ' + id + ' is being stopped');
+          let msg = {type: 'qoper8_terminate'};
+          sendMessage(msg, worker);
+          await isStopped(id);
+          q.log('Worker Thread ' + id + ' has been stopped');
+        }
+        else {
+          q.log('Waiting for Worker Thread ' + id + ' to become available');
+          await isNowAvailable(id);
+          let msg = {type: 'qoper8_terminate'};
+          sendMessage(msg, worker);
+          await isStopped(id);
+          q.log('Worker Thread ' + id + ' has been stopped');
+        }
+      }
+      q.emit('stop');
+      q.log('No Worker Threads are running.  QOper8 is no longer handling messages');
+    };
+
+    this.start = function() {
+      stopped = false;
+      q.log('QOper8 is started and will handle messages');
+      processQueue();
     }
-  }
 
-  message(obj, callback) {
-    if (!obj.qoper8) obj.qoper8 = {};
-    obj.qoper8.callback =  callback || false
-    this.addToQueue(obj);
   }
@@ -263,22 +351,4 @@
 
-  on(type, callback) {
-    if (!this.listeners.has(type)) {
-      this.listeners.set(type, callback);
-    }
-  }
-
-  off(type) {
-    if (this.listeners.has(type)) {
-      this.listeners.delete(type);
-    }
-  }
-  emit(type, data) {
-    if (this.listeners.has(type)) {
-      let handler =  this.listeners.get(type);
-      handler.call(this, data);
-    }
-  }
-
 }
 
 export {QOper8};

src/QOper8.min.js

@@ -1,1 +0,1 @@
-let workerCode='let QWorker=class{constructor(){this.listeners=new Map,this.logging=!1;let e=new Map,r=!1,t=!1,i=this,o=!1,s=!1,n=!1,a=6e4,l=18e4,g=new Map,p=!1,d=Date.now(),h=function(){i.log("Worker "+r+" sending request to shut down");clearInterval(QOper8Worker.timer),postMessage({qoper8:{shutdown:!0}}),i.emit("shutdown_signal_sent")},c=function(e){(e=e||{}).qoper8||(e.qoper8={}),e.qoper8.finished=!0,postMessage(e),i.emit("finished",e),o=!1,s&&h()};this.onMessage=function(f){let m;if(d=Date.now(),o=!0,f.qoper8&&f.qoper8.init&&void 0!==f.qoper8.id)return t?(m="QOper8 Worker "+r+" has already been initialised",i.emit("error",m),c({error:m,originalMessage:f})):(r=f.qoper8.id,n=f.qoper8.uuid,f.qoper8.workerInactivityCheckTime&&(a=f.qoper8.workerInactivityCheckTime),f.qoper8.workerInactivityLimit&&(l=f.qoper8.workerInactivityLimit),f.qoper8.handlersByMessageType&&(g=f.qoper8.handlersByMessageType),i.logging=f.qoper8.logging,p=setInterval(function(){let e=Date.now()-d;i.log("Worker "+r+" inactive for "+e),i.log("Inactivity limit: "+l),e>l&&(o?(i.log("Worker "+r+" flagged for termination"),s=!0):h())},a),i.log("new worker "+r+" started..."),i.emit("started",{id:r}),t=!0,c());if(!f.qoper8||!f.qoper8.uuid)return m="Invalid message sent to QOper8 Worker "+r,i.emit("error",m),c({error:m,originalMessage:f});if(f.qoper8.uuid!==n)return m="Invalid UUID on message sent to QOper8 Worker "+r,i.emit("error",m),c({error:m,originalMessage:f});let u=JSON.parse(JSON.stringify(f));if(delete f.qoper8.uuid,delete u.qoper8,i.log("Message received by worker "+r+": "+JSON.stringify(u,null,2)),i.emit("received",{message:u}),!f.type&&!f.handlerUrl)return m="No type or handler specified in message sent to worker "+r,i.emit("error",m),c({error:m,originalMessage:u});if(!f.type||!g.has(f.type))return m="No handler for messages of type "+f.type,i.log(m),i.emit("error",m),c({error:m,originalMessage:u});if(!e.has(f.type)){let t=g.get(f.type);i.log("fetching "+t);try{importScripts(t);let o=self.handler;e.set(f.type,o),i.emit("handler_imported",{handlerUrl:t})}catch(e){return m="Unable to load Handler Url "+t,i.log(m),i.log(JSON.stringify(e,null,2)),i.emit("error",m),c({error:m,originalMessage:u,workerId:r})}}e.get(f.type).call(i,f,c)}}log(e){this.logging&&console.log(Date.now()+": "+e)}on(e,r){this.listeners.has(e)||this.listeners.set(e,r)}off(e){this.listeners.has(e)&&this.listeners.delete(e)}emit(e,r){if(this.listeners.has(e)){this.listeners.get(e).call(this,r)}}},QOper8Worker=new QWorker;onmessage=async function(e){QOper8Worker.onMessage(e.data)};';class QOper8{constructor(e){(e=e||{}).workerInactivityCheckInterval&&(e.workerInactivityCheckInterval=1e3*e.workerInactivityCheckInterval),e.workerInactivityLimit&&(e.workerInactivityLimit=6e4*e.workerInactivityLimit),this.name="QOper8",this.build="2.4",this.buildDate="7 August 2022",this.logging=e.logging||!1,this.poolSize=e.poolSize||1,this.worker={inactivityCheckInterval:e.workerInactivityCheckInterval||6e4,inactivityLimit:e.workerInactivityLimit||12e5},this.handlersByMessageType=e.handlersByMessageType||new Map,this.listeners=new Map;let r="https:"===window.location.protocol?([1e7]+-1e3+-4e3+-8e3+-1e11).replace(/[018]/g,e=>(e^crypto.getRandomValues(new Uint8Array(1))[0]&15>>e/4).toString(16)):"xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx".replace(/[xy]/g,function(e){let r=16*Math.random()|0,i="x"==e?r:3&r|8;return i.toString(16)}),i=new Map,t=new Map,o=new Map,n=[],s=0,a=this;function l(){if(a.log("try processing queue: length "+n.length),0===n.length)return void a.log("Queue empty");let e=function(){for(const[e,r]of i){if(r.id=e,t.get(+r.id))return r;a.log("worker "+e+" is not available")}return!1}();e?(a.log("worker "+e.id+" was available. Sending message to it"),function(e){if(0===n.length)return;let r=n.shift(),i=e.id;o.set(i,r.qoper8.callback),delete r.qoper8.callback,t.set(+i,!1),g(r,e),a.emit("sentToWorker",{message:r,workerId:i})}(e)):(a.log("no available workers"),i.size<a.poolSize&&(a.log("starting new worker"),function(){let e,r=a.createUrl(workerCode);r?e=new Worker(r):(e={}).postMessage=function(e){};e.onmessage=function(r){let n=r.data,s=JSON.parse(JSON.stringify(n));if(delete s.qoper8,a.emit("replyReceived",{reply:s,workerId:e.id}),a.log("response received from Worker: "+e.id),a.log(JSON.stringify(s,null,2)),o.has(e.id)){let r=o.get(e.id);r&&r(n),o.delete(e.id)}n.qoper8&&(n.qoper8.finished?(t.set(+e.id,!0),l()):n.qoper8.shutdown&&(a.log("Master shutting down worker "+e.id),i.delete(e.id),a.emit("workerTerminated",e.id),e.terminate()))},e.id=s++,g({qoper8:{init:!0,id:e.id,handlersByMessageType:a.handlersByMessageType,workerInactivityCheckTime:a.worker.inactivityCheckTime,workerInactivityLimit:a.worker.inactivityLimit,logging:a.logging}},e),i.set(e.id,e),a.emit("workerStarted",e.id)}()))}function g(e,i){e.qoper8||(e.qoper8={}),e.qoper8.uuid=r,i.postMessage(e)}this.addToQueue=function(e){n.push(e),a.emit("addedToQueue",e),l()}}log(e){this.logging&&console.log(Date.now()+": "+e)}message(e,r){e.qoper8||(e.qoper8={}),e.qoper8.callback=r||!1,this.addToQueue(e)}send(e){let r=this;return new Promise(i=>{r.message(e,function(e){i(e)})})}createUrl(e){let r,i;try{r=new Blob([e],{type:"application/javascript"})}catch(i){try{let i=new(window.BlobBuilder||window.WebKitBlobBuilder||window.MozBlobBuilder);i.append(e),r=i.getBlob("application/javascript")}catch(e){return!1}}try{i=(window.URL||window.webkitURL).createObjectURL(r)}catch(e){i=!1}return i}on(e,r){this.listeners.has(e)||this.listeners.set(e,r)}off(e){this.listeners.has(e)&&this.listeners.delete(e)}emit(e,r){if(this.listeners.has(e)){this.listeners.get(e).call(this,r)}}}export{QOper8};
+let workerCode='let QWorker=class{constructor(){this.logging=!1;let e=new Map,r=new Map,t=!1,o=!1,i=this,n=!1,s=!1,a=!1,l=6e4,g=18e4,p=new Map,d=!1,f=Date.now(),h=0,u=function(){i.log("Worker "+t+" sending request to shut down");clearInterval(QOper8Worker.timer),postMessage({qoper8:{shutdown:!0}}),i.emit("shutdown_signal_sent")},c=function(e){(e=e||{}).qoper8||(e.qoper8={}),e.qoper8.finished=!0,postMessage(e),i.emit("finished",e),n=!1,s&&u()};this.getMessageCount=function(){return h},this.on=function(r,t){e.has(r)||e.set(r,t)},this.off=function(r){e.has(r)&&e.delete(r)},this.emit=function(r,t){if(e.has(r)){e.get(r).call(this,t)}},this.onMessage=function(e){let m;if(f=Date.now(),n=!0,e.qoper8&&e.qoper8.init&&void 0!==e.qoper8.id)return o?(m="QOper8 Worker "+t+" has already been initialised",i.emit("error",m),c({error:m,originalMessage:e})):(t=e.qoper8.id,a=e.qoper8.uuid,e.qoper8.workerInactivityCheckInterval&&(l=e.qoper8.workerInactivityCheckInterval),e.qoper8.workerInactivityLimit&&(g=e.qoper8.workerInactivityLimit),e.qoper8.handlersByMessageType&&(p=e.qoper8.handlersByMessageType),i.logging=e.qoper8.logging,d=setInterval(function(){let e=Date.now()-f;i.log("Worker "+t+" inactive for "+e),i.log("Inactivity limit: "+g),e>g&&(n?(i.log("Worker "+t+" flagged for termination"),s=!0):u())},l),i.log("new worker "+t+" started..."),i.emit("started",{id:t}),o=!0,c());if(!o)return m="QOper8 Worker "+t+" has not been initialised",i.emit("error",m),c({error:m,originalMessage:e});if(!e.qoper8||!e.qoper8.uuid)return m="Invalid message sent to QOper8 Worker "+t,i.emit("error",m),c({error:m,originalMessage:e});if(e.qoper8.uuid!==a)return m="Invalid UUID on message sent to QOper8 Worker "+t,i.emit("error",m),c({error:m,originalMessage:e});let y=JSON.parse(JSON.stringify(e));if(delete e.qoper8.uuid,delete y.qoper8,i.log("Message received by worker "+t+": "+JSON.stringify(y,null,2)),i.emit("received",{message:y}),"qoper8_terminate"!==e.type){if(!e.type&&!e.handlerUrl)return m="No type or handler specified in message sent to worker "+t,i.emit("error",m),c({error:m,originalMessage:y});if(!e.type||!p.has(e.type))return m="No handler for messages of type "+e.type,i.log(m),i.emit("error",m),c({error:m,originalMessage:y});if(!r.has(e.type)){let o=p.get(e.type);i.log("fetching "+o);try{importScripts(o);let n=self.handler;r.set(e.type,n),i.emit("handler_imported",{handlerUrl:o})}catch(e){return m="Unable to load Handler Url "+o,i.log(m),i.log(JSON.stringify(e,null,2)),i.emit("error",m),c({error:m,originalMessage:y,workerId:t})}}h++,r.get(e.type).call(i,e,c)}else u()}}log(e){this.logging&&console.log(Date.now()+": "+e)}},QOper8Worker=new QWorker;onmessage=async function(e){QOper8Worker.onMessage(e.data)};';class QOper8{constructor(e){(e=e||{}).workerInactivityCheckInterval&&(e.workerInactivityCheckInterval=1e3*e.workerInactivityCheckInterval),e.workerInactivityLimit&&(e.workerInactivityLimit=6e4*e.workerInactivityLimit),this.name="QOper8",this.build="2.5",this.buildDate="15 August 2022",this.logging=e.logging||!1;let r=+e.poolSize||1,t=e.maxPoolSize||32;r>t&&(r=t);let i=e.disabled||!1,o=e.workerInactivityCheckInterval||6e4,n=e.workerInactivityLimit||12e5;this.handlersByMessageType=e.handlersByMessageType||new Map;let a=new Map,s="https:"===window.location.protocol?([1e7]+-1e3+-4e3+-8e3+-1e11).replace(/[018]/g,e=>(e^crypto.getRandomValues(new Uint8Array(1))[0]&15>>e/4).toString(16)):"xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx".replace(/[xy]/g,function(e){let r=16*Math.random()|0;return("x"==e?r:3&r|8).toString(16)}),l=new Map,g=new Map,p=new Map,d=[],c=0,u=!1,h=0,m=this;function f(){if(m.log("try processing queue: length "+d.length),0===d.length)return void m.log("Queue empty");let e=function(){for(const[e,r]of l){if(r.id=e,g.get(+r.id))return r;m.log("worker "+e+" is not available")}return!1}();e?(m.log("worker "+e.id+" was available. Sending message to it"),function(e){if(0===d.length)return;let r=d.shift(),t=e.id;p.set(t,r.qoper8.callback),delete r.qoper8.callback,g.set(+t,!1),w(r,e),m.emit("sentToWorker",{message:r,workerId:t})}(e)):(m.log("no available workers"),l.size<r&&(m.log("starting new worker"),function(){let e,r=m.createUrl(workerCode);r?e=new Worker(r):(e={}).postMessage=function(e){};e.onmessage=function(r){let t=r.data,i=JSON.parse(JSON.stringify(t));if(delete i.qoper8,m.emit("replyReceived",{reply:i,workerId:e.id}),m.log("response received from Worker: "+e.id),m.log(JSON.stringify(i,null,2)),p.has(e.id)){let r=p.get(e.id);r&&r(t,e.id),p.delete(e.id)}t.qoper8&&(t.qoper8.finished?(g.set(+e.id,!0),m.emit("worker"+e.id+"Available"),u||f()):t.qoper8.shutdown&&(m.log("Master shutting down worker "+e.id),l.delete(e.id),m.emit("workerTerminated",e.id),m.emit("worker"+e.id+"Terminated"),e.terminate()))},e.id=c++,w({qoper8:{init:!0,id:e.id,handlersByMessageType:m.handlersByMessageType,workerInactivityCheckInterval:o,workerInactivityLimit:n,logging:m.logging}},e),l.set(e.id,e),m.emit("workerStarted",e.id)}()))}function w(e,r){e.qoper8||(e.qoper8={}),e.qoper8.uuid=s,r.postMessage(e)}function y(e){return new Promise(r=>{m.on("worker"+e+"Available",function(){m.off("worker"+e+"Available"),r()})})}function k(e){return new Promise(r=>{m.on("worker"+e+"Terminated",function(){m.off("worker"+e+"Terminated"),r()})})}this.log=function(e){!i&&this.logging&&console.log(Date.now()+": "+e)},this.getMessageCount=function(){return h},this.on=function(e,r){a.has(e)||a.set(e,r)},this.off=function(e){a.has(e)&&a.delete(e)},this.emit=function(e,r){if(a.has(e)){a.get(e).call(m,r)}},this.getQueueLength=function(){return d.length},this.message=function(e,r){e.qoper8||(e.qoper8={}),e.qoper8.callback=r||!1,function(e){u?e.qoper8&&e.qoper8.callback&&e.qoper8.callback({error:"QOper8 has been stopped"}):(h++,d.push(e),m.emit("addedToQueue",e),f())}(e)},this.stop=async function(){u=!0;for(const[e,r]of l)if(g.get(+e)){m.log("Web Worker "+e+" is being stopped"),w({type:"qoper8_terminate"},r),await k(e),m.log("Worker Thread "+e+" has been stopped")}else{m.log("Waiting for Worker Thread "+e+" to become available"),await y(e),w({type:"qoper8_terminate"},r),await k(e),m.log("Worker Thread "+e+" has been stopped")}m.emit("stop"),m.log("No Worker Threads are running.  QOper8 is no longer handling messages")},this.start=function(){u=!1,m.log("QOper8 is started and will handle messages"),f()}}send(e){let r=this;return new Promise(t=>{r.message(e,function(e){t(e)})})}createUrl(e){let r,t;try{r=new Blob([e],{type:"application/javascript"})}catch(t){try{let t=new(window.BlobBuilder||window.WebKitBlobBuilder||window.MozBlobBuilder);t.append(e),r=t.getBlob("application/javascript")}catch(e){return!1}}try{t=(window.URL||window.webkitURL).createObjectURL(r)}catch(e){t=!1}return t}}export{QOper8};

src/QOper8Worker.js

@@ -26,3 +26,3 @@ /*
 
-7 August 2022
+15 August 2022
 
@@ -33,5 +33,5 @@ */
   constructor() {
-    this.listeners = new Map();
     this.logging = false;
 
+    let listeners = new Map();
     let handlers = new Map();
@@ -49,2 +49,3 @@ let id = false;
     let lastActivityAt = Date.now();
+    let noOfMessages = 0;
 
@@ -94,2 +95,25 @@ let shutdown = function() {
 
+    this.getMessageCount = function() {
+      return noOfMessages;
+    };
+
+    this.on = function(type, callback) {
+      if (!listeners.has(type)) {
+        listeners.set(type, callback);
+      }
+    };
+
+    this.off = function(type) {
+      if (listeners.has(type)) {
+        listeners.delete(type);
+      }
+    };
+
+    this.emit = function(type, data) {
+      if (listeners.has(type)) {
+        let handler =  listeners.get(type);
+        handler.call(this, data);
+      }
+    };
+
     this.onMessage = function(obj) {
@@ -115,7 +139,8 @@
         uuid = obj.qoper8.uuid;
-        if (obj.qoper8.workerInactivityCheckTime) delay = obj.qoper8.workerInactivityCheckTime; 
-        if (obj.qoper8.workerInactivityLimit) inactivityLimit = obj.qoper8.workerInactivityLimit; 
+        if (obj.qoper8.workerInactivityCheckInterval) delay = obj.qoper8.workerInactivityCheckInterval; 
+        if (obj.qoper8.workerInactivityLimit) inactivityLimit = obj.qoper8.workerInactivityLimit;
         if (obj.qoper8.handlersByMessageType) {
           handlersByMessageType = obj.qoper8.handlersByMessageType;
         }
+
         q.logging = obj.qoper8.logging;
@@ -131,2 +156,11 @@ startTimer();
 
+      if (!initialised) {
+        error = 'QOper8 Worker ' + id + ' has not been initialised';
+        q.emit('error', error);
+        return finished({
+          error: error,
+          originalMessage: obj
+        });
+      }
+
       if (!obj.qoper8 || !obj.qoper8.uuid) {
@@ -156,2 +190,7 @@ error = 'Invalid message sent to QOper8 Worker ' + id;
 
+      if (obj.type === 'qoper8_terminate') {
+        shutdown();
+        return;
+      }
+
       if (!obj.type && !obj.handlerUrl) {
@@ -188,2 +227,3 @@ error = 'No type or handler specified in message sent to worker ' + id;
         }
+        noOfMessages++;
         let handler = handlers.get(obj.type);
@@ -209,21 +249,2 @@ handler.call(q, obj, finished);
   }
-
-  on(type, callback) {
-    if (!this.listeners.has(type)) {
-      this.listeners.set(type, callback);
-    }
-  }
-
-  off(type) {
-    if (this.listeners.has(type)) {
-      this.listeners.delete(type);
-    }
-  }
-
-  emit(type, data) {
-    if (this.listeners.has(type)) {
-      let handler =  this.listeners.get(type);
-      handler.call(this, data);
-    }
-  }
 };
@@ -230,0 +251,0 @@

New alerts

License Policy Violation

License

This package is not allowed per your license policy. Review the package's license to ensure compliance.

Found 1 instance in 1 package

Fixed alerts

License Policy Violation

License

This package is not allowed per your license policy. Review the package's license to ensure compliance.

Found 1 instance in 1 package

Improved metrics

Total package byte prevSize

65020

increased by25.4%

Number of package files

14

increased by16.67%

Lines of code

760

increased by28.81%

Number of lines in readme file

577

increased by13.81%

No dependency changes