gatsby-worker
Utility to execute tasks in forked processes. Highly inspired by jest-worker
.
Example
File worker.ts
:
export async function heavyTask(param: string): Promise<string> {
return await heavyProcessing(param)
}
export async function setupStep(param: string): Promise<void> {
await heavySetup(param)
}
File parent.ts
:
import { WorkerPool } from "gatsby-worker"
const workerPool = new WorkerPool<typeof import("./worker")>(
require.resolve(`./worker`),
{
numWorkers: 5,
env: {
CUSTOM_ENV_VAR_TO_SET_IN_WORKER: `foo`,
},
silent: false,
}
)
const arrayOfPromises = workerPool.all.setupStep(`bar`)
const singlePromise = workerPool.single.heavyTask(`baz`)
API
Constructor
const workerPool = new WorkerPool<TypeOfWorkerModule>(
workerPath: string,
options?: {
numWorkers?: number
env?: Record<string, string>,
silent?: boolean,
}
)
.single
const singlePromise = workerPool.single.heavyTask(`baz`)
.all
const arrayOfPromises = workerPool.all.setupStep(`baz`)
.end
const arrayOfPromises = workerPool.end()
isWorker
import { isWorker } from "gatsby-worker"
if (isWorker) {
} else {
}
Messaging
gatsby-worker
allows sending messages from worker to main and from main to worker at any time.
Sending messages from worker
File message-types.ts
:
interface IPingMessage {
type: `PING`
}
interface IAnotherMessageFromChild {
type: `OTHER_MESSAGE_FROM_CHILD`
payload: {
foo: string
}
}
export type MessagesFromChild = IPingMessage | IAnotherMessageFromChild
interface IPongMessage {
type: `PONG`
}
interface IAnotherMessageFromParent {
type: `OTHER_MESSAGE_FROM_PARENT`
payload: {
foo: string
}
}
export type MessagesFromParent = IPongMessage | IAnotherMessageFromParent
File worker.ts
:
import { getMessenger } from "gatsby-worker"
import { MessagesFromParent, MessagesFromChild } from "./message-types"
const messenger = getMessenger<MessagesFromParent, MessagesFromChild>()
if (messenger) {
messenger.send({ type: `PING` })
messenger.send({
type: `OTHER_MESSAGE_FROM_CHILD`,
payload: {
foo: `bar`,
},
})
messenger.onMessage(msg => {
switch (msg.type) {
case `PONG`: {
break
}
case `OTHER_MESSAGE_FROM_PARENT`: {
break
}
}
})
}
File parent.ts
:
import { getMessenger } from "gatsby-worker"
import { MessagesFromParent, MessagesFromChild } from "./message-types"
const workerPool = new WorkerPool<
typeof import("./worker"),
MessagesFromParent,
MessagesFromChild
>(
workerPath: require.resolve(`./worker`)
)
workerPool.sendMessage(
{
type: `OTHER_MESSAGE_FROM_PARENT`,
payload: {
foo: `baz`
}
},
1
)
workerPool.onMessage((msg: MessagesFromChild, workerId: number): void => {
switch(msg.type) {
case: `PING`: {
workerPool.sendMessage({ type: `PONG` }, workerId)
break
}
}
})
Usage with unit tests
If you are working with source files that need transpilation, you will need to make it possible to load untranspiled modules in child processes.
This can be done with @babel/register
(or similar depending on your build toolchain). Example setup:
const testWorkerPool = new WorkerPool<WorkerModuleType>(workerModule, {
numWorkers,
env: {
NODE_OPTIONS: `--require ${require.resolve(`./ts-register`)}`,
},
})
This will execute additional module before allowing adding runtime support for new JavaScript syntax or support for TypeScript. Example ts-register.js
:
require(`@babel/register`)({
extensions: [`.js`, `.ts`],
configFile: require.resolve(relativePathToYourBabelConfig),
ignore: [/node_modules/],
})