thread-loader

What is thread-loader?

The thread-loader package is used to offload expensive loaders to a worker pool. It can be particularly useful when dealing with resource-intensive loaders in a webpack build process, as it can help to speed up compilation by parallelizing the work.

What are thread-loader's main functionalities?

Offloading Loaders to Worker Pool

This feature allows you to offload loaders like 'babel-loader' to a worker pool. The code sample shows how to use 'thread-loader' in a webpack configuration file to process JavaScript files with 'babel-loader' in a separate thread.

module.exports = {
  module: {
    rules: [
      {
        test: /\.js$/,
        include: path.resolve('src'),
        use: [
          'thread-loader',
          'babel-loader'
        ]
      }
    ]
  }
};

Custom Worker Pool

This feature allows you to specify the number of workers in the pool. The code sample demonstrates how to set up a custom worker pool with two workers using 'thread-loader' options.

module.exports = {
  module: {
    rules: [
      {
        test: /\.js$/,
        include: path.resolve('src'),
        use: [
          {
            loader: 'thread-loader',
            options: {
              workers: 2
            }
          },
          'babel-loader'
        ]
      }
    ]
  }
};

Worker Pool Warm-up

This feature allows you to warm up the worker pool before running the actual loaders. The code sample shows how to use 'thread-loader' to pre-load 'babel-loader' and '@babel/preset-env' to the worker pool.

const threadLoader = require('thread-loader');

const warmupOptions = {
  // pool options, like passed to loader options
  // must match loader options to boot the correct pool
};

threadLoader.warmup(warmupOptions, [
  // modules to load
  // can be any module, i. e.
  'babel-loader',
  '@babel/preset-env'
]);

Other packages similar to thread-loader

happypack

HappyPack is a package similar to thread-loader that also enables parallel processing of files in webpack. It works by transforming the files in parallel using worker threads and then compiling them together. Compared to thread-loader, HappyPack provides more detailed configuration options but is no longer actively maintained.

parallel-webpack

Parallel-webpack allows you to run multiple instances of webpack in parallel, which can significantly speed up the build process. It differs from thread-loader in that it parallelizes the entire build process rather than individual loaders. This can be more efficient for large projects with many entry points.

cache-loader

Cache-loader is a webpack loader that caches the result of expensive loader functions on disk. While it doesn't parallelize work like thread-loader, it can speed up subsequent builds by reusing previous results, thus reducing the need to run the loaders again.

README

thread-loader

Runs the following loaders in a worker pool.

Install

npm install --save-dev thread-loader

Usage

Put this loader in front of other loaders. The following loaders run in a worker pool.

Loaders running in a worker pool are limited. Examples:

• Loaders cannot emit files.
• Loaders cannot use custom loader API (i. e. by plugins).
• Loaders cannot access the webpack options.

Each worker is separate node.js process, which has an overhead of ~600ms. There is also an overhead of inter-process communication.

Use this loader only for expensive operations!

Examples

webpack.config.js

module.exports = {
  module: {
    rules: [
      {
        test: /\.js$/,
        include: path.resolve("src"),
        use: [
          "thread-loader",
          "expensive-loader"
        ]
      }
    ]
  }
}

with options

use: [
  {
    loader: "thread-loader",
    options: {
      // the number of spawned workers, defaults to number of cpus
      workers: 2,

      // number of jobs a worker processes in parallel
      // defaults to 20
      workerParallelJobs: 50,

      // additional node.js arguments
      workerNodeArgs: ['--max-old-space-size', '1024'],

      // timeout for killing the worker processes when idle
      // defaults to 500 (ms)
      // can be set to Infinity for watching builds to keep workers alive
      poolTimeout: 2000,

      // number of jobs the poll distributes to the workers
      // defaults to 200
      // decrease of less efficient but more fair distribution
      poolParallelJobs: 50
    }
  },
  "expensive-loader"
]

Maintainers

sokra

Changelog

1.0.3 (2017-05-27)

Bug Fixes

• resolve: fix passing error to worker (6561f57)

<a name="1.0.2"></a>