Security News
Research
Data Theft Repackaged: A Case Study in Malicious Wrapper Packages on npm
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
thread-loader
Advanced tools
The thread-loader package is used to offload expensive loaders to a worker pool. It can be particularly useful when dealing with resource-intensive loaders in a webpack build process, as it can help to speed up compilation by parallelizing the work.
Offloading Loaders to Worker Pool
This feature allows you to offload loaders like 'babel-loader' to a worker pool. The code sample shows how to use 'thread-loader' in a webpack configuration file to process JavaScript files with 'babel-loader' in a separate thread.
module.exports = {
module: {
rules: [
{
test: /\.js$/,
include: path.resolve('src'),
use: [
'thread-loader',
'babel-loader'
]
}
]
}
};
Custom Worker Pool
This feature allows you to specify the number of workers in the pool. The code sample demonstrates how to set up a custom worker pool with two workers using 'thread-loader' options.
module.exports = {
module: {
rules: [
{
test: /\.js$/,
include: path.resolve('src'),
use: [
{
loader: 'thread-loader',
options: {
workers: 2
}
},
'babel-loader'
]
}
]
}
};
Worker Pool Warm-up
This feature allows you to warm up the worker pool before running the actual loaders. The code sample shows how to use 'thread-loader' to pre-load 'babel-loader' and '@babel/preset-env' to the worker pool.
const threadLoader = require('thread-loader');
const warmupOptions = {
// pool options, like passed to loader options
// must match loader options to boot the correct pool
};
threadLoader.warmup(warmupOptions, [
// modules to load
// can be any module, i. e.
'babel-loader',
'@babel/preset-env'
]);
HappyPack is a package similar to thread-loader that also enables parallel processing of files in webpack. It works by transforming the files in parallel using worker threads and then compiling them together. Compared to thread-loader, HappyPack provides more detailed configuration options but is no longer actively maintained.
Parallel-webpack allows you to run multiple instances of webpack in parallel, which can significantly speed up the build process. It differs from thread-loader in that it parallelizes the entire build process rather than individual loaders. This can be more efficient for large projects with many entry points.
Cache-loader is a webpack loader that caches the result of expensive loader functions on disk. While it doesn't parallelize work like thread-loader, it can speed up subsequent builds by reusing previous results, thus reducing the need to run the loaders again.
Runs the following loaders in a worker pool.
npm install --save-dev thread-loader
or
yarn add -D thread-loader
or
pnpm add -D thread-loader
Put this loader in front of other loaders. The following loaders run in a worker pool.
Loaders running in a worker pool are limited. Examples:
Each worker is a separate node.js process, which has an overhead of ~600ms. There is also an overhead of inter-process communication.
Use this loader only for expensive operations!
webpack.config.js
module.exports = {
module: {
rules: [
{
test: /\.js$/,
include: path.resolve('src'),
use: [
'thread-loader',
// your expensive loader (e.g babel-loader)
],
},
],
},
};
with options
use: [
{
loader: 'thread-loader',
// loaders with equal options will share worker pools
options: {
// the number of spawned workers, defaults to (number of cpus - 1) or
// fallback to 1 when require('os').cpus() is undefined
workers: 2,
// number of jobs a worker processes in parallel
// defaults to 20
workerParallelJobs: 50,
// additional node.js arguments
workerNodeArgs: ['--max-old-space-size=1024'],
// Allow to respawn a dead worker pool
// respawning slows down the entire compilation
// and should be set to false for development
poolRespawn: false,
// timeout for killing the worker processes when idle
// defaults to 500 (ms)
// can be set to Infinity for watching builds to keep workers alive
poolTimeout: 2000,
// number of jobs the poll distributes to the workers
// defaults to 200
// decrease of less efficient but more fair distribution
poolParallelJobs: 50,
// name of the pool
// can be used to create different pools with elsewise identical options
name: 'my-pool',
},
},
// your expensive loader (e.g babel-loader)
];
prewarming
To prevent the high delay when booting workers it possible to warmup the worker pool.
This boots the max number of workers in the pool and loads specified modules into the node.js module cache.
const threadLoader = require('thread-loader');
threadLoader.warmup(
{
// pool options, like passed to loader options
// must match loader options to boot the correct pool
},
[
// modules to load
// can be any module, i. e.
'babel-loader',
'babel-preset-es2015',
'sass-loader',
],
);
Please take a moment to read our contributing guidelines if you haven't yet done so.
FAQs
Runs the following loaders in a worker pool
The npm package thread-loader receives a total of 1,304,177 weekly downloads. As such, thread-loader popularity was classified as popular.
We found that thread-loader demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 3 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Research
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
Research
Security News
Attackers used a malicious npm package typosquatting a popular ESLint plugin to steal sensitive data, execute commands, and exploit developer systems.
Security News
The Ultralytics' PyPI Package was compromised four times in one weekend through GitHub Actions cache poisoning and failure to rotate previously compromised API tokens.