Security News
ESLint is Now Language-Agnostic: Linting JSON, Markdown, and Beyond
ESLint has added JSON and Markdown linting support with new officially-supported plugins, expanding its versatility beyond JavaScript.
node-oom-heapdump
Advanced tools
Create a V8 heap snapshot right before an "Out of Memory" error occurs, or create a heap snapshot on request.
Node module which will create a V8 heap snapshot right before an "Out of Memory" error occurs.
It can also create heapdumps and CPU profiles on request like 'v8-profiler', but does this off-process so it doesn't interfere with execution of the main process.
Tested on Node.js 8.x, but should also work fine using Node.js 6.3 upwards (According to: https://chromedevtools.github.io/devtools-protocol/v8/).
When running nodejs processes in a low memory environment, every out of memory that occurs is interesting. To figure out why a process went out of memory, a heap snapshot (e.g. heapdump) can help a lot. This module creates a heap snapshot right before a suspected out of memory error occurs. It shows what the heap was filled with right before the out of memory error occured.
There are several modules around which can create heapdumps (v8-profiler, node-heapdump), but these run in the same process as the one going out of memory. Often, creating heapdump won't work when the node process is already struggling. This module creates the heap snapshot from a separate process, which solves this issue.
It uses 'gc-stats' to determine when an out of memory error is about to occur and then fires up a new process which uses 'chrome-remote-interface' to connect with the DevTools protocol (https://chromedevtools.github.io/devtools-protocol/v8/) of the calling process. That process uses HeapProfiler to actually create the heapdump and then exits.
Just run "npm test" to see it in action. It creates a heapdump named "my_heapdump.heapsnapshot" in the 'tests' directory of this module.
npm install node-oom-heapdump
Just add the following snippet to your node process.
let path = require('path');
require('node-oom-heapdump')({
threshold: 70,
path: path.resolve(__dirname, 'my_heapdump')
});
Your node process should at least be started with the "--inspect" (or --inspect=port) flag. When the module is loaded, the configured port is verified. If it doesn't respond correctly, a console warning will be shown.
When running in a low memory environment, the following flags are advised:
These might impact performance though.
Besides creating heapdumps when an out of memory error occurs, there also is an API for creating heapdumps and CPU profiles on request. See below for the currently available API.
Notice that you cannot create a heapdump while a CPU profile is being generated and vice versa; an Error will be thrown if this is the case.
let nodeOomHeapdump = require("node-oom-heapdump")({
heapdumpOnOOM: false
});
/**
* Returns the path to the created heap snapshot in a promise, or rejects on error
* @param {String} snapshotPath - path of the snapshot
* @return {Promise} Promise containing the heap snapshot path on success or error on rejection
*/
nodeOomHeapdump.createHeapSnapshot("myheapsnapshotpath").then((snapshotPath) => {
// do something with heap snapshot
// and delete again from disk
nodeOomHeapdump.deleteHeapSnapshot(snapshotPath);
}).catch((err) => {
// handle error
});
/**
* Deletes all previously created heapsnapshots from disk
*/
nodeOomHeapdump.deleteAllHeapSnapshots();
/**
* Deletes a particular snapshot from disk
* @param {String} snapshotPath - path of the heap snapshot to delete
* @return {Promise}
*/
nodeOomHeapdump.deleteHeapSnapshot(snapshotPath);
/**
* Returns the path to the created CPU profile in a promise, or rejects on error
* @param {String} cpuProfilePath - path of the CPU profile
* @param {number} duration - the duration of the CPU profile in ms (default: 30000ms)
* @return {Promise} the CPU profile path on success or error on rejection
*/
nodeOomHeapdump.createCpuProfile("mycpuprofilepath", 10000).then((cpuProfilePath) => {
// do something with CPU profile
// and delete again from disk
nodeOomHeapdump.deleteCpuProfile(cpuProfilePath);
}).catch((err) => {
// handle error
});
/**
* Deletes all previously created CPU profiles from disk
*/
nodeOomHeapdump.deleteAllCpuProfiles();
/**
* Deletes a particular CPU profile from disk
* @param {String} cpuProfilePath - path to the CPU profile to delete from disk
* @return {Promise}
*/
nodeOomHeapdump.deleteCpuProfile(cpuProfilePath);
When the OoM heapdump process kicks in, it's notorious for using a lot of memory. This is caused by a bug in V8/DevTools protocol and is reported here (https://bugs.chromium.org/p/chromium/issues/detail?id=768355); the protocol has no backpressure mechanism, which causes the heapdump to be pushed faster than the DevTools client can handle, causing in-memory buffering.
This is not a problem if your server/machine has memory to spare, but can cause issues in memory restricted environments like a Docker container. Once the process exceeds the container memory threshold, it will be killed by OoMKiller (if enabled). This leads to an empty heapsnapshot file (0 bytes).
Please vote for that issue to be fixed!
As mentioned in https://github.com/blueconic/node-oom-heapdump/issues/1, this module cannot handle volatile memory growth caused by large allocations at once. This limitation is caused by the way this module works:
Since this module relies on monitoring GC activity, this issue cannot be fixed. It would only be fixable if Node.js would provide some kind of hook before an Out of Memory occurs (described here: https://github.com/node-inspector/v8-profiler/issues/109).
FAQs
Create a V8 heap snapshot when an "Out of Memory" error occurs, or create a heap snapshot or CPU profile on request.
The npm package node-oom-heapdump receives a total of 942 weekly downloads. As such, node-oom-heapdump popularity was classified as not popular.
We found that node-oom-heapdump demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
ESLint has added JSON and Markdown linting support with new officially-supported plugins, expanding its versatility beyond JavaScript.
Security News
Members Hub is conducting large-scale campaigns to artificially boost Discord server metrics, undermining community trust and platform integrity.
Security News
NIST has failed to meet its self-imposed deadline of clearing the NVD's backlog by the end of the fiscal year. Meanwhile, CVE's awaiting analysis have increased by 33% since June.