@isoftdata/universal-object-htp-utility

Memory Leak Detection for inventory.js

This document provides instructions on how to use the memory leak detection test script to identify potential memory leaks in the inventory.js module.

Overview

The inventory-memory-test.js script performs load testing on the main functions in inventory.js to detect memory leaks by:

• Running operations with different batch sizes
• Executing multiple iterations to observe memory usage patterns
• Logging memory usage before and after each operation
• Analyzing the results to identify potential memory leaks

Running the Test

Basic Usage

# Navigate to the project root
cd universal-object-htp-utility

# Run the test with garbage collection enabled
node --expose-gc tests/inventory-memory-test.js

Advanced Options

For more detailed garbage collection information:

node --expose-gc --trace-gc tests/inventory-memory-test.js

For heap snapshots (requires additional setup):

node --inspect tests/inventory-memory-test.js

Then open Chrome and navigate to chrome://inspect to connect to the Node process and use the Memory tab to take heap snapshots.

Understanding the Results

The test generates two types of output:

• Console Output: Shows real-time memory usage and a summary analysis
• CSV File: memory-usage.csv contains detailed memory metrics for further analysis

Interpreting the Analysis

The script automatically analyzes the results and looks for:

• Consistent Memory Growth: If memory usage (especially heapUsed) consistently increases across iterations with the same batch size, it suggests a memory leak.

• Memory Difference: The average difference between memory usage before and after operations. Large positive values indicate memory not being properly released.

• Growth Patterns: Operations showing a high percentage of growth across iterations are flagged as potential leak sources.

Memory Leak Patterns

Different memory leak patterns will show up in different ways:

• Linear Growth: A steady increase in memory usage across iterations indicates a classic memory leak where objects aren't being garbage collected.

• Stair-Step Pattern: Memory increases during operations but doesn't fully return to baseline after GC, creating a stair-step pattern. This suggests partial leaks or retained references.

• Operation-Specific Spikes: If memory spikes occur consistently with specific operations, focus your investigation on those functions.

Customizing the Test

You can modify the test script to:

• Change the number of iterations (iterations variable)
• Adjust the batch sizes (batchSizes array)
• Add more operations to test
• Modify the test data generation to match your specific use cases

Fixing Common Memory Leaks

If the test identifies potential memory leaks, here are common areas to investigate:

• Database Connections: Ensure all connections are properly released, especially in error cases.

• Event Listeners: Check if any event listeners are not properly removed.

• Closures: Look for closures that might retain references to large objects.

• Caching: Check for objects being cached without proper cleanup mechanisms.

• Circular References: Look for circular references that might prevent garbage collection.

Example: Fixing a Connection Leak

If you identify a connection leak in the transaction handling, you might fix it like this:

// Before
try {
  const connectionWithTransaction = await db.getTransactionFromPool(htpConnection);
  // ... operations
  await db.commitAndRelease(connectionWithTransaction);
} catch (err) {
  // Missing connection release in some error paths
  throw err;
}

// After
let connectionWithTransaction;
try {
  connectionWithTransaction = await db.getTransactionFromPool(htpConnection);
  // ... operations
  await db.commitAndRelease(connectionWithTransaction);
  connectionWithTransaction = null; // Clear reference
} catch (err) {
  if (connectionWithTransaction) {
    await db.rollbackAndRelease(connectionWithTransaction);
    connectionWithTransaction = null; // Clear reference
  }
  throw err;
}

Additional Tools

For more comprehensive memory analysis, consider using:

• clinic.js: A suite of tools for Node.js diagnostics

  npm install -g clinic
  clinic doctor -- node tests/inventory-memory-test.js
  

• memwatch-next: Detects memory leaks by watching for garbage collection events

  npm install memwatch-next
  

  Then modify the test to use memwatch for leak detection.

• heapdump: Generate heap snapshots programmatically

  npm install heapdump