![PyPI Slashes Malware Response Time: 90% of Issues Resolved in Under 24 Hours](https://cdn.sanity.io/images/cgdhsj6q/production/eee3e3fc69826f43dfc3366b71472d981003a5cb-1024x1024.webp?w=400&fit=max&auto=format)
Security News
PyPI Slashes Malware Response Time: 90% of Issues Resolved in Under 24 Hours
PyPI has drastically improved its malware response times, resolving 90% of issues in under 24 hours and removing 900 projects since March 2024.
memoizee
Advanced tools
Complete memoize/cache solution. Works with any type and length of function arguments
Package description
The memoizee npm package is a complete memoization library for JavaScript. It allows for caching the results of function calls based on the arguments provided. This can significantly improve performance for expensive function calls that are invoked repeatedly with the same parameters.
Basic memoization
This feature allows you to memoize any function, caching its results so that subsequent calls with the same arguments return immediately without recomputing.
const memoize = require('memoizee');
const expensiveFunction = (arg) => { /* complex computation */ };
const memoized = memoize(expensiveFunction);
Memoization with options
This feature allows you to customize the memoization behavior with options such as maxAge for cache expiration, preFetch for renewing cache just before it expires, and max for limiting the number of cache entries.
const memoize = require('memoizee');
const memoized = memoize(expensiveFunction, { maxAge: 1000, preFetch: 0.5, max: 10 });
Primitive mode
Primitive mode ensures that the cache keys are based on the value of the arguments rather than their reference, which is useful for arguments that are primitive values.
const memoize = require('memoizee');
const memoized = memoize(expensiveFunction, { primitive: true });
Asynchronous function memoization
This feature allows memoization of asynchronous functions, caching the resolved value of the promise.
const memoize = require('memoizee');
const expensiveAsyncFunction = async (arg) => { /* complex async computation */ };
const memoized = memoize(expensiveAsyncFunction, { promise: true });
WeakMap-based memoization
This feature uses WeakMap to store cache entries, which allows for garbage collection of non-referenced keys, useful for memoizing functions that accept objects as arguments.
const memoize = require('memoizee/weak');
const memoized = memoize(expensiveFunction);
Lodash provides a memoize function that caches the result of function calls. It's simpler than memoizee and doesn't offer as many options for customization, but it's widely used due to its inclusion in the Lodash utility library.
Fast-memoize is a high-performance memoization library that claims to be the fastest memoization library for JavaScript. It offers a simple API but lacks some of the advanced features and customization options provided by memoizee.
This is a less known library that offers memoization capabilities. It provides a simple API for memoizing functions but does not have the extensive feature set and customization options that memoizee offers.
Readme
Originally derived from es5-ext package.
Memoization is best technique to save on memory or CPU cycles when we deal with repeated operations. For detailed insight see: http://en.wikipedia.org/wiki/Memoization
this
counts in)var memoize = require('memoizee');
var fn = function (one, two, three) { /* ... */ };
memoized = memoize(fn);
memoized('foo', 3, 'bar');
memoized('foo', 3, 'bar'); // Cache hit
In your project path:
$ npm install memoizee
Browser bundle can be easily created with help of modules-webmake. Assuming that you have latest Node.js and Git installed, following will work in command shell of any system (Linux/MacOS/Windows):
$ npm install -g webmake
$ git clone git://github.com/medikoo/memoize.git
$ cd memoize
$ npm install
$ cd ..
$ webmake --name=memoize memoize/lib/index.js memoize.js
Last command bundles memoize with all it's functionalities, but you may need just a subset, you can have that by addressing specific modules directly, e.g. with following you will build just primitive mode with support for asynchronous functions:
$ webmake --name=memoize --include=memoize/lib/ext/async.js memoize/lib/primitive.js memoize.js
If you work with AMD modules, add amd option, so generated bundle is one:
$ webmake --name=memoize --amd memoize/lib/index.js memoize.js
Mind that memoize relies on some EcmaScript5 features, so for older browsers you need to load as well es5-shim
All below options can be applied in any combination
By default fixed number of arguments that function take is assumed (it's read from function's length
property) this can be overridden:
memoized = memoize(fn, { length: 2 });
memoized('foo'); // Assumed: 'foo', undefined
memoized('foo', undefined); // Cache hit
memoized('foo', 3, {}); // Third argument is ignored (but passed to underlying function)
memoized('foo', 3, 13); // Cache hit
Dynamic length behavior can be forced by setting length to false
, that means memoize will work with any number of arguments.
memoized = memoize(fn, { length: false });
memoized('foo');
memoized('foo'); // Cache hit
memoized('foo', undefined);
memoized('foo', undefined); // Cache hit
memoized('foo', 3, {});
memoized('foo', 3, 13);
memoized('foo', 3, 13); // Cache hit
If we work with large result sets, or memoize hot functions, default mode may not perform as fast as we expect. In that case it's good to run memoization in primitive mode. To provide fast access, results are saved in hash instead of an array. Generated hash ids are result of arguments to string convertion. Mind that this mode will work correctly only if stringified arguments produce unique strings.
memoized = memoize(fn, { primitive: true });
memoized('/path/one');
memoized('/path/one'); // Cache hit
When not working in primitive mode but expecting arguments of certain type it's good to coerce them before doing memoization. We can do that by passing additional resolvers array:
memoized = memoize(fn, { length: 2, resolvers: [String, Boolean] });
memoized(12, [1,2,3].length);
memoized("12", true); // Cache hit
memoized({ toString: function () { return "12"; } }, {}); // Cache hit
Note. If your arguments are collections (arrays or hashes) that you want to memoize by content (not by self objects), you need to cast them to strings, for that just use primitive mode. Arrays have standard string representation and work with primitive mode out of a box, for hashes you need to define toString
method, that will produce unique string descriptions.
Similarly if you want to memoize functions by their code representation not by their objects, you should use primitive mode.
With async option we indicate that we memoize asynchronous function.
Operations that result with an error are not cached.
afn = function (a, b, cb) {
setTimeout(function () {
cb(null, a + b);
}, 200);
};
memoized = memoize(afn, { async: true });
memoized(3, 7, function (err, res) {
memoized(3, 7, function (err, res) {
// Cache hit
});
});
memoized(3, 7, function (err, res) {
// Cache hit
});
When we are defining a prototype, we may want to define method that will memoize it's results in relation to each instance. Basic way to obtain that would be:
var Foo = function () {
this.bar = memoize(this.bar.bind(this));
// ... constructor logic
};
Foo.prototype.bar = function () {
// ... method logic
};
With method option we can configure memoization directly on prototype:
var Foo = function () {
// ... constructor logic
};
Foo.prototype.bar = memoize(function () {
// ... method logic
}, { method: 'bar' });
Additionally we may provide descriptor which would be used for defining method on instance object:
var Foo = function () {
// ... constructor logic
};
Foo.prototype.bar = memoize(function () {
// ... method logic
}, { method: { name: 'bar', descriptor: { configurable: true } } });
Clear data for particular call.
memoized.clear('foo', true);
Arguments passed to clear
are treated with same rules as input arguments passed to function
Clear all cached data:
memoized.clearAll();
With maxAge option we can ensure that cache for given call is cleared after predefined period of time
memoized = memoize(fn, { maxAge: 1000 });
memoized('foo', 3);
memoized('foo', 3); // Cache hit
setTimeout(function () {
memoized('foo', 3); // No longer in cache, re-executed
memoized('foo', 3); // Cache hit
}, 2000);
Additionally we may ask to pre-fetch in a background a value that is about to expire. Pre-fetch is invoked only if value is accessed close to its expiry date. By default it needs to be within at least 33% of maxAge timespan before expire:
memoized = memoize(fn, { maxAge: 1000, preFetch: true }); // Defaults to 0.33
memoized('foo', 3);
memoized('foo', 3); // Cache hit
setTimeout(function () {
memoized('foo', 3); // Cache hit
}, 500);
setTimeout(function () {
memoized('foo', 3); // Cache hit, silently pre-fetched in next tick
}, 800);
setTimeout(function () {
memoized('foo', 3); // Cache hit
}, 1300);
Pre-fetch timespan can be customized:
memoized = memoize(fn, { maxAge: 1000, preFetch: 0.6 });
memoized('foo', 3);
memoized('foo', 3); // Cache hit
setTimeout(function () {
memoized('foo', 3); // Cache hit, silently pre-fetched in next tick
}, 500);
setTimeout(function () {
memoized('foo', 3); // Cache hit
}, 1300);
Thanks @puzrin for helpful suggestions concerning this functionality
We can track number of references returned from cache, and manually clear them. When last reference is cleared, cache is purged automatically:
memoized = memoize(fn, { refCounter: true });
memoized('foo', 3); // refs: 1
memoized('foo', 3); // Cache hit, refs: 2
memoized('foo', 3); // Cache hit, refs: 3
memoized.clearRef('foo', 3); // refs: 2
memoized.clearRef('foo', 3); // refs: 1
memoized.clearRef('foo', 3); // refs: 0, Cache purged for 'foo', 3
memoized('foo', 3); // Re-executed, refs: 1
With max option you can limit cache size, it's backed with LRU algorithm
memoized = memoize(fn, { max: 2 });
memoized('foo', 3);
memoized('bar', 7);
memoized('foo', 3); // Cache hit
memoized('bar', 7); // Cache hit
memoized('lorem', 11); // Cache cleared for 'foo', 3
memoized('bar', 7); // Cache hit
memoized('foo', 3); // Re-executed, Cache cleared for 'lorem', 11
memoized('lorem', 11); // Re-executed, Cache cleared for 'bar', 7
memoized('foo', 3); // Cache hit
memoized('bar', 7); // Re-executed, Cache cleared for 'lorem', 11
You can register callback that is called on each value being removed from cache:
memoized = memoize(fn, { dispose: function (value) { /*…*/ } });
var foo3 = memoized('foo', 3);
var bar7 = memoized('bar', 7);
memoized.clear('foo', 3); // Dispose called with foo3 value
memoized.clear('bar', 7); // Dispose called with bar7 value
Simple benchmark tests can be found in benchmark folder. Currently it's just plain simple calculation of fibonacci sequences. To run it you need to install other test candidates:
$ npm install underscore lodash lru-cache
Example output taken under Node v0.8.9 on 2008 MBP Pro:
Fibonacci 3000 x10:
1: 21ms Memoizee (primitive mode)
1: 21ms Lo-dash
3: 23ms Underscore
4: 88ms Memoizee (primitive mode) LRU (max: 1000)
5: 178ms Memoizee (object mode)
6: 234ms Memoizee (object mode) LRU (max: 1000)
7: 2852ms lru-cache LRU (max: 1000)
If you want to make sure how much you benefit from memoization or just check if memoization works as expected, loading profile module will give access to all valuable information.
Module needs to be imported before any memoization (that we want to track) is configured. Mind also that running profile module affects performance, it's best not to use it in production environment
var memProfile = require('memoizee/lib/ext/profile');
Access statistics at any time:
memProfile.statistics; // Statistcs accessible for programmatical use
console.log(memProfile.log()); // Output statistics data in readable form
Example console output:
------------------------------------------------------------
Memoize statistics:
Init Cache %Cache Source location
11604 35682 75.46 (all)
2112 19901 90.41 at /Users/medikoo/Projects/_packages/next/lib/fs/is-ignored.js:276:12
2108 9087 81.17 at /Users/medikoo/Projects/_packages/next/lib/fs/is-ignored.js:293:10
6687 2772 29.31 at /Users/medikoo/Projects/_packages/next/lib/fs/watch.js:125:9
697 3922 84.91 at /Users/medikoo/Projects/_packages/next/lib/fs/is-ignored.js:277:15
------------------------------------------------------------
$ npm test
FAQs
Memoize/cache function results
The npm package memoizee receives a total of 2,447,000 weekly downloads. As such, memoizee popularity was classified as popular.
We found that memoizee demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
PyPI has drastically improved its malware response times, resolving 90% of issues in under 24 hours and removing 900 projects since March 2024.
Security News
Research
The Socket Research team breaks down an obfuscated script designed to facilitate unauthorized file uploads to multiple external services.
Security News
Node.js has automated its security release process, doubling the number of releases, and is re-evaluating unsupported experimental features with the Next 10 group to enhance security.