Socket
Socket
Sign inDemoInstall

cacache

Package Overview
Dependencies
56
Maintainers
2
Versions
100
Alerts
File Explorer

Advanced tools

Install Socket

Detect and block malicious and high-risk dependencies

Install

    cacache

General content-addressable cache system that maintains a filesystem registry of file data.


Version published
Weekly downloads
29M
decreased by-1.36%
Maintainers
2
Install size
1.46 MB
Created
Weekly downloads
 

Package description

What is cacache?

The cacache npm package is a local key and content addressable cache that stores data reliably, with an API that supports fetching, inserting, and deleting cache data. It is primarily used for caching data that can be retrieved by a unique key, and it ensures data integrity by using content-addressable storage.

What are cacache's main functionalities?

Getting data from cache

This feature allows you to retrieve data from the cache using a unique key. If the data is present and the integrity checks out, it will be returned.

const cacache = require('cacache');
const key = 'my-key';
cacache.get('/path/to/cache', key).then(console.log);

Inserting data into cache

This feature is used to insert data into the cache. The data is stored with a unique key, and its integrity is verified upon retrieval.

const cacache = require('cacache');
const key = 'my-key';
const data = 'some data to cache';
cacache.put('/path/to/cache', key, data).then(console.log);

Deleting data from cache

This feature allows you to remove data from the cache using the associated key. This is useful for invalidating or updating cached data.

const cacache = require('cacache');
const key = 'my-key';
cacache.rm.entry('/path/to/cache', key).then(console.log);

Other packages similar to cacache

Changelog

Source

6.0.0 (2017-03-05)

Bug Fixes

  • api: keep memo cache mostly-internal (2f72d0a)
  • content: use the rest of the string, not the whole string (fa8f3c3)
  • deps: removed format-number@2.0.2 (1187791)
  • deps: removed inflight@1.0.6 (0d1819c)
  • deps: rimraf@2.6.1 (9efab6b)
  • deps: standard@9.0.0 (4202cba)
  • deps: tap@10.3.0 (aa03088)
  • deps: weallcontribute@1.0.8 (ad4f4dc)
  • docs: add security note to hashKey (03f81ba)
  • hashes: change default hashAlgorithm to sha512 (ea00ba6)
  • hashes: missed a spot for hashAlgorithm defaults (45997d8)
  • index: add length header before JSON for verification (fb8cb4d)
  • index: change index filenames to sha1s of keys (bbc5fca)
  • index: who cares about race conditions anyway (b1d3888)
  • perf: bulk-read get+read for massive speed (d26cdf9)
  • perf: use bulk file reads for index reads (79a8891)
  • put-stream: remove tmp file on stream insert error (65f6632)
  • put-stream: robustified and predictibilized (daf9e08)
  • put-stream: use new promise API for moves (1d36013)
  • readme: updated to reflect new default hashAlgo (c60a2fa)
  • verify: tiny typo fix (db22d05)

Features

  • api: converted external api (7bf032f)
  • cacache: exported clearMemoized() utility (8d2c5b6)
  • cache: add versioning to content and index (31bc549)
  • content: collate content files into subdirs (c094d9f)
  • deps: @npmcorp/move@1.0.0 (bdd00bf)
  • deps: bluebird@3.4.7 (3a17aff)
  • deps: promise-inflight@1.0.1 (a004fe6)
  • get: added memoization support for get (c77d794)
  • get: export hasContent (2956ec3)
  • index: add hashAlgorithm and format insert ret val (b639746)
  • index: collate index files into subdirs (e8402a5)
  • index: promisify entry index (cda3335)
  • memo: added memoization lib (da07b92)
  • memo: export memoization api (954b1b3)
  • move-file: add move fallback for weird errors (5cf4616)
  • perf: bulk content write api (51b536e)
  • put: added memoization support to put (b613a70)
  • read: switched to promises (a869362)
  • rm: added memoization support to rm (4205cf0)
  • rm: switched to promises (a000d24)
  • util: promise-inflight ownership fix requests (9517cd7)
  • util: use promises for api (ae204bb)
  • verify: converted to Promises (f0b3974)

BREAKING CHANGES

  • cache: index/content directories are now versioned. Previous caches are no longer compatible and cannot be migrated.
  • util: fix-owner now uses Promises instead of callbacks
  • index: Previously-generated index entries are no longer compatible and the index must be regenerated.
  • index: The index format has changed and previous caches are no longer compatible. Existing caches will need to be regenerated.
  • hashes: Default hashAlgorithm changed from sha1 to sha512. If you rely on the prior setting, pass opts.hashAlgorithm in explicitly.
  • content: Previously-generated content directories are no longer compatible and must be regenerated.
  • verify: API is now promise-based
  • read: Switches to a Promise-based API and removes callback stuff
  • rm: Switches to a Promise-based API and removes callback stuff
  • index: this changes the API to work off promises instead of callbacks
  • api: this means we are going all in on promises now

Readme

Source

cacache npm version license Travis AppVeyor Coverage Status

cacache is a Node.js library for managing caches of keyed data that can be looked up both by key and by a digest of the content itself. This means that by-content lookups can be very very fast, and that stored content is shared by different keys if they point to the same data.

Install

$ npm install --save cacache

Table of Contents

Example

const cacache = require('cacache')
const fs = require('fs')

const tarball = '/path/to/mytar.tgz'
const cachePath = '/tmp/my-toy-cache'
const key = 'my-unique-key-1234'
let tarballDigest = null

// Cache it! Use `cachePath` as the root of the content cache
fs.createReadStream(
  tarball
).pipe(
  cacache.put.stream(
    cachePath, key
  ).on('digest', (d) => tarballDigest = d)
).on('finish', function () {
  console.log(`Saved ${tarball} to ${cachePath}.`)
})

const destination = '/tmp/mytar.tgz'

// Copy the contents out of the cache and into their destination!
cacache.get.stream(
  cachePath, key
).pipe(
  fs.createWriteStream(destination)
).on('finish', () => {
  console.log('done extracting!')
})

// The same thing, but skip the key index.
cacache.get.stream.byDigest(
  cachePath, tarballDigest
).pipe(
  fs.createWriteStream(destination)
).on('finish', () => {
  console.log('done extracting using sha1!')
})

Features

  • Extraction by key or by content digest (shasum, etc).
  • Deduplicated content by digest -- two inputs with same key are only saved once
  • Consistency checks, both on insert and extract.
  • (Kinda) concurrency-safe and fault tolerant.
  • Streaming support.
  • Metadata storage.

Contributing

The cacache team enthusiastically welcomes contributions and project participation! There's a bunch of things you can do if you want to contribute! The Contributor Guide has all the information you need for everything from reporting bugs to contributing entire new features. Please don't hesitate to jump in if you'd like to, or even ask us questions if something isn't clear.

API

> cacache.ls(cache) -> Promise

Lists info for all entries currently in the cache as a single large object. Each entry in the object will be keyed by the unique index key, with corresponding get.info objects as the values.

Example
cacache.ls(cachePath).then(console.log)
// Output
{
  'my-thing': {
    key: 'my-thing',
    digest: 'deadbeef',
    path: '.testcache/content/deadbeef',
    time: 12345698490,
    metadata: {
      name: 'blah',
      version: '1.2.3',
      description: 'this was once a package but now it is my-thing'
    }
  },
  'other-thing': {
    key: 'other-thing',
    digest: 'bada55',
    path: '.testcache/content/bada55',
    time: 11992309289
  }
}
> cacache.get(cache, key, [opts]) -> Promise({data, metadata, digest})

Returns an object with the cached data, digest, and metadata identified by key. The data property of this object will be a Buffer instance that presumably holds some data that means something to you. I'm sure you know what to do with it! cacache just won't care.

If there is no content identified by key, or if the locally-stored data does not pass the validity checksum, the promise will be rejected.

A sub-function, get.byDigest may be used for identical behavior, except lookup will happen by content digest, bypassing the index entirely. This version of the function only returns data itself, without any wrapper.

Note

This function loads the entire cache entry into memory before returning it. If you're dealing with Very Large data, consider using get.stream instead.

Example
// Look up by key
cache.get(cachePath, 'my-thing').then(console.log)
// Output:
{
  metadata: {
    thingName: 'my'
  },
  digest: 'deadbeef',
  hashAlgorithm: 'sha512'
  data: Buffer#<deadbeef>
}

// Look up by digest
cache.get.byDigest(cachePath, 'deadbeef', {
  hashAlgorithm: 'sha512'
}).then(console.log)
// Output:
Buffer#<deadbeef>
> cacache.get.stream(cache, key, [opts]) -> Readable

Returns a Readable Stream of the cached data identified by key.

If there is no content identified by key, or if the locally-stored data does not pass the validity checksum, an error will be emitted.

metadata and digest events will be emitted before the stream closes, if you need to collect that extra data about the cached entry.

A sub-function, get.stream.byDigest may be used for identical behavior, except lookup will happen by content digest, bypassing the index entirely. This version does not emit the metadata and digest events at all.

Example
// Look up by key
cache.get.stream(
  cachePath, 'my-thing'
).on('metadata', metadata => {
  console.log('metadata:', metadata)
}).on('hashAlgorithm', algo => {
  console.log('hashAlgorithm:', algo)
}).on('digest', digest => {
  console.log('digest:', digest)
}).pipe(
  fs.createWriteStream('./x.tgz')
)
// Outputs:
metadata: { ... }
hashAlgorithm: 'sha512'
digest: deadbeef

// Look up by digest
cache.get.stream.byDigest(
  cachePath, 'deadbeef', { hashAlgorithm: 'sha512' }
).pipe(
  fs.createWriteStream('./x.tgz')
)
> cacache.get.info(cache, key) -> Promise

Looks up key in the cache index, returning information about the entry if one exists. If an entry does not exist, the second argument to cb will be falsy.

Fields
  • key - Key the entry was looked up under. Matches the key argument.
  • digest - Content digest the entry refers to.
  • hashAlgorithm - Hashing algorithm used to generate digest.
  • path - Filesystem path relative to cache argument where content is stored.
  • time - Timestamp the entry was first added on.
  • metadata - User-assigned metadata associated with the entry/content.
Example
cacache.get.info(cachePath, 'my-thing').then(console.log)

// Output
{
  key: 'my-thing',
  digest: 'deadbeef',
  path: '.testcache/content/deadbeef',
  time: 12345698490,
  metadata: {
    name: 'blah',
    version: '1.2.3',
    description: 'this was once a package but now it is my-thing'
  }
}
> cacache.put(cache, key, data, [opts]) -> Promise

Inserts data passed to it into the cache. The returned Promise resolves with a digest (generated according to opts.hashAlgorithm) after the cache entry has been successfully written.

Example
fetch(
  'https://registry.npmjs.org/cacache/-/cacache-1.0.0.tgz'
).then(data => {
  cacache.put(
    cachePath, 'registry.npmjs.org|cacache@1.0.0', data
  )
}).then(digest => {
  console.log('digest is', digest)
})
> cacache.put.stream(cache, key, [opts]) -> Writable

Returns a Writable Stream that inserts data written to it into the cache. Emits a digest event with the digest of written contents when it succeeds.

Example
request.get(
  'https://registry.npmjs.org/cacache/-/cacache-1.0.0.tgz'
).pipe(
  cacache.put.stream(
    cachePath, 'registry.npmjs.org|cacache@1.0.0'
  ).on('digest', d => console.log('digest is ${d}'))
)
> cacache.put options

cacache.put functions have a number of options in common.

metadata

Arbitrary metadata to be attached to the inserted key.

size

If provided, the data stream will be verified to check that enough data was passed through. If there's more or less data than expected, insertion will fail with an EBADSIZE error.

digest

If present, the pre-calculated digest for the inserted content. If this option if provided and does not match the post-insertion digest, insertion will fail with an EBADCHECKSUM error.

To control the hashing algorithm, use opts.hashAlgorithm.

hashAlgorithm

Default: 'sha512'

Hashing algorithm to use when calculating the digest for inserted data. Can use any algorithm listed in crypto.getHashes() or 'omakase'/'お任せします' to pick a random hash algorithm on each insertion. You may also use any anagram of 'modnar' to use this feature.

uid/gid

If provided, cacache will do its best to make sure any new files added to the cache use this particular uid/gid combination. This can be used, for example, to drop permissions when someone uses sudo, but cacache makes no assumptions about your needs here.

memoize

Default: null

If provided, cacache will memoize the given cache insertion in memory, bypassing any filesystem checks for that key or digest in future cache fetches. Nothing will be written to the in-memory cache unless this option is explicitly truthy.

There is no facility for limiting memory usage short of cacache.clearMemoized(), so be mindful of the sort of data you ask to get memoized!

Reading from existing memoized data can be forced by explicitly passing memoize: false to the reader functions, but their default will be to read from memory.

> cacache.rm.all(cache) -> Promise

Clears the entire cache. Mainly by blowing away the cache directory itself.

Example
cacache.rm.all(cachePath).then(() => {
  console.log('THE APOCALYPSE IS UPON US 😱')
})
> cacache.rm.entry(cache, key) -> Promise

Removes the index entry for key. Content will still be accessible if requested directly by content address (get.stream.byDigest).

Example
cacache.rm.entry(cachePath, 'my-thing').then(() => {
  console.log('I did not like it anyway')
})
> cacache.rm.content(cache, digest) -> Promise

Removes the content identified by digest. Any index entries referring to it will not be usable again until the content is re-added to the cache with an identical digest.

Example
cacache.rm.content(cachePath, 'deadbeef').then(() => {
  console.log('data for my-thing is gone!')
})
> cacache.clearMemoized()

Completely resets the in-memory entry cache.

> cacache.verify(cache, opts) -> Promise

Checks out and fixes up your cache:

  • Cleans up corrupted or invalid index entries.
  • Garbage collects any content entries not referenced by the index.
  • Checks digests for all content entries and removes invalid content.
  • Fixes cache ownership.
  • Removes the tmp directory in the cache and all its contents.

When it's done, it'll return an object with various stats about the verification process, including amount of storage reclaimed, number of valid entries, number of entries removed, etc.

This function should not be run while other processes are running cacache. It assumes it'll be used offline by a human or a coordinated process. Concurrent verifies are protected by a lock, but there's no guarantee others won't be reading/writing on the cache.

Options
  • opts.uid - uid to assign to cache and its contents
  • opts.gid - gid to assign to cache and its contents
  • opts.hashAlgorithm - defaults to 'sha512'. Hash to use for content checks.
Example
echo somegarbage >> $CACHEPATH/content/deadbeef
cacache.verify(cachePath).then(stats => {
  // deadbeef collected, because of invalid checksum.
  console.log('cache is much nicer now! stats:', stats)
})
> cacache.verify.lastRun(cache) -> Promise

Returns a Date representing the last time cacache.verify was run on cache.

Example
cacache.verify(cachePath).then(() => {
  cacache.verify.lastRun(cachePath).then(lastTime => {
    console.log('cacache.verify was last called on' + lastTime)
  })
})

Keywords

FAQs

Last updated on 05 Mar 2017

Did you know?

Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.

Install

Related posts

SocketSocket SOC 2 Logo

Product

  • Package Alerts
  • Integrations
  • Docs
  • Pricing
  • FAQ
  • Roadmap

Stay in touch

Get open source security insights delivered straight into your inbox.


  • Terms
  • Privacy
  • Security

Made with ⚡️ by Socket Inc