Socket
Socket
Sign inDemoInstall

http-range-fetcher

Package Overview
Dependencies
Maintainers
1
Versions
14
Alerts
File Explorer

Advanced tools

Socket logo

Install Socket

Detect and block malicious and high-risk dependencies

Install

http-range-fetcher

caching, aggregating fetch manager for doing lots of HTTP range requests


Version published
Weekly downloads
549
increased by39.69%
Maintainers
1
Weekly downloads
 
Created
Source

http-range-fetcher

NPM version Build Status

Cache/manager for HTTP byte-range requests that merges requests together and caches results. Designed for applications that request lots of small byte ranges over HTTP that are often adjacent to each other.

Works both in node or webpack/browserify. Respects HTTP caching semantics, with the exception of setting a default minimum TTL of 1 second on requests that are not really supposed to be cached (e.g. Cache-Control: no-cache). You can turn that behavior off by setting minimumTTL to 0 though.

Install

$ npm install --save http-range-fetcher

Usage

const { HttpRangeFetcher } = require('http-range-fetcher')

const cache = new HttpRangeFetcher({})
cache.getRange('http://foo.bar/baz.bam', 20, 10)
.then( response => {
  assert(response.buffer.length === 10)
  assert(response.headers['content-range'] === '20-29/23422')
  // response objects contain `headers` and `buffer`.  the `headers` object
  // contains the original headers that came from the server in response to the
  // aggregated call, except the Content-Range header has been overwritten
  // to match the requested range, and it adds a X-Resource-Length header that
  // conveniently gives the total length of the remote resource so you don't
  // have to parse the Content-Range header.
  assert(response.headers['x-resource-length'] === '23422')
})

// these will be aggregated behind the scenes
// as a single request for a big chunk of the remote file,
// which will be cached to satisfy subsequent requests
Promise.all([
    cache.getRange('http://foo.bar/baz.bam', 20, 10),
    cache.getRange('http://foo.bar/baz.bam', 30, 10),
    cache.getRange('http://foo.bar/baz.bam', 40, 10),
    cache.getRange('http://foo.bar/baz.bam', 50, 10),
    cache.getRange('http://foo.bar/baz.bam', 60, 10),
    cache.getRange('http://foo.bar/baz.bam', 70, 10),
])
.then(fetchResults => {
    fetchResults.forEach(res => assert(res.buffer.length === 10))
})

API

Table of Contents

HttpRangeFetcher

smart cache that fetches chunks of remote files. caches chunks in an LRU cache, and aggregates upstream fetches

Parameters

  • $0 Object
    • $0.fetch (optional, default crossFetchBinaryRange) callback with signature (key, start, end) => Promise({ headers, buffer })
    • $0.size (optional, default 10000000) size in bytes of cache to keep
    • $0.chunkSize (optional, default 32768) size in bytes of cached chunks
    • $0.aggregationTime (optional, default 100) time in ms over which to pool requests before dispatching them
    • $0.minimumTTL (optional, default 1000) time in ms a non-cacheable response will still be cached
    • $0.maxFetchSize (optional, default chunkSize * 4) maximum size of an aggregated request
    • $0.maxExtraFetch (optional, default chunkSize) max number of additional bytes to fetch when aggregating requests that don't actually overlap
getRange

Fetch a range of a remote resource.

Parameters

  • key string the resource's unique identifier, this would usually be a URL. This is passed along to the fetch callback.
  • position number? offset in the file at which to start fetching (optional, default 0)
  • length number? number of bytes to fetch, defaults to the remainder of the file
  • options object? request options (optional, default {})
    • options.signal AbortSignal? object that can be used to abort the fetch. See AbortController on MDN for details

Returns Promise for a response object containing { headers, buffer }

stat

Fetches the first few bytes of the remote file (if necessary) and uses the returned headers to populate a fs-like stat object.

Currently, this attempts to set size, mtime, and mtimeMs, if the information is available from HTTP headers.

Parameters

Returns Promise for a stats object like { size, mtime, mtimeMs }

reset

Throw away all cached data, resetting the cache.

Academic Use

This package was written with funding from the NHGRI as part of the JBrowse project. If you use it in an academic project that you publish, please cite the most recent JBrowse paper, which will be linked from jbrowse.org.

License

MIT © Robert Buels

FAQs

Package last updated on 26 Jun 2023

Did you know?

Socket

Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.

Install

Related posts

SocketSocket SOC 2 Logo

Product

  • Package Alerts
  • Integrations
  • Docs
  • Pricing
  • FAQ
  • Roadmap
  • Changelog

Packages

npm

Stay in touch

Get open source security insights delivered straight into your inbox.


  • Terms
  • Privacy
  • Security

Made with ⚡️ by Socket Inc