A cacheable dns.lookup(…) that respects TTL. 

A cacheable dns.lookup(…) that respects TTL

sindresorhus

szmarczak

cacheable-lookup

import {Resolver, promises as dnsPromises, lookup} from 'dns';

type AsyncResolver = dnsPromises.Resolver;

export type EntrySource = 'query' | 'cache';

set(hostname: string, entries: EntryObject[], ttl: number): TPromise<void | boolean | this>;

get(hostname: string): TPromise<EntryObject[] | undefined>;

delete(hostname: string): TPromise<boolean>;

	 * Custom cache instance. If `undefined`, it will create a new one.

	 * Limits the cache time (TTL). If set to `0`, it will make a new DNS query each time.

	 * DNS Resolver used to make DNS queries.

	 * When the DNS server responds with `ENOTFOUND` or `ENODATA` and the OS reports that the entry is available,

	 * it will use `dns.lookup(...)` directly for the requested hostnames for the specified amount of time (in seconds).

	 * If you don't query internal hostnames (such as `localhost`, `database.local` etc.),

	 * it is strongly recommended to set this value to `0`.

	 * The time how long it needs to remember failed queries (TTL in seconds).

	 * **Note**: This option is independent, `options.maxTtl` does not affect this.

	 * The fallback function to use when the DNS server responds with `ENOTFOUND` or `ENODATA`.

	 * **Note**: This has no effect if the `fallbackDuration` option is less than `1`.

	 * The IP address (can be an IPv4 or IPv5 address).

	 * Whether this entry comes from the cache or a query

	 * One or more supported getaddrinfo flags. Multiple flags may be passed by bitwise ORing their values.

	 * The record family. Must be `4` or `6`. IPv4 and IPv6 addresses are both returned by default.

	 * When `true`, the callback returns all resolved addresses in an array. Otherwise, returns a single address.

	 * The DNS servers used to make queries. Can be overridden - doing so will clear the cache.

	 * @see https://nodejs.org/api/dns.html#dns_dns_lookup_hostname_options_callback

	lookup(hostname: string, family: IPFamily, callback: (error: NodeJS.ErrnoException | null, address: string, family: IPFamily) => void): void;

	lookup(hostname: string, callback: (error: NodeJS.ErrnoException | null, address: string, family: IPFamily) => void): void;

lookup(hostname: string, options: LookupOptions & {all: true}, callback: (error: NodeJS.ErrnoException | null, result: ReadonlyArray<EntryObject>) => void): void;

	lookup(hostname: string, options: LookupOptions, callback: (error: NodeJS.ErrnoException | null, address: string, family: IPFamily) => void): void;

	 * The asynchronous version of `dns.lookup(…)`.

lookupAsync(hostname: string, options: LookupOptions & {all: true}): Promise<ReadonlyArray<EntryObject>>;

lookupAsync(hostname: string, options: LookupOptions): Promise<EntryObject>;

lookupAsync(hostname: string): Promise<EntryObject>;

lookupAsync(hostname: string, family: IPFamily): Promise<EntryObject>;

	 * An asynchronous function which returns cached DNS lookup entries. This is the base for `lookupAsync(hostname, options)` and `lookup(hostname, options, callback)`.

query(hostname: string): Promise<ReadonlyArray<EntryObject>>;

	 * An asynchronous function which makes a new DNS lookup query and updates the database. This is used by `query(hostname, family)` if no entry in the database is present. Returns an array of objects with `address`, `family`, `ttl` and `expires` properties.

queryAndCache(hostname: string): Promise<ReadonlyArray<EntryObject>>;

	 * Attaches itself to an Agent instance.

	 * Removes itself from an Agent instance.

	 * Updates interface info. For example, you need to run this when you plug or unplug your WiFi driver.

	 * **Note:** Running `updateInterfaceInfo()` will trigger `clear()` only on network interface removal.

	 * Clears the cache for the given hostname. If the hostname argument is not present, the entire cache will be emptied.

  "description": "A cacheable dns.lookup(…) that respects TTL",

    "test": "xo && nyc --reporter=lcovonly --reporter=text ava && tsd"

    "url": "git+https://github.com/szmarczak/cacheable-lookup.git"

    "url": "https://github.com/szmarczak/cacheable-lookup/issues"

  "homepage": "https://github.com/szmarczak/cacheable-lookup#readme",

> A cacheable [`dns.lookup(…)`](https://nodejs.org/api/dns.html#dns_dns_lookup_hostname_options_callback) that respects TTL :tada:

[![Node CI](https://github.com/szmarczak/cacheable-lookup/workflows/Node%20CI/badge.svg)](https://github.com/szmarczak/cacheable-lookup/actions)

[![codecov](https://codecov.io/gh/szmarczak/cacheable-lookup/branch/master/graph/badge.svg)](https://codecov.io/gh/szmarczak/cacheable-lookup)

[![npm](https://img.shields.io/npm/dm/cacheable-lookup.svg)](https://www.npmjs.com/package/cacheable-lookup)

[![install size](https://packagephobia.now.sh/badge?p=cacheable-lookup)](https://packagephobia.now.sh/result?p=cacheable-lookup)

Making lots of HTTP requests? You can save some time by caching DNS lookups :zap:

const CacheableLookup = require('cacheable-lookup');

http.get('http://example.com', {lookup: cacheable.lookup}, response => {

### Attaching CacheableLookup to an Agent

http.get('http://example.com', response => {

Returns a new instance of `CacheableLookup`.

Type: `Map` | [`Keyv`](https://github.com/lukechilds/keyv/)

Custom cache instance. If `undefined`, it will create a new one.

**Note**: If you decide to use Keyv instead of the native implementation, the performance will drop by 10x. Memory leaks may occur as it doesn't provide any way to remove all the deprecated values at once.

**Tip**: [`QuickLRU`](https://github.com/sindresorhus/quick-lru) is fully compatible with the Map API, you can use it to limit the amount of cached entries. Example:

The maximum lifetime of the entries received from the specifed DNS server (TTL in seconds).

If set to `0`, it will make a new DNS query each time.

**Pro Tip**: This shouldn't be lower than your DNS server response time in order to prevent bottlenecks. For example, if you use Cloudflare, this value should be greater than `0.01`.

When the DNS server responds with `ENOTFOUND` or `ENODATA` and the OS reports that the entry is available, it will use `dns.lookup(...)` directly for the requested hostnames for the specified amount of time (in seconds).

**Note**: You should avoid setting this to `0` unless the provided DNS servers' database is limited to few domains.

**Note**: This is not present when falling back to `dns.lookup(...)`!

Whether this entry was loaded from the cache or came from a query (`cache` or `query`)

When `options.all` is `false`, then `callback(error, address, family, expires, ttl)` is called.

When `options.all` is `true`, then `callback(error, entries)` is called.

The DNS servers used to make queries. Can be overridden - doing so will clear the cache.

#### [lookup(hostname, options, callback)](https://nodejs.org/api/dns.html#dns_dns_lookup_hostname_options_callback)

The asynchronous version of `dns.lookup(…)`.

Returns an [entry object](#entry-object).

If `options.all` is true, returns an array of entry objects.

The same as the [`dns.lookup(…)`](https://nodejs.org/api/dns.html#dns_dns_lookup_hostname_options_callback) options.

An asynchronous function which returns cached DNS lookup entries.

This is the base for `lookupAsync(hostname, options)` and `lookup(hostname, options, callback)`.

Returns an array of objects with `address`, `family`, `ttl` and `expires` properties.

An asynchronous function which makes two DNS queries: A and AAAA. The result is cached.

This is used by `query(hostname)` if no entry in the database is present.

Updates interface info. For example, you need to run this when you plug or unplug your WiFi driver.

**Note:** Running `updateInterfaceInfo()` will trigger `clear()` only on network interface removal.

Clears the cache for the given hostname. If the hostname argument is not present, the entire cache will be emptied.

CacheableLookup#lookupAsync                x 2,896,251 ops/sec ±1.07% (85 runs sampled)

CacheableLookup#lookupAsync.all            x 2,842,664 ops/sec ±1.11% (88 runs sampled)

CacheableLookup#lookupAsync.all.ADDRCONFIG x 2,598,283 ops/sec ±1.21% (88 runs sampled)

CacheableLookup#lookup                     x 2,565,913 ops/sec ±1.56% (85 runs sampled)

CacheableLookup#lookup.all                 x 2,609,039 ops/sec ±1.01% (86 runs sampled)

CacheableLookup#lookup.all.ADDRCONFIG      x 2,416,242 ops/sec ±0.89% (85 runs sampled)

dns#lookup                                 x 7,272     ops/sec ±0.36% (86 runs sampled)

dns#lookup.all                             x 7,249     ops/sec ±0.40% (86 runs sampled)

dns#lookup.all.ADDRCONFIG                  x 5,693     ops/sec ±0.28% (85 runs sampled)

Fastest is CacheableLookup#lookupAsync.all

 - [cacheable-request](https://github.com/lukechilds/cacheable-request) - Wrap native HTTP requests with RFC compliant cache support

const kCacheableLookupCreateConnection = Symbol('cacheableLookupCreateConnection');

const kCacheableLookupInstance = Symbol('cacheableLookupInstance');

const supportsALL = typeof ALL === 'number';

	if (!(agent && typeof agent.createConnection === 'function')) {

		throw new Error('Expected an Agent instance as the first argument');

		entry.address = `::ffff:${entry.address}`;

	for (const device of Object.values(os.networkInterfaces())) {

const ignoreNoResultErrors = dnsPromise => {

			error.code === 'ENOENT' // Windows: name exists, but not this record type

		this._dnsLookup = lookup && promisify(lookup);

				callback(null, result.address, result.family, result.expires, result.ttl);

				callback(null, result.address, result.family, result.expires, result.ttl, result.source);

	async lookupAsync(hostname, options = {}) {

		let cached = await this.query(hostname);

			const filtered = cached.filter(entry => entry.family === 6);

				if ((supportsALL && options.hints & ALL) || filtered.length === 0) {

			cached = cached.filter(entry => entry.family === 4);

			cached = cached.filter(entry => entry.family === 6 ? _iface.has6 : _iface.has4);

			const error = new Error(`cacheableLookup ENOTFOUND ${hostname}`);

		let cached = await this._cache.get(hostname);

			const pending = this._pending[hostname];

				const newPromise = this.queryAndCache(hostname);

				this._pending[hostname] = newPromise;

		// ANY is unsafe as it doesn't trigger new queries in the underlying server.

			ignoreNoResultErrors(this._resolve4(hostname, ttl)),

			ignoreNoResultErrors(this._resolve6(hostname, ttl))

			entry.expires = now + (entry.ttl * 1000);

			aaaaTtl = Math.max(aaaaTtl, entry.ttl);

				// Passing {all: true} doesn't return all IPv4 and IPv6 entries.

				// See https://github.com/szmarczak/cacheable-lookup/issues/42

				ignoreNoResultErrors(this._dnsLookup(hostname, all4)),

				ignoreNoResultErrors(this._dnsLookup(hostname, all6))

			cacheTtl = Math.min(cacheTtl, this.maxTtl) * 1000;

			data[kExpires] = Date.now() + cacheTtl;

				await this._cache.set(hostname, data, cacheTtl);

					const cacheError = new Error('Cache Error. Please recreate the CacheableLookup instance.');

		if (this._hostnamesToFallback.has(hostname)) {

			return this._dnsLookup(hostname, all);

		@@ -7,2 +7,3 @@ import {Resolver, promises as dnsPromises, lookup} from 'dns';
		export type IPFamily = 4 \| 6;
		export type EntrySource = 'query' \| 'cache';

		@@ -76,2 +77,6 @@ type TPromise<T> = T \| Promise<T>;
		readonly expires?: number;
		/**
		* Whether this entry comes from the cache or a query
		*/
		readonly source?: EntrySource;
		}
		@@ -78,0 +83,0 @@

		{
		"name": "cacheable-lookup",
		"version": "6.0.4",
		"version": "6.1.0",
		"description": "A cacheable dns.lookup(…) that respects TTL",
		@@ -5,0 +5,0 @@ "engines": {

		@@ -159,2 +159,8 @@ # cacheable-lookup

		#### source

		Note: This is not present when falling back to `dns.lookup(...)`!

		Whether this entry was loaded from the cache or came from a query (`cache` or `query`)

		### Entry object (callback-style)
		@@ -161,0 +167,0 @@

		@@ -163,3 +163,3 @@ 'use strict';
		} else {
		callback(null, result.address, result.family, result.expires, result.ttl);
		callback(null, result.address, result.family, result.expires, result.ttl, result.source);
		}
		@@ -215,2 +215,3 @@ }, callback);
		async query(hostname) {
		let source = 'cache';
		let cached = await this._cache.get(hostname);
		@@ -224,2 +225,3 @@
		} else {
		source = 'query';
		const newPromise = this.queryAndCache(hostname);
		@@ -237,3 +239,3 @@ this._pending[hostname] = newPromise;
		cached = cached.map(entry => {
		return {...entry};
		return {...entry, source};
		});
		@@ -240,0 +242,0 @@

Improved metrics