@mavenoid/dataloader

@mavenoid/dataloader

Utilities for caching, batching and deduping requests in GraphQL. Inspired by dataloader but extended using WeakMap to create a more convenient API for GraphQL

Installation

yarn add @mavenoid/dataloader

Usage

import {
  getDataLoadersForContextType,
  RequestsBatch,
} from '@mavenoid/dataloader';

const {batch, dedupe} = getDataLoadersForContextType<ResolverContextType>();

const getUserDeduped = dedupe(
  async (userID: number, _resolverContext: ResolverContextType) => {
    return await getUserUncached(userID);
  },
);

// You can load a user within a GraphQL resolver using:
//
//   getUserDeduped(userID, resolverContext)
//
// You can call this as many times as you like, and `getUserUncached` will only be called
// once per request.
//
// if you modify a user within a request, you can delete it from the cache using:
//
//   getUserDeduped.cache(resolverContext).delete(userID)
//
// or
//
//   getUserDeduped.cache(resolverContext).clear()

const getUserBatched = batch(
  async (userIDs: number[], _resolverContext: ResolverContextType) => {
    // load all the requested userIDs in one go, and then return a map from
    // the requested IDs to the results
    const users = await db_users.find({id: anyOf(userIDs)}).all();
    return new Map(users.map((user) => [user.id, user]));
  },
  {maxBatchSize: 100},
);

// You can load a user within a GraphQL resolver using:
//
//   getUserBatched(userID, resolverContext)
//
// You can call this as many times as you like, and the database will only be queried
// once per request. The requests will be batched so that up to 100 users are loaded
// at a time.
//
// if you modify a user within a request, you can delete it from the cache using:
//
//   getUserBatched.cache(resolverContext).delete(userID)
//
// or
//
//   getUserBatched.cache(resolverContext).clear()

Handling undefined

The batch methods return undefined for any requested records that are not included in the results. If this is not what you want, you can add a fallback using:

import {
  handleUndefinedResult,
  errorOnUndefinedResult,
} from '@mavenoid/dataloader';

const getUserOrNull = handleUndefinedResult(getUserBatched, () => null);
const getUserOrError = errorOnUndefinedResult(
  getUserBatched,
  (id) => `Could not find user: ${id}`,
);

Handling requests for lists of dependent records

The groupRecordsByKey can be used to group a list of records by a foreign key. It will preserve the order of the records within each group.

const getUserMessagesBatched = handleUndefinedResult(
  batch(
    async (userIDs: number[], _resolverContext: ResolverContextType) => {
      const messages = await db_messages
        .find({sender_id: anyOf(userIDs)})
        .all();
      return groupRecordsByKey(messages, (m) => m.sender_id);
    },
    {maxBatchSize: 100},
  ),
  () => [],
);

Handling requests with composite keys

If you need to query using more than one key, you can use an object. JavaScript compares objects by reference, rather than by value though. This means you will need to transform the key into a string or number wherever it is used to look up a value in a Map/Cache. We provide a few helpers to make that easier.

const getTranslatedMessage = batch(
  async (
    requests: {messageID: number; language: string}[],
    _resolverContext: ResolverContextType,
  ) => {
    const messages = await db_message_translations.query(
      sql`
        SELECT * FROM message_translations
        WHERE ${sql.join(
          requests.map(
            (r) => sql`(message_id=${r.messageID} AND lang=${r.language})`,
          ),
          ' OR ',
        )}
      `,
    );
    return transformMapKey(
      extractRecordsByKey(messages, (m) => `${m.message_id}/${m.lang}`),
    )(
      (req: {messageID: number; language: string}) =>
        `${req.messageID}/${req.language}`,
    );
  },
  {
    maxBatchSize: 100,
    getCacheKey: (req) => `${req.messageID}/${req.language}`,
  },
);