@octokit/plugin-paginate-rest

What is @octokit/plugin-paginate-rest?

The @octokit/plugin-paginate-rest npm package is a plugin for Octokit, which is a GitHub API client. It simplifies the process of paginating through REST API responses, allowing users to easily retrieve all items from a list endpoint that supports pagination. This is particularly useful when dealing with GitHub's REST API, which often limits the number of items returned in a single response.

What are @octokit/plugin-paginate-rest's main functionalities?

Paginate through REST API responses

This feature allows users to paginate through the issues of a GitHub repository. The code sample demonstrates how to use the plugin with Octokit to retrieve all issues from a specified repository.

const { Octokit } = require('@octokit/core');
const { paginateRest } = require('@octokit/plugin-paginate-rest');

const MyOctokit = Octokit.plugin(paginateRest);
const octokit = new MyOctokit({ auth: 'personal-access-token' });

async function getAllIssues(owner, repo) {
  const issues = await octokit.paginate('GET /repos/{owner}/{repo}/issues', {
    owner: owner,
    repo: repo
  });
  return issues;
}

Other packages similar to @octokit/plugin-paginate-rest

simple-paginator

Simple-paginator is a package that provides simple pagination functionality. It is not specifically designed for GitHub's REST API or any other API, but it can be used to add pagination to any list of items. Unlike @octokit/plugin-paginate-rest, it does not integrate with Octokit and does not automatically handle API pagination.

README

plugin-paginate-rest.js

Octokit plugin to paginate REST API endpoint responses

Usage

Browsers	Load @octokit/plugin-paginate-rest and @octokit/core (or core-compatible module) directly from esm.sh <script type="module"> import { Octokit } from "https://esm.sh/@octokit/core"; import { paginateRest, composePaginateRest, } from "https://esm.sh/@octokit/plugin-paginate-rest"; </script>
Node	Install with npm install @octokit/core @octokit/plugin-paginate-rest. Optionally replace @octokit/core with a core-compatible module import { Octokit } from "@octokit/core"; import { paginateRest, composePaginateRest, } from "@octokit/plugin-paginate-rest";

const MyOctokit = Octokit.plugin(paginateRest);
const octokit = new MyOctokit({ auth: "secret123" });

// See https://developer.github.com/v3/issues/#list-issues-for-a-repository
const issues = await octokit.paginate("GET /repos/{owner}/{repo}/issues", {
  owner: "octocat",
  repo: "hello-world",
  since: "2010-10-01",
  per_page: 100,
});

If you want to utilize the pagination methods in another plugin, use composePaginateRest.

function myPlugin(octokit, options) {
  return {
    allStars({owner, repo}) => {
      return composePaginateRest(
        octokit,
        "GET /repos/{owner}/{repo}/stargazers",
        {owner, repo }
      )
    }
  }
}

[!IMPORTANT] As we use conditional exports, you will need to adapt your tsconfig.json. See the TypeScript docs on package.json "exports".

octokit.paginate()

The paginateRest plugin adds a new octokit.paginate() method which accepts the same parameters as octokit.request. Only "List ..." endpoints such as List issues for a repository are supporting pagination. Their response includes a Link header. For other endpoints, octokit.paginate() behaves the same as octokit.request().

The per_page parameter is usually defaulting to 30, and can be set to up to 100, which helps retrieving a big amount of data without hitting the rate limits too soon.

An optional mapFunction can be passed to map each page response to a new value, usually an array with only the data you need. This can help to reduce memory usage, as only the relevant data has to be kept in memory until the pagination is complete.

const issueTitles = await octokit.paginate(
  "GET /repos/{owner}/{repo}/issues",
  {
    owner: "octocat",
    repo: "hello-world",
    since: "2010-10-01",
    per_page: 100,
  },
  (response) => response.data.map((issue) => issue.title),
);

The mapFunction gets a 2nd argument done which can be called to end the pagination early.

const issues = await octokit.paginate(
  "GET /repos/{owner}/{repo}/issues",
  {
    owner: "octocat",
    repo: "hello-world",
    since: "2010-10-01",
    per_page: 100,
  },
  (response, done) => {
    if (response.data.find((issue) => issue.title.includes("something"))) {
      done();
    }
    return response.data;
  },
);

Alternatively you can pass a request method as first argument. This is great when using in combination with @octokit/plugin-rest-endpoint-methods:

const issues = await octokit.paginate(octokit.rest.issues.listForRepo, {
  owner: "octocat",
  repo: "hello-world",
  since: "2010-10-01",
  per_page: 100,
});

octokit.paginate.iterator()

If your target runtime environments supports async iterators (such as most modern browsers and Node 10+), you can iterate through each response

const parameters = {
  owner: "octocat",
  repo: "hello-world",
  since: "2010-10-01",
  per_page: 100,
};
for await (const response of octokit.paginate.iterator(
  "GET /repos/{owner}/{repo}/issues",
  parameters,
)) {
  // do whatever you want with each response, break out of the loop, etc.
  const issues = response.data;
  console.log("%d issues found", issues.length);
}

Alternatively you can pass a request method as first argument. This is great when using in combination with @octokit/plugin-rest-endpoint-methods:

const parameters = {
  owner: "octocat",
  repo: "hello-world",
  since: "2010-10-01",
  per_page: 100,
};
for await (const response of octokit.paginate.iterator(
  octokit.rest.issues.listForRepo,
  parameters,
)) {
  // do whatever you want with each response, break out of the loop, etc.
  const issues = response.data;
  console.log("%d issues found", issues.length);
}

composePaginateRest and composePaginateRest.iterator

The compose* methods work just like their octokit.* counterparts described above, with the differenct that both methods require an octokit instance to be passed as first argument

How it works

octokit.paginate() wraps octokit.request(). As long as a rel="next" link value is present in the response's Link header, it sends another request for that URL, and so on.

Most of GitHub's paginating REST API endpoints return an array, but there are a few exceptions which return an object with a key that includes the items array. For example:

• Search repositories (key items)
• List check runs for a specific ref (key: check_runs)
• List check suites for a specific ref (key: check_suites)
• List repositories for an installation (key: repositories)
• List installations for a user (key installations)
• Compare commits (key commits)

octokit.paginate() is working around these inconsistencies so you don't have to worry about it.

If a response is lacking the Link header, octokit.paginate() still resolves with an array, even if the response returns a single object.

Types

The plugin also exposes some types and runtime type guards for TypeScript projects.

Types	import { PaginateInterface, PaginatingEndpoints, } from "@octokit/plugin-paginate-rest";
Guards	import { isPaginatingEndpoint } from "@octokit/plugin-paginate-rest";

PaginateInterface

An interface that declares all the overloads of the .paginate method.

PaginatingEndpoints

An interface which describes all API endpoints supported by the plugin. Some overloads of .paginate() method and composePaginateRest() function depend on PaginatingEndpoints, using the keyof PaginatingEndpoints as a type for one of its arguments.

import { Octokit } from "@octokit/core";
import {
  PaginatingEndpoints,
  composePaginateRest,
} from "@octokit/plugin-paginate-rest";

type DataType<T> = "data" extends keyof T ? T["data"] : unknown;

async function myPaginatePlugin<E extends keyof PaginatingEndpoints>(
  octokit: Octokit,
  endpoint: E,
  parameters?: PaginatingEndpoints[E]["parameters"],
): Promise<DataType<PaginatingEndpoints[E]["response"]>> {
  return await composePaginateRest(octokit, endpoint, parameters);
}

isPaginatingEndpoint

A type guard, isPaginatingEndpoint(arg) returns true if arg is one of the keys in PaginatingEndpoints (is keyof PaginatingEndpoints).

import { Octokit } from "@octokit/core";
import {
  isPaginatingEndpoint,
  composePaginateRest,
} from "@octokit/plugin-paginate-rest";

async function myPlugin(octokit: Octokit, arg: unknown) {
  if (isPaginatingEndpoint(arg)) {
    return await composePaginateRest(octokit, arg);
  }
  // ...
}

Contributing

See CONTRIBUTING.md

License

MIT