@code-pushup/ci

@code-pushup/ci

🔎🔬 Quality metrics for your software project. 📉🔍

• ⚙️ Configure what you want to track using your favourite tools.
• 🤖 Integrate it in your CI.
• 🌈 Visualize reports in a beautiful dashboard.

This package exports provider-agnostic core logic for running Code PushUp in CI pipelines. It serves as the base for the following provider integrations:

GitHub Actions	code-pushup/github-action
GitLab CI/CD	code-pushup/gitlab-pipelines-template

Setup

npm install @code-pushup/ci

yarn add @code-pushup/ci

pnpm add @code-pushup/ci

Usage

The runInCI function implements the full CI flow:

import { runInCI } from '@code-pushup/ci';

const result = await runInCI(
  {
    /* Git refs */
  },
  {
    /* Provider API client */
  },
  {
    /* Options */
  },
);

Parameters

Git refs

For each CI run, you must pass in the commit SHA and Git ref (e.g. main) of what was pushed. These values can be detected from the CI environment, the details depend on which provider is being used.

If only the head is supplied, then Code PushUp will collect a new report and optionally upload it to portal (depending on your Code PushUp config). If triggered by a pull request, then specify the base ref as well. This will additionally compare reports from both source and target branches and post a comment to the PR.

Property	Required	Type	Description
head	yes	string | { ref: string, sha: string }	Current branch/commit
base	no	string | { ref: string, sha: string }	Branch targeted by PR

Provider API client

The PR flow requires interacting with the Git provider's API to post a comparison comment. Wrap these requests in functions and pass them in as an object which configures the provider.

Property	Required	Type	Description
createComment	yes	(body: string) => Promise<Comment>	Posts a new comment to PR
updateComment	yes	(id: number, body: string) => Promise<Comment>	Updates existing PR comment
listComments	yes	() => Promise<Comment[]>	Fetches all comments from PR
maxCommentChars	yes	number	Character limit for comment body
downloadReportArtifact	no	(project?: string) => Promise<string | null>	Fetches previous (root/project) report.json for base branch and returns path, used as cache to speed up comparison

A Comment object has the following required properties:

Property	Type	Description
id	number	Comment ID
body	string	Content of comment as Markdown string
url	string	Web link to comment in PR

Options

Optionally, you can override default options for further customization:

Property	Type	Default	Description
monorepo	boolean | MonorepoTool	false	Enables monorepo mode
parallel	boolean | number	false	Enables parallel execution in monorepo mode
projects	string[] | null	null	Custom projects configuration for monorepo mode
task	string	'code-pushup'	Name of command to run Code PushUp per project in monorepo mode
nxProjectsFilter	string | string[]	'--with-target={task}'	Arguments passed to nx show projects, only relevant for Nx in monorepo mode 1
directory	string	process.cwd()	Directory in which Code PushUp CLI should run
config	string | null	null 2	Path to config file (--config option)
silent	boolean	false	Hides logs from CLI commands (errors will be printed)
bin	string	'npx --no-install code-pushup'	Command for executing Code PushUp CLI
detectNewIssues	boolean	true	Toggles if new issues should be detected and returned in newIssues property
logger	Logger	console	Logger for reporting progress and encountered problems
skipComment	boolean	false	Toggles if comparison comment is posted to PR
configPatterns	ConfigPatterns | null	null	Additional configuration which enables faster CI runs
searchCommits	boolean | number	false	If base branch has no cached report in portal, extends search up to 100 recent commits

The Logger object has the following required properties:

Property	Type	Description
error	(message: string) => void	Prints error log
warn	(message: string) => void	Prints warning log
info	(message: string) => void	Prints info log
debug	(message: string) => void	Prints debug log

Standalone mode

By default, it is assumed that Code PushUp is set up to run on the whole repo with one command (standalone mode). If you want to run Code PushUp on multiple projects separately, you should enable monorepo mode.

Standalone result

In standalone mode, the resolved object will include paths to report files (JSON and Markdown formats), as well as diff files, comment ID and new issues in case of PR comparisons.

const result = await runInCI(refs, api);

if (result.mode === 'standalone') {
  const {
    // output files, can be uploaded as job artifact
    files: { current, comparison },
    // ID of created/updated PR comment
    commentId,
    // array of source code issues, can be used to annotate changed files in PR
    newIssues,
  } = result;
}

Monorepo mode

For monorepo setups, Code PushUp reports can be collected and compared individually per project. All project comparisons are then combined into a single PR comment.

Use the monorepo option to activate monorepo mode:

await runInCI(refs, api, {
  monorepo: true,
});

The runInCI function will try to detect which monorepo tool you're using from the file system. The following tools are supported out of the box:

• Nx
• Turborepo
• Yarn workspaces
• pnpm workspace
• npm workspaces

If you're using one of these tools, you can also skip auto-detection by setting monorepo option to 'nx', 'turbo', 'yarn', 'pnpm' or 'npm'.

If none of these tools are detected, then the fallback is to run Code PushUp in all folders which have a package.json file. If that's not what you want, then you can also configure folder patterns using the projects option (supports globs):

await runInCI(refs, api, {
  monorepo: true,
  projects: ['frontend', 'backend/*'],
});

Based on which monorepo tool is used, Code PushUp CLI commands will be executed using a package.json script, Nx target, Turbo task, or binary executable (as fallback). By default, these are expected to be called code-pushup, but you can override the name using the task option:

await runInCI(refs, api, {
  monorepo: 'nx',
  task: 'analyze', // custom Nx target
});

Parallel tasks

By default, tasks are run sequentially for each project in the monorepo. The parallel option enables parallel execution for tools which support it (Nx, Turborepo, PNPM, Yarn 2+).

await runInCI(refs, api, {
  monorepo: true,
  parallel: true,
});

The maximum number of concurrent tasks can be set by passing in a number instead of a boolean:

await runInCI(refs, api, {
  monorepo: true,
  parallel: 3,
});

Monorepo result

In monorepo mode, the resolved object includes the merged diff at the top-level, as well as a list of projects. Each project has its own report files and issues.

const result = await runInCI(refs, api);

if (result.mode === 'monorepo') {
  const {
    // array of objects with result for each project
    projects,
    // ID of created/updated PR comment
    commentId,
    // merged report-diff.md used in PR comment, can also be uploaded as job artifact
    files: { comparison },
  } = result;

  for (const project of projects) {
    const {
      // detected project name (from package.json, project.json or folder name)
      name,
      // output files, can be uploaded as job artifacts
      files: { current, comparison },
      // array of source code issues, can be used to annotate changed files in PR
      newIssues,
    } = project;
  }
}

Advanced usage

Faster CI runs with configPatterns

By default, the print-config command is run sequentially for each project in order to reliably detect how code-pushup is configured - specifically, where to read output files from (persist config) and whether portal may be used as a cache (upload config). This allows for each project to be configured in its own way without breaking anything, but for large monorepos these extra code-pushup print-config executions can accumulate and significantly slow down CI pipelines.

As a more scalable alternative, configPatterns may be provided. A user declares upfront how every project is configured, which allows print-config to be skipped. It's the user's responsibility to ensure this configuration holds for every project (it won't be checked). The configPatterns support string interpolation, substituting {projectName} with each project's name. Other than that, each project's code-pushup.config must have exactly the same persist and upload configurations.

await runInCI(refs, api, {
  monorepo: true,
  configPatterns: {
    persist: {
      outputDir: '.code-pushup/{projectName}',
      filename: 'report',
      format: ['json', 'md'],
    },
    // optional: will use portal as cache when comparing reports in PRs
    upload: {
      server: 'https://api.code-pushup.example.com/graphql',
      apiKey: 'cp_...',
      organization: 'example',
      project: '{projectName}',
    },
  },
});

Search latest commits for previous report

When comparing reports, the report for the base branch can be cached. If a project has an upload configuration, then the Portal API is queried for a report matching that commit. If no such report was uploaded, then the report is looked up in CI artifacts (implemented by downloadReportArtifact in ProviderApiClient). If there's no report to be found, then the base branch is checked out and the previous report is collected.

In some scenarios, there may not be a report for the latest commit in main branch, but some other recent commit may have a usable report - e.g. if nxProjectsFilter is used with --affected flag. In that case, the searchCommits option can be enabled. Then a limited number of recent commits in the main branch will be checked.

await runInCI(refs, api, {
  monorepo: 'nx',
  nxProjectsFilter: '--with-target=code-pushup --affected',
  // checks 10 most recent commits by default
  searchCommits: true,
  // optionally, number of searched commits may be extended up to 100
  // searchCommits: 30
});

Footnotes

• The {task} pattern is replaced with the task value, so the default behaviour is to list projects using npx nx show projects --with-target=code-pushup --json. The nxProjectsFilter options gives Nx users the flexibility to filter projects in alternative ways supported by the Nx CLI (e.g. --affected, --projects, --exclude, --type) - refer to options in Nx docs for details. ↩

• By default, the code-pushup.config file is autodetected as described in @code-pushup/cli docs. ↩