cypress-web-vitals

cypress-web-vitals

A Web Vitals command for Cypress.

Quantifying the quality of user experience on your site.

About

Web Vitals is a Google initiative which provides a set of quality signals and thresholds that are essential to delivering a great user experience on the web.

cypress-web-vitals allows you to test against these signals within your Cypress workflows through a set of custom commands:

• cy.vitals() - self contained command for performing a Web Vitals audit of page load performance.
• cy.startVitalsCapture() - command for starting a journey based Web Vitals audit, enabling capture of interaction to next paint (INP).
• cy.reportVitals() - command for reporting on a journey based Web Vitals audit started with cy.startVitalsCapture().

Getting started

Install the dependencies

In your terminal:

$ yarn add -D cypress-web-vitals cypress-real-events
# or
$ npm install --save-dev cypress-web-vitals cypress-real-events

Note: cypress-web-vitals currently makes use of cypress-real-events to click the page as a real user would to calculate first input delay. Hence it is needed as a peer-dependency.

Add the commands

Add the following line to your cypress/support/commands.js:

import "cypress-web-vitals";

Write your tests

describe("Web Vitals", () => {
  it("should pass the audits for a page load", () => {
    cy.vitals({ url: "https://www.google.com/" });
  });

  it("should pass the audits for a customer journey", () => {
    cy.startVitalsCapture({
      url: "https://www.google.com/",
    });

    cy.findByRole("combobox", { name: "Search" }).realClick();
    cy.findByRole("listbox").should("be.visible");

    cy.reportVitals();
  });
});

Note: this example is making use of the Cypress Testing Library for interacting with the page. This is not required for use of this package nor is it included in the package.

Examples

Example Cypress test setups with a variety of tests using cypress-web-vitals for Cypress 9.x, 10.x, 12.x and 13.x are available in the ./examples directory.

API

cy.vitals([WebVitalsConfig])

Performs and audit against the Google Web Vitals.

cy.vitals();
cy.vitals(webVitalsConfig);

Example:

cy.vitals({ firstInputSelector: "main" }); // Use the `main` element of the page for clicking to capture the FID.
cy.vitals({ thresholds: { cls: 0.2 } }); // Test the page against against a CLS threshold of 0.2.

WebVitalsConfig:

• Optional firstInputSelector: string | string[] - Selector(s) for the element(s) to click for capturing FID. Can be a single element selector or an array, all of which will be clicked. For each selector the first matching element is used. Default: "body".
• Optional onReport: Function - Callback for custom handling of the report results, e.g. for sending results to application monitoring platforms.
• Optional strict: boolean - Enables strict mode in which tests will fail if metrics with specified thresholds cannot be calculated.
• Optional thresholds: WebVitalsThresholds - The thresholds to audit the Web Vitals against. If not provided Google's 'Good' scores will be used (see below). If provided, any missing Web Vitals signals will not be audited.
• Optional url: string - The url to audit. If not provided you will need to have called cy.visit(url) prior to the command.
• Optional headers: string - Additional headers that will be provided to cy.visit() if url is provided.
• Optional auth: string - Additional auth that will be provided to cy.visit() if url is provided.
• Optional vitalsReportedTimeout: number - Time in ms to wait for Web Vitals to be reported before failing. Default: 10000.

cy.startVitalsCapture([StartWebVitalsCaptureConfig])

Starts an audit against the Google Web Vitals.

cy.startVitalsCapture();
cy.startVitalsCapture(startWebVitalsCaptureConfig);

Example:

cy.startVitalsCapture({
  url: "https://www.google.com/",
});

StartWebVitalsCaptureConfig:

• Optional url: string - The url to audit. If not provided you will need to have called cy.visit(url) prior to the command.
• Optional headers: string - Additional headers that will be provided to cy.visit() if url is provided.
• Optional auth: string - Additional auth that will be provided to cy.visit() if url is provided.

cy.reportVitals([ReportWebVitalsConfig])

Completes and reports on an audit against the Google Web Vitals.

cy.reportVitals();
cy.reportVitals(reportWebVitalsConfig);

cy.reportVitals({ thresholds: { inp: 500 } }); // Test the page against against an INP threshold of 500.

ReportWebVitalsConfig:

• Optional onReport: Function - Callback for custom handling of the report results, e.g. for sending results to application monitoring platforms.
• Optional strict: boolean - Enables strict mode in which tests will fail if metrics with specified thresholds cannot be calculated.
• Optional thresholds: WebVitalsThresholds - The thresholds to audit the Web Vitals against. If not provided Google's 'Good' scores will be used (see below). If provided, any missing Web Vitals signals will not be audited.
• Optional vitalsReportedTimeout: number - Time in ms to wait for Web Vitals to be reported before failing. Default: 10000.

WebVitalsThresholds

• Optional lcp: number - Threshold for largest contentful paint (LCP).
• Optional fid: number - Threshold for first input delay (FID).
• Optional cls: number - Threshold for cumulative layout shift (CLS).
• Optional fcp: number - Threshold for first contentful paint (FCP).
• Optional ttfb: number - Threshold for time to first byte (TTFB).
• Optional inp: number - Threshold for interaction to next paint (INP).

Google 'Good' scores

{
  "lcp": 2500,
  "fid": 100,
  "cls": 0.1,
  "fcp": 1800,
  "ttfb": 600,
  "inp": 200
}

Custom Reporting

It can be useful to have direct access to the raw data so you can generate custom reports, send to APM, etc.

This can be achieved through the optional onReport callback for cy.vitals() or cy.reportVitals() which receives the raw report before cypress-web-vitals passes or fails the test.

describe("Web Vitals", () => {
  it("should log the report to APM", () => {
    cy.vitals({
      url: "https://www.google.com/",
      onReport({ results, strict, thresholds }) {
        console.log(results); // { lcp: ..., fid: ..., }
      },
    });
  });
});

The report results contains values for all signals, not just the values specified in the provided or default thresholds. Signals that couldn't be obtained are reported as null.

How does it work?

When using the cy.vitals() command:

• The url is visited with the HTML response intercepted and modified by Cypress to include the Web Vitals module script and some code to record the Web Vitals values.
• The audit then waits for the page load event to allow for the values of LCP and CLS to settle; which are subject to change as different parts of the page load into view.
• Several elements (if exist) starting with the provided element(s) (based on firstInputSelector) are then clicked in quick succession to simulate a user clicking on the page to aid FID reporting. Note: if choosing a custom element(s), don't pick something that will navigate away from the page otherwise the plugin will fail to capture the Web Vitals metrics.
• Next the audit simulates a page visibility state change which is required for the CLS web-vital to be reported.
• The audit then attempts to wait for any outstanding Web Vitals to be reported for which thresholds have been provided.
• Finally the Web Vitals values are compared against the thresholds, logging successful results and throwing an error for any unsuccessful signals. Note: if the audit was unable to record a web-vital then it is logged, but the test will not fail.

The cy.startVitalsCapture() and cy.reportVitals() commands perform steps 1-2 and 4-6 respectively. There is no clicking of elements (step 3) with these commands as the expectation is for you to provide your own customer journey interactions.

Contributing

Please check out the CONTRIBUTING docs.

Changelog

Please check out the CHANGELOG docs.

License

cypress-web-vitals is licensed under the MIT License.