@archer-edu/qa-test-cli

QA TEST CLI

A CLI tool for managing QA tests with Ghost Inspector & Lighthouse.

Pre-Installation Requirements

• A stable version of Chromium is required for Lighthouse analysis.

• A .env in the project root with the following set:

GHOST_API_KEY={{GHOST_INSPECTOR_API_KEY}}
CHROME_PATH="which chromium"

Check out this article for QA Test CLI if unsure how to get the values required for configuration.

Optional

The environment variable GHOST_SUITE_FOLDER_ID can be utilized by qa-test-cli to place any suite or test cases pushed to the Ghost Inspector dashboard.

Please consult with the team before changing this value from the default.

GHOST_SUITE_FOLDER_ID={{GHOST_INSPECTOR_FOLDER_ID}}

You can also add in the Ghost Inspector Organization Id (you will be prompted if you don't).

GHOST_ORGANIZATION_ID={{GHOST_INSPECTOR_ORGANIZATION_ID}}

The next step will prompt for these values if skipped.

Installation & Initialization

The qa-test-cli uses Node 14+ and can be installed via Yarn or NPM.

yarn add --dev @archer-edu/qa-test-cli && yarn testctl init
--OR--
npm install --save-dev @archer-edu/qa-test-cli && npm testctl init
--OR--
npx @archer-edu/qa-test-cli init

This will install qa-test-cli and step through the creation of a qa-test-cli.yml config in the project root. For more info on available configuration options, check out the Schema for QA-Test-CLI config

Using Ghost Inspector with TestCtl

Create Ghost Inspector Suites

Once initialized, it's possible to scaffold test suites for Ghost Inspector.

To set up a test suite, run the following to guide you through the process:

testctl {scaffold-suite | ss} [DESIRED_SUITE_NAME]

The [DESIRED_SUITE_NAME] is an optional pass-in argument, if not supplied this command will prompt you for it.

Note: When prompted for the "Ghost Inspector Suite Id" (i.e. in the case of an existing suite created in the dashboard), if no is selected then testctl will attempt to retrieve a new one from Ghost Inspector.

When complete, it will create a qa-test-suite.yml config for managing Ghost Inspector test suites in the set directory.

For more info on available configuration options, check out the Schema for QA-Test-CLI config

Create Ghost Inspector Tests

After a suite is created, run the following to add a test case:

testctl {scaffold-test | st} [SUITE_KEY] [DESIRED_TEST_NAME]

The [SUITE_KEY] & [DESIRED_TEST_NAME] are optional pass-in arguments, if not supplied this command will prompt to supply them.

Note: When prompted for the "Ghost Inspector Test Case Id" (i.e. in the case of an existing test created in the dashboard), if no is selected then testctl will attempt to retrieve a new one from Ghost Inspector for the associated "Ghost Inspector Suite Id"

This command will create a [test-key].gi-test.yml config in the set directory.

For more info on available configuration options, check out the Schema for Ghost Inspector test cases

Note: If a suite-key is not provided, and only 1 suite is present in the qa-test-cli config, it will default to that.

This command will prompt for a name for the test if one was not provided, then prompt for the test's starting URL path. The result produces a [test-key].gi-test.yml config in the default .tests directory relative to the suite config.

Running Ghost Inspector Tests

Ghost Inspector tests can be run by targeting a build distribution folder or a valid URL.

yarn testctl {ghostinspector | gh} <TARGET_BUILD_DIRECTORY|TARGET_URL>

Note: The default port for directories is set to 3000.

Using Hosted URL

yarn testctl {ghostinspector | gh}  https://localhost:3002

Note If no port is provided, the default HTTP (80) or HTTPS (443) port is used.

Pulling & Pushing Ghost Inspector Tests

To pull or push tests to the Ghost Inspector dashboard:

1. Make sure the targeted suites in the `qa-test-cli.yml` have `ghostInspectorSuiteId` set.
2. Make sure you have `GHOST_API_KEY` set in a `.env` file.

Without these settings present, the CLI will not be able to push to or pull from the Ghost Inspector dashboard.

testctl pull

testctl push

Configuring Scheduling for Ghost Inspector Tests

You can configure scheduling for Ghost Inspector tests by editing the qa-test-suite.yml configuration file. Add a schedule property to specify when the test suite should run. Here's an example of how to configure the schedule:

schedule:
  - days: [0, 1, 2, 3, 4, 5, 6]   # Run on all days of the week
    hours: [9]                    # Run at 9 AM
    minutes: [0]                  # Run at the start of the hour

In this example, the test suite is scheduled to run every day of the week at 9 AM.

For more details on configuring the schedule, refer to the Schema for QA-Test-CLI config.

Using Lighthouse with TestCtl

The Lighthouse configuration added by the init process includes two passes based on best practices for mobile and desktop.

For more information on available configuration options, check out the Schema for QA-Test-CLI config.

Note: You can configure qa-test-cli.yml with any valid option available in Lighthouse.

Basic Usage

Lighthouse analysis can be run by targeting a build distribution folder or a valid URL.

yarn testctl {lighthouse | lh} <TARGET_BUILD_DIRECTORY|TARGET_URL>

Note: The default port for directories is set to 3000.

Note: For SPAs, ensure you set the entryFile config or pass the provide the --entry_file argument for accurate routing & rendering (i.e. index.html).

Sitemap Crawl

To perform an analysis for every route, you must include the -i, --include_sitemap arg. If the project's sitemap.xml is in a different location than the default (/sitemap.xml), pass the --sitemap_path arg with a valid URI path.

Note: the -s, --sitemap_path arg defaults to '/sitemap.xml'.

yarn testctl lh <TARGET_BUILD_DIRECTORY|TARGET_URL> -i -s <PATH_TO_SITEMAP>

Regex Filtering for Sitemap URLs

You can now filter URLs from the sitemap using regular expressions, allowing you to include or exclude URLs based on specific patterns. This feature is useful for breaking down large sitemaps into smaller, more manageable sets for analysis.

Inclusive Filtering Example

To include only URLs that contain the word "technology":

yarn testctl lh https://example.com -i -r ".*\/technology(/|$)" 

This command will only analyze URLs that match the pattern /technology/.

Explanation:

• .*: Matches any character (except newlines) zero or more times. This acts as a wildcard for the protocol and domain part of the URL.
• /technology: Matches the /technology path exactly.
• (/|$): Matches either a forward slash / (indicating further subdirectories) or the end of the URL $.

Exclusive Filtering Example

To exclude URLs that contain the word "archive":

yarn testctl lh https://example.com -i -r "^(?!.*\/archive).*$"

This command will analyze all URLs except those that match the pattern containing "archive".

Explanation:

• ^(?!.*/archive): This uses a negative lookahead to exclude any URL that contains the /archive segment.
• ^: Asserts the start of the URL.
• (?!.*/archive): The negative lookahead ensures that the pattern does not match if /archive is anywhere in the URL.
• .*$: Matches the rest of the URL.

Local Development

Clone or download the @archer-edu/qa-test-cli repository.

Now just install all dependencies and build in the project root.

yarn install
yarn run build

To test any changes in another project, link your local repository to the other project.

In your local @archer-edu/qa-test-cli repository:

npm link

Then in the target project:

npm link @archer-edu/qa-test-cli