tf-requests

TF Requests

A utility script for interacting with the Testing Farm (TF) API, tailored for GitLab CI environments.

Testing Farm API documentation: https://testing-farm.gitlab.io/api/

Overview

tf-requests is designed to facilitate interactions with the Testing Farm service directly from a GitLab CI job. Using environment variables, possibly sourced from various CI environments, this script constructs, validates, and sends payloads to the Testing Farm API. It then monitors the job's progress and reports back the results, ensuring a seamless integration into your CI/CD pipelines.

Functionalities

• Load configuration from environment variables.
• Generate a payload based on the configuration, variables, and secrets.
  • Calculate the proper TF compose for each case. Including the jobs for physical boards.
• Make a POST request to the specified API endpoint with the generated payload.
• Monitor the job associated with the request for completion.
• Retrieve and display the results of the job.
• Store results for future reference.
• Handle errors and failures based on predefined error codes.

Use cases

General use cases

• Trigger a build TF job and await the results.
• Trigger an integration test TF job on an image built by the same pipeline and await the results.
• Trigger a package test TF job on already published images (built by a different pipeline) and await the results.

Build TF job

It's a tmt plan that runs on TF and builds a new OS image using a compose generated by the same GitLab pipeline. It needs some secrets to talk to AWS.

All build jobs run on virtual instances (AWS) and using a standard TF compose (CentOS Stream9 or RHEL9-nigthly)

Integration test TF job

It's a tmt plan that runs on TF using a pre-existing OS image. It runs some tests inside the image.

They could be integration tests (smoke-tests, remote-update-test or environment-details) that run in a OS image created by the same GitLab pipeline.

Some test jobs run on virtual instances (AWS), but others on physical boards (like QDrive3 or RideSX4). Those boards need a special TF compose format (a JSON-like string containing the URLs for the rootfs and boot images). For the others, the TF compose is the name of the AMI generated for the build job that builds the testing image. It should be passed in a variable (IMAGE_KEY).

Package test TF jobs

As in the previous test job, tt's a tmt plan that runs on TF using a pre-existing OS image. It runs some tests inside the image.

Unlike the previous test case, the package tests use an image already built and published by another pipeline. It could be the latest one (nightly) or a formal release (like ER3 or similar).

This works only downstream, as it's where we run the package testing.

Specific use cases

• Trigger jobs upstream
  • Build job
  • Smoke-test job
    • On regular virtual instances
    • On physical boards
  • Remote-update test job
  • Environment details "test" job
• Tigger jobs downstream
  • Build job
  • Smoke-test job
    • On regular virtual instances
    • On physical boards
  • Remote-update test job
  • Environment details "test" job
  • Package test
    • On regular virtual instances
    • On physical boards

Examples

You can find examples of how to use tf-requests in GitLab jobs at the the .gitlab-ci.yml file, at the integration jobs. There are examples of different jobs with the expected environment variables:

• build-test: a job for building an image.
• smoke-test: a job for testing a image built by the same pipeline.
• envinronment-test: a job for checking the image's environment.
• package-test: a job for testing one specific package (glibc) on a nightly image.
• package-test-meta: the same as the previous one, but using META_ variables for backward compatibility.
• package-test-aboot: the same as the first package test, but for testing it on a QDrive3 board.

Extra features

• Just calculate and show the payload (no API call made)

• Pass a list of the ENV variables names for the variables and the secrets, so they get use in the right payload section

  Example:

  For a GitLab CI job defined with the following variables:

  ARCH=aarch64
  TF_API_KEY=YOUR_API_KEY
  AWS_SECRET=some_secret_key
  CI_REPO_URL=http://your-repo.git
  ...
  

  You can run the script with:

  tf-request --secrets AWS_SECRET --variables ARCH,CI_REPO_URL
  

• Support meta variables (META_VARIABLES, META_CONTEXT, META_SECRETS). These variables are supported for backawrd compatibility.

• Support variables with a path for the pyload structure. Use a separator that can be a shell variable name, like __. These new type of variables allow send new variables without the need of hardcoding them in the request or the use of the META variables. Please, use this for new variables instead of the META if possible.

  For example, for this element:

  {
    "top_dir": {
      "second_level": {
        "key": "value"
      }
    }
  }
  

  The ENV variable would be:

  top_dir__second_level__key="value"
  

Usage

• Set up your GitLab CI job with the necessary environment variables. This can include pipeline variables, job-specific variables, and any other required data.

• Run the tf_request in your job script:

  tf-requests --secrets SECRET_1,SECRET_2 --variables VAR_1,VAR_2
  

• The script will construct the payload, send it to the Testing Farm API, and monitor the job status. Once the job completes, results will be reported directly in the GitLab CI logs.

Example

For a GitLab CI job defined with the following variables:

ARCH=aarch64
TF_API_KEY=YOUR_API_KEY
AWS_SECRET=some_secret_key
CI_REPO_URL=http://your-repo.git
...

You can run the script with:

tf-requests --secrets AWS_SECRET --variables ARCH,CI_REPO_URL

Development

Setting up your development environment:

git clone <repository-url>
cd tf_requests
pdm install --dev

Running tests (pytest):

pdm run test

Running linting (flake8):

pdm run lint

Check types (mypy):

pdm run check-types

Check formatting (black and isort):

pdm run check-formatting