@redocly/christi

API Test Runner

The key benefits of christi:

• Declarative integration testing of HTTP API endpoints.
• Reuses OpenAPI definitions to generate tests and automate assertions based on schemas.
• Lightweight and fast enough to run from CICD with each commit.

You MUST have a working API server running in order to run the tests.

Quickstart

Step 1: Install christi

npm i @redocly/christi --registry http://3.236.95.236:8000/

Step 2: Generate a test config file

christi auto-generate-config <your-OAS-definition-file>

Step 3: Run the tests

christi test-file.yaml

Step 4: Edit the test config file

Create your own flows (chained requests) for testing.

Commands

Run tests

christi <your-test-file> [-f | --flow] [-v | --verbose]

Run API tests based on the test-file config.

Options

Option	Type	Description
-f, --flow	Array<String>	Flow names to run. Example: christi test-file.yaml --flow first-flow second-flow
-v, --verbose	boolean	Verbose mode. Example: christi test-file.yaml --verbose
--har-output	string	Path for the har file for saving logs. Example christi test-file.yaml --har-output='logs.har'

Run the tests by running the following command: christi <your-test-file>.

Auto generate tests based on OpenAPI

christi auto-generate-config <your-OAS-definition-file> [-o | --output-file] [--extended]

Auto generate the test-config file based on the OpenAPI definition file. If examples are provided in the OpenAPI definition, they are used as input data for test requests. In case schema is provided, config generates with fake data based on the definition schema. By default, data for requests come from the definition in run-time. To materialize tests with the data, use the --extended option.

:::info Tip

Use --extended option only to learn how christi uses data from a definition. In most cases it is better to use the data from the definition in run-time.

:::

Options

Option	Type	Description
-o, --output-file	string	Path to the OAS definition file. If file name not provided default name is used - auto-generate.test.yaml. Example: christi auto-generate-config OAS-file.yaml -o=example.tests.yaml
--extended	boolean	By default, data for requests come from the definition in runtime. This option generates a test config file with data populated from the definition. Example: christi auto-generate-config OAS-file.yaml -o=example.tests.yaml --extended.

Help

christi [--version] [--help]

Prints the version or help information.

Configuration file

The configuration file is a YAML file with the following fixed fields.

Property	Type	Description
flows	Map[string, [TestCase]]	REQUIRED. Define sequence of API requests and expected responses.
before	[TestCase]	List of calls that run before the flows begin and before constructing defaults.
definition	string	Path to your OAS definition.
inherit	string	Definition data reuse strategy. Possible values: auto, none. Default value: auto.
serverUrl	string	Base path for all the requests with relative path.
defaults	Defaults	Default parameters used for every request.

TestCase object

Property	Type	Description
path	string	REQUIRED. The path to the API is appended to the server.
method	string	REQUIRED. HTTP method. Possible values: GET, POST, PUT, PATCH, DELETE, OPTION, (TODO add more).
name	string	Unique name assigned to the test useful for output and request chaining.
requestBody	object	JSON (YAML) object.
parameters	[Parameter]	List of parameters and values.
expect	Expect	Expectations of the response.
assert	Assertion	Custom assertion. See the assertion description for more details.
inherit	string	Definition data reuse strategy. Possible values: auto, none. Default value: auto.
content-type	string	Request body content type. It is different from the Content-Type header. Use it only if your server doesn't accept this header.

Flows example

This example contains the flows object:

flows:
  my-testing-flow:
    - path: https://jsonplaceholder.typicode.com/posts/1
      method: get
      expect:
        status: 200
        body:
          matchesObject:
            id: 1

Each flow is isolated and consists of steps declared as an array. It MUST contain the path and method properties. You can also specify a name to able to reference its response in the next steps.

Chaining requests example

You can also create a chain of requests and reference each response (see How to reference response):

flows:
  my-testing-flow:
    - name: createItem
      path: /items
      method: post
      requestBody:
        title: foo
      expect:
        status: 201
    - path: /items/{id}
      method: get
      parameters:
        - name: id
          in: path
          value: ${responses.createItem.body.id}
      expect:
        status: 200
        schema:
          type: object
          properties:
            id:
              type: number
            title:
              type: string

The status and schema fields could be omitted and inferred from the OAS definition.

Expect object

Property	Type	Description
status	integer | [integer]	HTTP status code of the response (or list of possible status codes).
schema	object	JSON schema. If an OpenAPI definition is provided, the appropriate schema is determined automatically (and if this value is set it overrides that).
body	Body	Compare expectations about the response body.
mimeType	string	Expected mime type of the response.

Body object

Property	Type	Description
equal	any	Compares response body to be strict equal to passed values
matches	string	Compares response body to match passed string.
matchesObject	object	Compares properties in the response body to those in the defined object.

Parameters object

Property	Type	Description
name	string	REQUIRED. Parameter name.
in	string	REQUIRED. Parameter location. Possible values: query, header, path, cookie.
value	any	REQUIRED. Parameter value.

Defaults object

Property	Type	Description
parameters	[Parameter]	Default parameters to include in every request.

Authorization example

defaults:
  parameters:
    - in: header
      name: Authorization
      value: Bearer ${secrets.TOKEN}

The runner gets secrets from the env variables:

TOKEN=<your-token> christi <your-test-file>

Before example

before configures API requests that run before all of the flows and before constructing Defaults and has the same structure as a flow.

before:
  - path: /signin
    name: signIn
    method: post
    inherit: none
    requestBody:
      email: [e-mail]
      password: ****
    expect:
      status: 200

Inheritance works the same as for flows. The only difference is that you can refer to this block from other flows:

flows:
  my-testing-flow:
    - name: createItem
      path: /items
      method: post
      parameters:
        - in: header
          name: Authorization
          value: Bearer ${ beforeResponses.signIn.body.token }
      requestBody:
        title: foo

Or you can refer from defaults object:

defaults:
  parameters:
    - in: header
      name: Authorization
      value: Bearer ${beforeResponses.signIn.body.token}

How to reference responses

Responses are scoped to their flow. In order to extract any data (status, body, etc.) from other responses these responses must be declared prior to that request and must have a name field provided. Example: ${responses.<yourRequestName>.body} OR ${responses.<yourRequestName>.status}

There might be only one reference ${...} per value (for example, you cannot concatenate multiple values into a single value). If it is embedded in a string (for example, Bearer ${responses.signIn.body.token}) it is treated as a string.

See Context for more data available.

Definition reuse strategies

If you provided an OAS definition, you can reuse it in the flow.

You can define the strategy using inherit keyword in each step of the flow. By default it is inherit: auto which means that the runner will try to figure out parameters (only required ones), requestBody, expect.status, and expect.schema from the definition if it exists. If you want to redefine any of these fields you can do it explicitly.

You can opt-out any data from the definition by using inherit: none on each test step. It is also possible to define the default inherit strategy on the top level for the whole test file.

Priority order

The runner first applies the data from the definition (least priority) if provided, then defaults, and then the flows (highest priority).

Context

Context is accessible by using ${ <fieldsFromContext> } notation.

Property	Type	Description
secrets	Map[string, string]	Secrets passed as environment variables.
definition	Map[string, any]	JSON representation of the linked OAS definition.
responses	Map[string, any]	Responses from requests of the specific flow. NOTE: responses are scoped to their flows (you cannot access response from different flow). responses fill in as it goes through the requests. See How to reference responses for details.
requests	Map[string, any]	Requests data of the specific flow. NOTE: requests are scoped to their flows (you cannot access response from different flow).
beforeResponses	Map[string, any]	Access responses from TestCase items that run before as part of the setup process.
beforeRequests	Map[string, any]	Access requests of TestCase items that run before as part of the setup process.
faker	Faker Object	Faker to generate random data. See Faker Object for more details.
storage	Storage	Key-value storage object.

Key-Value Store

You can use key-value storage in the Context to store and retrieve data between requests. Example

flows:
  using-storage:
    - path: /items
      method: post
      assert: |
        (expect, response, ctx) => {
          ctx.storage.setItem('created-item-id', response.body.id)
        }
    - path: /items/{id}
      method: get
      parameters:
        - in: path
          name: id
          value: ${storage.data.created-item-id}
      expect:
        status: 200
        body:
          matchObject:
            id: ${storage.data.created-item-id}

Examples

If you have an OAS definition, you may provide path to the file:

definition: ./path/to/definition.yaml

and then reference in a flow or in the defaults section, e. g.: ${definition.paths.items/{id}.responses.201.content.application/json.schema}.

Faker Object

Data Type	Type	Usage example	Output example
string	FakeString	${ faker.string.fullName() }	Camille Mohr
number	FakeNumber	${ faker.number.integer({ min: 10, max: 20 }) }	12
address	FakeAddress	${ faker.address.city() }	Lake Raoulfort
date	FakeDate	${ faker.date.past() }	Sat Oct 20 2018 04:19:38 GMT-0700 (Pacific Daylight Time)

Example:

flows:
  create-feedback:
    - name: addFeedback
      path: /feedback
      method: post
      requestBody:
        contentId: ${ faker.string.uuid() }
        rating: '${ faker.number.integer({ min: 1, max: 5 }) }'
        suggestion: A suggestion.
        sentiment: true
        reason: one

FakeString

Method	Parameters	Usage example
userName	-	${ faker.string.userName() }
firstName	-	${ faker.string.firstName() }
lastName	-	${ faker.string.lastName() }
fullName	-	${ faker.string.fullName() }
email	{ provider? : string }	${ faker.string.email({ provider: 'gmail' }) }
uuid	-	${ faker.string.uuid() }
string	{ length?: number }	${ faker.string.string({ length: 5 }) }

FakeNumber

Method	Parameters	Usage example
integer	{ min?: number; max?: number }	${ faker.number.integer({ max: 30 }) }
float	{ min?: number; max?: number; precision?: number }	${ faker.number.float({ precision: 0.001 }) }

FakeAddress

Method	Parameters	Usage example
city	-	${ faker.address.city() }
country	-	${ faker.address.country() }
zipCode	-	${ faker.address.zipCode() }
street	-	${ faker.address.street() }

FakeDate

[Note: is there a way to format the date?]

Method	Parameters	Usage example
past	-	${ faker.date.past() }
future	-	${ faker.date.future() }

Assertion

You can write your custom assertions by providing string as a valid JavaScript function as in the following example.

assert: |
  (expect, response) => {
    expect(response.body.message).match(/OK/i);
  }

Or use a reference to a .js file as in the following example.

assert:
  $ref: ./custom-assertion.js

Assertion function accepts the following arguments.

Argument	Description
expect	Expect function. (Caveat: currently it supports function expressions only, e.g. () => {} or (function a () {}). Function declaration must be wrapped by grouping operator () in order vm.Script to parse it properly).
response	Response of the request under testing.