@leaflink/oast

Frosting Open Api Spec Ingestor

Table of Contents:

• Build
• Make Commands
• CLI
  • Usage and Examples
  • Command Options
  • Non Zero Exit Codes
• Ingestor Transform Configuration File
  • Top Level File Structure
  • Defining Source
  • Defining Transformations
  • Defining Operations
  • Transformation Functions
    • preplace
    • vadd
    • vreplace
• Authoring New Transforms
  • BaseTransformation Class
  • Options Schema
  • Connecting New Transform Function to Ingest

Build

Suggested Node Version is 18.10.0. We also use nvm for node version management.

To build, run make build

Then run oast in your shell and verify install ran correctly

Makefile Commands

make test - Runs Jest tests with --silent

make test-ci - Runs Coveralls Report

make test-no-silent - Runs Jest tests without --silent

make lint - Check linting of JS files

make lint-fix - Fixes all auto fixable linting issues

Command Line Interface

Usage and Examples

You can set an .env file with GITHUB_TOKEN=<your-token> to bypass providing the -t argument when using github urls.

# Example Success with --silent flag
oast ingest --input ./upstream/marketplaceOas.yaml --config=./transformations/marketplace.yaml --output ./dist/ --silent

Output[]
0

# Example Failure on silent flag (does not output info logs but does not fail silently)
oast ingest --input ./upstream/marketplaceOas.yaml --config=./transformations/marketplace.yaml --output ./dist/ --silent

Output[]
Transformation Failed: (transformation.name} operation {delete} not supported
Exit code 1

# Example Success not silent
oast ingest --input ./upstream/marketplaceOas.yaml --config=./transformations/marketplace.yaml

Output[]
Transformation {transformation.name} succeeded performing {preplace} on document
Transformation {transformation.name} succeeded performing {vadd} on document
Success! New Output documented located at {output path}

# Example Failure not silent
oast ingest --input ./upstream/marketplaceOas.yaml --config=./transformations/marketplace.yaml

Output[]
Transformation Failed: (transformation.name} operation {delete} not supported
Exit code 1

Command options

help List all oast command options

ingest perform cli ingestor tools main transformation on upstream oas document

• --config or --c (required) location of the transformation config to be applied to the input file either path or github rawfile url. used in conjunction with transformation file.

• --input or -i (required) location of the upstream open api spec file, either a path or github rawfile url.

• --output or -o (optional) defaults to {CURRENT_DIRECTORY}/dist/{SOURCE_NAME}.yaml location of the output file. the name of the file will be the original upstream oas document. Only can be used in conjunction with --input

• --silent or -s (optional) outputs only the exit code: 0 on success, non-zero on failure

• --token or -t (required when using URL args) github access token, or personal access token.

Non Zero Exit Codes

As this will likely be integrated into CI pipelines, the tool will raise non zero in a number of cases when the command fails. If the command succeeds, a code 0 will be provided. The non zero exit code circumstances are provided below

The application cli tool will output a non zero exit code when:

• The operation provided within the transformation config is not supported

• The command oast transform is run and document is not found provided to the -i argument

• The json path provided within the transformation config evaluates to nothing (in some cases)

• Any uncaught exceptions when running the main oast transform command

CODE	Description
0	successful
1	general exception occured
9	invalid input provided: command args, jsonPath, or FunctionOptions

Ingestor Transform Configuration File

YAML configuration file for targeting generated OAS documents, and transforming them in a declarative manner to suit our needs.

Top Level File Structure:

source:
  location: ...<location options>...
target:
  transformations: ...<transfrom options>...

The configuration yaml file requires two top level keys to be defined. The source is used along with the location options to find the source project that the OAS document is located in. This will be used to load the document prior to performing desired transformations.

The target key allows developers to write their desired transformation configurations that will be applied to the source OAS document.

Defining Source:

Location Options:

Github:

Targets a github repository and branch to source the OAS document from.

git:
  url: <repository-url>
  branch: <target-branch>
  tag: <tag>

example usage:

source:
  location:
    git:
      url: [e-mail]:LeafLink/leaflink.git
      branch: main
      tag: 1.0.0

Defining Transformations:

A transformation is comprised of three parts. The name and description for the transformation, and a jsonPath query. Finally you must provide an operation configuration that will be performed on the resulting jsonPath query results.

- name: <descriptive_name>
    description: <describe_operation>
    given: <jsonPath_query>
    then:
      ...<operation configuration>...

example usage:

- name: Update Frosting API Paths
    description: Replace upstream API paths with Gateway specific paths.
    given: $.paths[*]
    then:
      ...<operation configuration>...

This would apply the operation configuration to all the results returned by the jsonPath query. Note that we also gave both a name and description that defines what our intent was behind this transformation.

Defining Operation Configuration:

The operation configuration is how we perform work on the resulting jsonPath query. Each transformation can only have one associated operation, and it must be a child of the then key . Below is a simple example of how to define an operation.

Note: The keys under functionOptions are dependent on the type of operation you are trying to perform.

...<transformation definition>...
then:
  function: preplace
  functionOptions:
  field: '@key'
  pattern: '^(/v1)(/leaflink)'
  replace: '/json/marketplace'

The function key defines what operation we are going to run by name. functionOptions defines the required options of this function these vary by the type of function you are trying to use.

So putting it all together we end up with a single transform configuration:

target:
  transformations:
    - name: Update Frosting API Paths
      description: Replace upstream API paths with Gateway specific paths.
      given: $.paths[*]
      then:
        function: preplace
        functionOptions:
        field: '@key'
        pattern: '^(/v1)(/leaflink)'
        replace: '/json/marketplace'

This will query the OAS document for any elements that match the query $.paths[*]. On those resulting elements we will use the regex pattern to replace all key values with ‘/json/marketplace’

Transformation Functions:

preplace

Regex replace on either a key or value on the resulting elements matched from the jsonPath query.

function: preplace
functionOptions:
  field: '@key' | '@value'
  pattern: string
  replace: string

FunctionOptions:

• field (required): Specify what value to target on the element, either ‘@key’ or ‘@value’

• pattern (required): valid regex string used for matching.

• replace (required): The string value to use when regex replacing on the field.

example usage:

function: preplace
functionOptions:
  field: '@key'
  pattern: '^(/v1)(/leaflink)'
  replace: '/json/marketplace'

---

vadd

For Objects:

Adds a given object to the desired path location

Note: Both the target, and value must be valid JSON objects.

function: vadd
functionOptions:
  value:
  ...<keys / values to add>...

FunctionOptions:

• value (required): A combination of yaml keys / values to add

example usage:

function: vadd
functionOptions:
  value:
    security:
      - bearerAuth: []

For Arrays:

Appends to a given array at the desired path location

Note: Both the target and value must be Arrays. Be sure that your jsonPath resolves to an Array, and not an object containing the array

example usage:

function: vadd
functionOptions:
  value:
    - name: someObjName
      value: someObjValue
    - name: anotherObjName
      value: anotherObjValue

---

vreplace

Replaces values at the given location, the jsonPath given, must return a valid path.

Note: The target value will be fully overwritten with the supplied value.

function: vreplace
functionOptions:
  value:
  ...<Object | Value>

FunctionOptions:

• value (required): The value that will replace the target of the jsonPath query.

example usage:

function: vreplace
functionOptions:
  value:
    - url: https://api.leaflink.com
      description: LeafLink API production URL.
    - url: https://staging-api.leaflink.com
      description: LeafLink API staging URL.

Replacing the values of a targeted JSON array with new values.

function: vreplace
functionOptions:
  value:
    bearerAuth:
      type: http
      scheme: bearer
      bearerFormat: JWT

Replacing the JSON bearerAuth object value at the given path.

function: vreplace
functionOptions:
  value: 'some string value'

You can also replace simple values such as ints, strings or arrays.

---

Authoring New Transform Functions

If you find the need for a certain type of transform that isn't already implemented, you can extend the BaseTransformation class to implement a new transform function.

Base Transform Class

This class provides a common abstract method transform that you will need to override in your custom transform function. The function signature is: async transform(options, spec)

• options: This is an object containing the query, as well as the FunctionOptions that will be passed in via the config.
• spec: The JSON representation of the OpenAPI spec yaml file. This will be passed in via the ingest function.

example:

class MyTransform extends BaseTransform {
  async transform(options,spec){
    // your implementation
  }
}

Options Schema

The BaseTransform class implements a few common transform function checks. The two main ones are:

• Valid Options Schema
• Valid jsonPath query

These checks are performed on each transformation invocation.

We require you to write an schema object that represents the rules surrounding the FunctionOptions that your transform will consume. We are using the yup library to do object validation.

You can add / modify these schemas at src/transformer/schemas.

example schema for preplace:

// defines the shape of the options object required for this function
const preplaceOptionsSchema = yup.object().shape({
  pattern: yup.string().required(),
  replace: yup.string().required(),
  query: yup.string().required(),
  field: yup.mixed().oneOf(['@key', '@value']).required()
});

If any of the validations fail, the function will not run.

In the example above we are specifying that pattern, replace, query and field are required, as well as the expected types of each. On field we are specifying that it is an enum that only accepts two types of input @key and @value.

When writing the schema, you want to be as strict as possible, this helps other developers using transforms to properly configure their FunctionOptions configuration. As well as prevent unexpected side effects by providing erroneous inputs.

Connecting New Transform Function to Ingest

Once you have written both the transform function, and it's accompanying schema. You will need to wire it into the transfomer for it to be used by the Ingestion command.

The transform is treated as a simple container object: const transformer = {};

Simply import both the transform function, and it's spec into the src/transformer/transformer.js file.

You then need to add the name of your function to the transform object, with a reference to your transform class.

example:

// ...other transform imports...

const MyTransform = require('from/some/path');
const myTransformOptionsSchema = require('from/some/schema/path');

const transformer = {};
// ... other transform assignments ...
transformer.myTransform = new MyTransform(myTransformOptionsSchema);

Note: The varible name you are assigning your transform class reference to on the transform object will become the configuration name for your transform function. So using previous example the function value in the configuration would be myTransform.

The transform function is now wired into the transformer and running ingest with the proper configuration will run it.

Semantic Versioning Release

The number of times we've had to a manual release is: 5