@stoplight/graph

Graph (temporary name)

Graph is a PoC for what will be the key piece of tech at Stoplight to analyze projects, extract and normalize relevant structured data, and query that data in memory.

Currently, the focus is on OAS2, OAS3 specifications and, although it's not a top priority, GraphQL might be the next supported format.

Motivations

• To date, many components in Stoplight have been dealing with API specification formats in different ways:

  • Prism, the mock server, is directly dealing with the source file of the only supported format so far (OAS2)

  • The Studio POC has an internal Graph library that's parsing an OpenAPI file and decomposing into a series of nodes that are then enriched with UI-specific elements (such as css classes, components to render, Icon providers)

  • The Platform Designer is directly dealing with source files, and laying form widgets/ui on top of them.

• All the various components in Stoplight have different opinions on the level of detail of a particular part of the document, as well as different way to deal with it. There's a lot of repeated code, as well as some unique parts depending on the piece of software dealing with the specification in particular.

Design Decisions

Node identification

Each node has three identifiers

• id - locally unique identifier. Any two nodes withing one graph instance WILL have distinct id. Some nodes that belong to two different graph instances MAY have different id.
• graphId - globally unique identifier. Any two nodes representing the same entity in a graph's hierarchy will have identical graphId, even if they belong to different graphs. graphId is applicable to both Virtual Nodes and Physical Nodes
• absolutePath - globally unique identifier that co-locating a node with a particular part of the file that node originates from. Similar to graphId but only applicable to Physical Nodes

Main node types

• Physical Node - nodes that originate directly from a file and represent a portion of the file's content. E.g. it can be a nested element in a json file.
• Virtual Node - nodes that have no physical representation in a file, but were most likely derived from a Physical Node. A good example of such node is the IHttpOperation node.

Graph "Laws"

1. Physical Node's parent can only be of Physical Node type
2. There is a one-to-one relation between a hook and a nodeType

Aims

As we want to support more API Specification formats, it is clear that the current solution wouldn't have scaled. That's what Graph (temporary name) aims to solve by providing:

1. A consistent set of elements and terminology to represent structured data, regardless of the format (OAS2, OAS3, GraphQL, next fancy api stuff).
2. A consistent way to source this data (a directory on your computer, a postgres database, etc).
3. A plugin system that hooks into the Graph construction lifecycle, and enables developers to add additional transformations, metadata, or even nodes to the graph.
4. A simple query system to pull data out of the graph.

API

The final API is still being defined/polished. However this is the current one

Plugins List

FileSystem

Listens of any node of type directory and recursively walks the directory, adding all the files that encounters on the way. It produces a node representing the root tree with a series of directory nodes with file leafes.

JSON

Listes for any node with content/type yaml|yml and parses the content using an internal parser. Returns an IParseResult object that looks like this:

{
  data: {},
  pointers: {},
  validations: [],
};

YAML

Listes for any node of with content/type json and parses the content using an internal parser. Returns an IParseResult object that looks like this:

{
  data: {},
  pointers: {},
  validations: [],
};

OpenAPI Specification 2.0

Listens for any PARSED node and will expand the graph with a series of nodes that's the result of the document decomposition, representing the various parts.

Writing a plugin

The Graph has been designed to be expandible by default. Actually the graph itself can't really do anything, most of the features are provided by external plugins.

A plugin is a container of hooks and loaders. Let's explore these concepts:

Hooks

An hook intercepts various parts of the graph's life and gives the possibility to react with actions. Each hook will need to specify a selector function — where you basically ask to be notified only for certain node types. Once such test pass, your "event handlers" will be called for such nodes. (The reason why there are quotes is because they're not really event handlers.)

selector?: (node: INodeInstance) => boolean; // Return true to express your interest in this kind of node
onWillCreateNode?: (node: Partial<Output>) => void; // The node is going to be added in the graph
onDidCreateNode?: (parent: Parent, opts: IGraphHookApi<Input, Output, Parent>) => Promise<any>; // The node has just been added in the graph

The HookApi provides some limited actions you can perform on the graph:

{
  createNode: (node: Input, opts?: { parent?: Parent }) => Promise<Output>;
  createEdge: (source: INodeInstance, destination: INodeInstance) => void;
  getNodeById: (id: string) => INodeInstance | undefined;
}

Loaders

Notes

This is a PoC — we haven't come up with the right API and data format yet.

Contributing

1. Clone repo.
2. Create / checkout feature/{name}, chore/{name}, or fix/{name} branch.
3. Install deps: yarn.
4. Make your changes.
5. Run tests: yarn test.prod.
6. Stage relevant files to git.
7. Commit: yarn commit. NOTE: Commits that don't follow the conventional format will be rejected. yarn commit creates this format for you, or you can put it together manually and then do a regular git commit.
8. Push: git push.
9. Open PR targeting the develop branch.