@flytio/int-sdk

INT SDK

Use version at least 1.1.0 -> Read here why

A standard library for integrations in Flyt. The library aims to help to speed up future integrations by extracting common parts. This way in case of any update you don't have to update the code, but only the version of the library. See original proposal.

How does it work?

The library sets up some things for you.

1. HTTP server

It runs an express server on port 8080 to expose metrics and liveness probe.

/metrics
/healthz

As in version 1.2.x it is now possible to extend the http server with endpoints that might be required by your integration. Look at this section of the documentation for instructions.

2. GRPC server

It creates a GRPC server and assigns your implementation to its services.

3. Prometheus

It registers Prometheus metrics and places some counters and histograms (using prom-client package) into the integration calls.

4. Axios

It exports HttpClientFactory that sets interceptors with a logger, and metrics and configures default timeout.

5. Handy wrappers

It also exports wrappers for Menu and Ordering services that simplify passing logger and httpInsntace down to the integration so you don't have to think about extracting requestId from metadata.

Example

You have to create a LoggerFactory instance that is shared between the integration factory and your implementation. This way you can keep the logs under the same app name. Also you may want to create an httpClient instance using HttpClientFactory insntace to make use of metrics and logs already set up using interceptors.

import {
  IntegrationFactory,
  LoggerFactory,
  HttpClientFactory,
  OrderingServiceWrapper,
  OrderingService,
} from '@flytio/int-sdk';
import { MyOrderingImplementation, APP_NAME } from './src/my-class';

// 1. create a logger factory instance
const loggerFactory = new LoggerFactory(APP_NAME);
// 2. use logger factory instance to create http client factory instance
const httpClientFactory = new HttpClientFactory(loggerFactory);
// 3. MyOrderingImplementation implements
// OrderingService exported from @flytio/int-sdk
const myOrdering = new MyOrderingImplementation();

// 4. OrderingServiceWrapper implements
// OrderingGrpcService exported from @flytio/protos
const ordering = new OrderingServiceWrapper(
  myOrdering,
  loggerFactory,
  httpClientFactory,
);

// 5. create an integration
const integration = new IntegrationFactory({
  loggerFactory,
  appName: APP_NAME,
  ordering,
});

// 6. and make grpc server and http server listen
integration.run();

Development

Code style

We use prettier to format code before commit (using husky hooks).

Our config.

Testing

For testing we use Jest. Unit tests are placed inside __tests__ directory. Integration tests are placed in __integration-tests__ directory.

Deployment

The library is distributed as an npm package and publishing happens automatically through CircleCI after merging to master.

Versioning

We follow semantic versioning approach. To ensure you bump the version number there is a script to run before you push.

Please update CHANGELOG.md before publishing new version.

LoggerFactory

It's a factory class that you create once giving the app name and you pass around the instance to create logger instances.

Besides appName it also accepts config where you can toggle transport types for example when you want to use Console during development.

Example of configuring LoggerFactory to use Console transport type

import { LoggerFactory, TransportType } from '@flytio/int-sdk'; 
import { APP_NAME } from "./config";

const loggerFactory = new LoggerFactory(APP_NAME, {
  useUDP: false,
  useConsole: true
});

HttpClientFactory

It's a factory class that you create once giving the loggerFactory instance and you pass around the instance of it to create http client (internally using Axios) instance.

Under the hood it sets up interceptors that log external requests and responses and also metrics.

So whenever you use an http instance it will automatically log the request and response for you

const httpClientFactory = new HttpClientFactory(loggerFactory);
const httpClient = httpClientFactory.getHttpClient(requestId);

const result = await httpClient.get(url);

in logs you're going to see something like

HTTP request: { url: 'http://something.com', headers: {... some headers } }
HTTP response { url: 'http://something.com', data: { ... some payload }}

Use a proxy

Since version 1.3.4, we added the ability to specify a default proxy that will be attached to all the requests (or alternatively to just some of them).

ProxyAgentFactory is included which can be used when an integration requires the ability to configure HTTP requests to pass through a proxy. It accepts four constructor arguments, url, port, user and pass. A new httpsProxyAgent is instantiated in the constructor on the private httpsProxyAgent property.

Create the proxy and pass it to the HttpClientFactory

By doing this, every request that you'll make through the httpClient will be forwarder through the proxy.

In order to do this, create a ProxyAgentFactory and pass it to HttpClientFactory inside the run-server.ts file:

const proxyAgentFactory = new ProxyAgentFactory(proxyUrl, proxyPort, proxyUser, proxyPass);
const httpClientFactory = new HttpClientFactory(loggerFactory, proxyAgentFactory);

const ordering = new OrderingServiceWrapper(
	new OrderingIntegration(),
	loggerFactory,
	httpClientFactory,
);

Don't forget to edit your certification test to manually go through the proxy too:

const getProxyAgent = (): httpsProxyAgent => {
	return new httpsProxyAgent(`http://${proxyUser}:${proxyPass}@${proxyUrl}:${proxyPort}`);
};

beforeAll(() => {
	const axiosConfig: AxiosRequestConfig = {};
	if (process.env.NODE_ENV === 'production' || process.env.NOCK_OFF === 'true') {
		axiosConfig.httpsAgent = getProxyAgent();
	}
	httpClient = axios.create(axiosConfig);
});

Use the ProxyAgentFactory directly if you need to attach the proxy to a specific request

If you need that only a specific endpoint call goes through the proxy, you can use the ProxyAgentFactory directly. This factory will contain a method to get the proxy agent that you can apply as httpAgent or httspAgent to the axios configuration that will be passed to the request directly:

import { ProxyAgentFactory } from '@flytio/int-sdk';

const proxyAgentFactory = new ProxyAgentFactory(
  url,
  port,
  user,
  pass
);

const config = {
  httpsAgent: proxyAgentFactory.getHttpsProxyAgent(),
};

axios.post(url, data, config);

Extending the HTTP Server

With the int-sdk version 1.2 is now possible to extend the http server to run also custom routes needed by integrations.

Please note that this won't publicly expose the route.

You'll also require to: and an ingress host in helm/production/values.yaml

• Add a kong migration (contact the devops team)
• Edit the file helm/production/values.yaml to include an ingress.host

Currently the only integration that supports this is redcat-ordering, so have a look at it for examples

The extra routes must be defined using the express.Router component, as defined in the express documentation (search for express.Router in the page) and then passed to the IntegrationFactory when instantiating the class.

An example:

In your integration just add the following code to register the express.Router component and define a route to handle GET requests made to the /test endpoint:

const router = express.Router();
router.get('/test', (req, res) => {
    res.status(201);
    res.json({ foo: 'bar' });
});

And then simply pass the router instance when creating the IntegrationFactory, before running it:

integration = new IntegrationFactory({
    ordering: orderingIntegrationInstance,
    loggerFactory,
    appName: 'test-http-server',
    router: router,
});
integration.run();

NOTE: The routes that you'll create in you integration are currently supporting only json body. This is because the json body-parser attached to the express app on its creation, as documented on ADR-05.