🚀 Big News: Socket Acquires Coana to Bring Reachability Analysis to Every Appsec Team.Learn more →
Socket
Book a DemoInstallSign in
Socket
Book a DemoInstallSign in

prometheus-api-metrics

Package Overview
Dependencies
Maintainers
3
Versions
35
Alerts
File Explorer

Advanced tools

License
Socket logo

Install Socket

Detect and block malicious and high-risk dependencies

Install

prometheus-api-metrics - npm Package Compare versions

Comparing version

3.1.0
to
3.2.0

.circleci/config.yml

23

CHANGELOG.md

@@ -1,8 +0,25 @@

# Master
# Changelog
# 3.1.0 - 3 September, 2020
## Master
### Features
- Add support for custom labels addition to metrics
### Improvements
- Add support for `prom-client v13`, which includes a few breaking changes, mainly the following functions are now async (return a promise):
```
registry.metrics()
registry.getMetricsAsJSON()
registry.getMetricsAsArray()
registry.getSingleMetricAsString()
```
More info at [`prom-client v13` Release Page](https://github.com/siimon/prom-client/releases/tag/v13.0.0).
## 3.1.0 - 3 September, 2020
- Added support for axios responses while using axios-time plugin
# 3.0.0 - 2 September, 2020
## 3.0.0 - 2 September, 2020

@@ -9,0 +26,0 @@ ### Breaking changes

15

package.json
{
"name": "prometheus-api-metrics",
"version": "3.1.0",
"version": "3.2.0",
"description": "API and process monitoring with Prometheus for Node.js micro-service",

@@ -44,3 +44,3 @@ "author": "Idan Tovi",

"peerDependencies": {
"prom-client": "12.x"
"prom-client": ">=12 <14"
},

@@ -55,3 +55,3 @@ "devDependencies": {

"@types/supertest": "^2.0.10",
"axios": "^0.20.0",
"axios": "^0.21.1",
"axios-time": "^1.0.0",

@@ -80,3 +80,3 @@ "body-parser": "^1.18.3",

"nyc": "^15.1.0",
"prom-client": "^12.0.0",
"prom-client": "^13.1.0",
"reflect-metadata": "^0.1.13",

@@ -86,9 +86,4 @@ "request": "^2.88.0",

"rewire": "^4.0.1",
"rxjs": "^5.5.12",
"rxjs": "^6.6.6",
"sinon": "^5.0.10",
"stryker": "^0.30.1",
"stryker-api": "^0.21.5",
"stryker-javascript-mutator": "^0.14.1",
"stryker-mocha-framework": "^0.12.5",
"stryker-mocha-runner": "^0.14.6",
"supertest": "^3.4.2",

@@ -95,0 +90,0 @@ "ts-node": "^7.0.1",

95

README.md

@@ -41,12 +41,12 @@ # Prometheus API Monitoring

- [Collect API metrics for each call](#usage)
- Response time in seconds
- Request size in bytes
- Response size in bytes
- Add prefix to metrics names - custom or project name
- Exclude specific routes from being collect
- Number of open connections to the server
- Response time in seconds
- Request size in bytes
- Response size in bytes
- Add prefix to metrics names - custom or project name
- Exclude specific routes from being collect
- Number of open connections to the server
- Process Metrics as recommended by Prometheus [itself](https://prometheus.io/docs/instrumenting/writing_clientlibs/#standard-and-runtime-collectors)
- Endpoint to retrieve the metrics - used for Prometheus scraping
- Prometheus format
- JSON format (`${path}.json`)
- Prometheus format
- JSON format (`${path}.json`)
- Support custom metrics

@@ -64,11 +64,15 @@ - [Http function to collect request.js HTTP request duration](#requestjs-http-request-duration-collector)

- metricsPath - Path to access the metrics. `default: /metrics`
- defaultMetricsInterval - the interval to collect the process metrics in milliseconds. `default: 10000`
- durationBuckets - Buckets for response time in seconds. `default: [0.001, 0.005, 0.015, 0.05, 0.1, 0.2, 0.3, 0.4, 0.5]`
- requestSizeBuckets - Buckets for request size in bytes. `default: [5, 10, 25, 50, 100, 250, 500, 1000, 2500, 5000, 10000]`
- responseSizeBuckets - Buckets for response size in bytes. `default: [5, 10, 25, 50, 100, 250, 500, 1000, 2500, 5000, 10000]`
- useUniqueHistogramName - Add to metrics names the project name as a prefix (from package.json)
- metricsPrefix - A custom metrics names prefix, the package will add underscore between your prefix to the metric name.
- excludeRoutes - Array of routes to exclude. Routes should be in your framework syntax.
- includeQueryParams - A boolean that indicate if to include query params in route, the query parameters will be sorted in order to eliminate the number of unique labels.
| Option | Type | Description | Default Value |
|--------------------------|-----------|-------------|---------------|
| `metricsPath` | `String` | Path to access the metrics | `/metrics` |
| `defaultMetricsInterval` | `Number` | Interval to collect the process metrics in milliseconds | `10000` |
| `durationBuckets` | `Array<Number>` | Buckets for response time in seconds | `[0.001, 0.005, 0.015, 0.05, 0.1, 0.2, 0.3, 0.4, 0.5]` |
| `requestSizeBuckets` | `Array<Number>` | Buckets for request size in bytes | `[5, 10, 25, 50, 100, 250, 500, 1000, 2500, 5000, 10000]` |
| `responseSizeBuckets` | `Array<Number>` | Buckets for response size in bytes | `[5, 10, 25, 50, 100, 250, 500, 1000, 2500, 5000, 10000]` |
| `useUniqueHistogramName` | `Boolean` | Add to metrics names the project name as a prefix (from package.json) | `false` |
| `metricsPrefix` | `String` | A custom metrics names prefix, the package will add underscore between your prefix to the metric name | |
| `excludeRoutes` | `Array<String>` | Array of routes to exclude. Routes should be in your framework syntax | |
| `includeQueryParams` | `Boolean` | Indicate if to include query params in route, the query parameters will be sorted in order to eliminate the number of unique labels | `false` |
| `additionalLabels` | `Array<String>` | Indicating custom labels that can be included on each `http_*` metric. Use in conjunction with `extractAdditionalLabelValuesFn`. |
| `extractAdditionalLabelValuesFn` | `Function` | A function that can be use to generate the value of custom labels for each of the `http_*` metrics. When using koa, the function takes `ctx`, when using express, it takes `req, res` as arguments | |

@@ -78,2 +82,3 @@ ### Access the metrics

To get the metrics in Prometheus format use:
```sh

@@ -84,2 +89,3 @@ curl http[s]://<host>:[port]/metrics

To get the metrics in JSON format use:
```sh

@@ -95,9 +101,9 @@ curl http[s]://<host>:[port]/metrics.json

## Custom Metrics
You can expand the API metrics with more metrics that you would like to expose.
All you have to do is:
All you have to do is:
Require prometheus client
```js

@@ -108,2 +114,3 @@ const Prometheus = require('prom-client');

Create new metric from the kind that you like
```js

@@ -118,2 +125,3 @@ const checkoutsTotal = new Prometheus.Counter({

Update it:
```js

@@ -130,12 +138,37 @@ checkoutsTotal.inc({

### Note
This will work only if you use the default Prometheus registry - do not use `new Prometheus.Registry()`
## Additional Metric Labels
You can define additional metric labels by using `additionalLabels` and `extractAdditionalLabelValuesFn` options.
For instance:
```js
const apiMetrics = require('prometheus-api-metrics');
app.use(apiMetrics({
additionalLabels: ['customer', 'cluster'],
extractAdditionalLabelValuesFn: (req, res) => {
const { headers } = req.headers;
return {
customer: headers['x-custom-header-customer'],
cluster: headers['x-custom-header-cluster']
}
}
}))
```
## Request.js HTTP request duration collector
This feature enables you to easily process the result of Request.js timings feature.
### Usage
#### Initialize
You can choose to initialized this functionality as a Class or not
**Class:**
```js

@@ -148,2 +181,3 @@ const HttpMetricsCollector = require('prometheus-api-metrics').HttpMetricsCollector;

**Singelton:**
```js

@@ -155,2 +189,3 @@ const HttpMetricsCollector = require('prometheus-api-metrics').HttpMetricsCollector;

#### Options
- durationBuckets - the histogram buckets for request duration.

@@ -161,6 +196,6 @@ - countClientErrors - Boolean that indicates whether to collect client errors as Counter, this counter will have target and error code labels.

For Example:
#### request
```js

@@ -173,2 +208,3 @@ request({ url: 'http://www.google.com', time: true }, (err, response) => {

#### request-promise-native
```js

@@ -182,8 +218,8 @@ return requestPromise({ method: 'POST', url: 'http://www.mocky.io/v2/5bd9984b2f00006d0006d1fd', route: 'v2/:id', time: true, resolveWithFullResponse: true }).then((response) => {

**Notes:**
**Notes:**
1. In order to use this feature you must use `{ time: true }` as part of your request configuration and then pass to the collector the response or error you got.
2. In order to use the timing feature in request-promise/request-promise-native you must also use `resolveWithFullResponse: true`
3. Override - you can override the `route` and `target` attribute instead of taking them from the request object. In order to do that you should set a `metrics` object on your request with those attribute:
``` js
```js
request({ method: 'POST', url: 'http://www.mocky.io/v2/5bd9984b2f00006d0006d1fd', metrics: { target: 'www.google.com', route: 'v2/:id' }, time: true }, (err, response) => {...};

@@ -194,2 +230,3 @@ });

#### axios
```js

@@ -209,6 +246,8 @@ const axios = require('axios');

**Notes:**
* In order to collect metrics from axios client the [`axios-time`](https://www.npmjs.com/package/axios-time) package is required.
**Notes:**
- In order to collect metrics from axios client the [`axios-time`](https://www.npmjs.com/package/axios-time) package is required.
## Usage in koa
This package supports koa server that uses [`koa-router`](https://www.npmjs.com/package/koa-router) and [`koa-bodyparser`](https://www.npmjs.com/package/koa-bodyparser)

@@ -224,3 +263,3 @@

```
```sh
npm test

@@ -238,2 +277,3 @@ ```

### 95th Response Time by specific route and status code
```

@@ -244,2 +284,3 @@ histogram_quantile(0.95, sum(rate(http_request_duration_seconds_bucket{<SERVICE_LABLE_FIELD>="<SERVICE_LABEL>", route="<ROUTE_NAME>", code="200"}[10m])) by (le))

### Median Response Time Overall
```

@@ -250,2 +291,3 @@ histogram_quantile(0.50, sum(rate(http_request_duration_seconds_bucket{<SERVICE_LABLE_FIELD>="<SERVICE_LABEL>"}[10m])) by (le))

### Median Request Size Overall
```

@@ -256,2 +298,3 @@ histogram_quantile(0.50, sum(rate(http_request_size_bytes_bucket{<SERVICE_LABLE_FIELD>="<SERVICE_LABEL>"}[10m])) by (le))

### Median Response Size Overall
```

@@ -262,2 +305,3 @@ histogram_quantile(0.50, sum(rate(http_response_size_bytes_bucket{<SERVICE_LABLE_FIELD>="<SERVICE_LABEL>"}[10m])) by (le))

### Avarage Memory Usage - All services
```

@@ -268,2 +312,3 @@ avg(nodejs_external_memory_bytes / 1024 / 1024) by (<SERVICE_LABLE_FIELD)

### Avarage Eventloop Latency - All services
```

@@ -270,0 +315,0 @@ avg(nodejs_eventloop_lag_seconds) by (<SERVICE_LABLE_FIELD)

24

src/express-middleware.js

@@ -34,3 +34,3 @@ const Prometheus = require('prom-client');

_handleResponse (req, res) {
_handleResponse(req, res) {
const responseLength = parseInt(res.get('Content-Length')) || 0;

@@ -41,5 +41,11 @@

if (route && utils.shouldLogMetrics(this.setupOptions.excludeRoutes, route)) {
this.setupOptions.requestSizeHistogram.observe({ method: req.method, route: route, code: res.statusCode }, req.metrics.contentLength);
req.metrics.timer({ route: route, code: res.statusCode });
this.setupOptions.responseSizeHistogram.observe({ method: req.method, route: route, code: res.statusCode }, responseLength);
const labels = {
method: req.method,
route,
code: res.statusCode,
...this.setupOptions.extractAdditionalLabelValuesFn(req, res)
};
this.setupOptions.requestSizeHistogram.observe(labels, req.metrics.contentLength);
req.metrics.timer(labels);
this.setupOptions.responseSizeHistogram.observe(labels, responseLength);
debug(`metrics updated, request length: ${req.metrics.contentLength}, response length: ${responseLength}`);

@@ -90,3 +96,3 @@ }

middleware(req, res, next) {
async middleware(req, res, next) {
if (!this.setupOptions.server && req.socket) {

@@ -102,13 +108,11 @@ this.setupOptions.server = req.socket.server;

res.set('Content-Type', Prometheus.register.contentType);
return res.end(Prometheus.register.metrics());
return res.end(await Prometheus.register.metrics());
}
if (routeUrl === `${this.setupOptions.metricsRoute}.json`) {
debug('Request to /metrics endpoint');
return res.json(Prometheus.register.getMetricsAsJSON());
return res.json(await Prometheus.register.getMetricsAsJSON());
}
req.metrics = {
timer: this.setupOptions.responseTimeHistogram.startTimer({
method: req.method
}),
timer: this.setupOptions.responseTimeHistogram.startTimer(),
contentLength: parseInt(req.get('content-length')) || 0

@@ -115,0 +119,0 @@ };

8

src/index.d.ts

@@ -1,3 +0,3 @@

import { RequestHandler, Response } from 'express';
import { Middleware } from 'koa';
import { Request, RequestHandler, Response } from 'express';
import { Context, Middleware } from 'koa';

@@ -23,2 +23,4 @@ export default function middleware(options?: ApiMetricsOpts) : RequestHandler;

includeQueryParams?: boolean;
additionalLabels?: string[];
extractAdditionalLabelValuesFn?: ((req: Request, res: Response) => Record<string, unknown>) | ((ctx: Context) => Record<string, unknown>)
}

@@ -31,2 +33,2 @@

prefix?: string;
}
}

28

src/koa-middleware.js

@@ -42,13 +42,11 @@ const Prometheus = require('prom-client');

if (route && utils.shouldLogMetrics(this.setupOptions.excludeRoutes, route)) {
this.setupOptions.requestSizeHistogram.observe({
const labels = {
method: ctx.req.method,
route: route,
code: ctx.res.statusCode
}, ctx.req.metrics.contentLength);
ctx.req.metrics.timer({ route: route, code: ctx.res.statusCode });
this.setupOptions.responseSizeHistogram.observe({
method: ctx.req.method,
route: route,
code: ctx.res.statusCode
}, responseLength);
route,
code: ctx.res.statusCode,
...this.setupOptions.extractAdditionalLabelValuesFn(ctx)
};
this.setupOptions.requestSizeHistogram.observe(labels, ctx.req.metrics.contentLength);
ctx.req.metrics.timer(labels);
this.setupOptions.responseSizeHistogram.observe(labels, responseLength);
debug(`metrics updated, request length: ${ctx.req.metrics.contentLength}, response length: ${responseLength}`);

@@ -108,3 +106,3 @@ }

middleware(ctx, next) {
async middleware(ctx, next) {
if (!this.setupOptions.server && ctx.req.socket) {

@@ -117,3 +115,3 @@ this.setupOptions.server = ctx.req.socket.server;

ctx.set('Content-Type', Prometheus.register.contentType);
ctx.body = Prometheus.register.metrics();
ctx.body = await Prometheus.register.metrics();
return next();

@@ -123,3 +121,3 @@ }

debug('Request to /metrics endpoint');
ctx.body = Prometheus.register.getMetricsAsJSON();
ctx.body = await Prometheus.register.getMetricsAsJSON();
return next();

@@ -129,5 +127,3 @@ }

ctx.req.metrics = {
timer: this.setupOptions.responseTimeHistogram.startTimer({
method: ctx.req.method
}),
timer: this.setupOptions.responseTimeHistogram.startTimer(),
contentLength: parseInt(ctx.request.get('content-length')) || 0

@@ -134,0 +130,0 @@ };

90

src/metrics-middleware.js

@@ -13,18 +13,61 @@ 'use strict';

return (options = {}) => {
const { metricsPath, defaultMetricsInterval = 10000, durationBuckets, requestSizeBuckets, responseSizeBuckets, useUniqueHistogramName, metricsPrefix, excludeRoutes, includeQueryParams } = options;
const {
metricsPath,
defaultMetricsInterval = 10000,
durationBuckets,
requestSizeBuckets,
responseSizeBuckets,
useUniqueHistogramName,
metricsPrefix,
excludeRoutes,
includeQueryParams,
additionalLabels = [],
extractAdditionalLabelValuesFn
} = options;
debug(`Init metrics middleware with options: ${JSON.stringify(options)}`);
setupOptions.metricsRoute = metricsPath || '/metrics';
setupOptions.excludeRoutes = excludeRoutes || [];
setupOptions.metricsRoute = utils.validateInput({
input: metricsPath,
isValidInputFn: utils.isString,
defaultValue: '/metrics',
errorMessage: 'metricsPath should be an string'
});
setupOptions.excludeRoutes = utils.validateInput({
input: excludeRoutes,
isValidInputFn: utils.isArray,
defaultValue: [],
errorMessage: 'excludeRoutes should be an array'
});
setupOptions.includeQueryParams = includeQueryParams;
setupOptions.defaultMetricsInterval = defaultMetricsInterval;
let metricNames = {
http_request_duration_seconds: 'http_request_duration_seconds',
app_version: 'app_version',
http_request_size_bytes: 'http_request_size_bytes',
http_response_size_bytes: 'http_response_size_bytes',
defaultMetricsPrefix: ''
};
metricNames = utils.getMetricNames(metricNames, useUniqueHistogramName, metricsPrefix, projectName);
setupOptions.additionalLabels = utils.validateInput({
input: additionalLabels,
isValidInputFn: utils.isArray,
defaultValue: [],
errorMessage: 'additionalLabels should be an array'
});
setupOptions.extractAdditionalLabelValuesFn = utils.validateInput({
input: extractAdditionalLabelValuesFn,
isValidInputFn: utils.isFunction,
defaultValue: () => ({}),
errorMessage: 'extractAdditionalLabelValuesFn should be a function'
});
const metricNames = utils.getMetricNames(
{
http_request_duration_seconds: 'http_request_duration_seconds',
app_version: 'app_version',
http_request_size_bytes: 'http_request_size_bytes',
http_response_size_bytes: 'http_response_size_bytes',
defaultMetricsPrefix: ''
},
useUniqueHistogramName,
metricsPrefix,
projectName
);
Prometheus.collectDefaultMetrics({ timeout: defaultMetricsInterval, prefix: `${metricNames.defaultMetricsPrefix}` });

@@ -34,8 +77,19 @@

const metricLabels = [
'method',
'route',
'code',
...additionalLabels
].filter(Boolean);
// Buckets for response time from 1ms to 500ms
const defaultDurationSecondsBuckets = [0.001, 0.005, 0.015, 0.05, 0.1, 0.2, 0.3, 0.4, 0.5];
// Buckets for request size from 5 bytes to 10000 bytes
const defaultSizeBytesBuckets = [5, 10, 25, 50, 100, 250, 500, 1000, 2500, 5000, 10000];
setupOptions.responseTimeHistogram = Prometheus.register.getSingleMetric(metricNames.http_request_duration_seconds) || new Prometheus.Histogram({
name: metricNames.http_request_duration_seconds,
help: 'Duration of HTTP requests in seconds',
labelNames: ['method', 'route', 'code'],
// buckets for response time from 1ms to 500ms
buckets: durationBuckets || [0.001, 0.005, 0.015, 0.05, 0.1, 0.2, 0.3, 0.4, 0.5]
labelNames: metricLabels,
buckets: durationBuckets || defaultDurationSecondsBuckets
});

@@ -46,4 +100,4 @@

help: 'Size of HTTP requests in bytes',
labelNames: ['method', 'route', 'code'],
buckets: requestSizeBuckets || [5, 10, 25, 50, 100, 250, 500, 1000, 2500, 5000, 10000] // buckets for request size from 5 bytes to 10000 bytes
labelNames: metricLabels,
buckets: requestSizeBuckets || defaultSizeBytesBuckets
});

@@ -54,4 +108,4 @@

help: 'Size of HTTP response in bytes',
labelNames: ['method', 'route', 'code'],
buckets: responseSizeBuckets || [5, 10, 25, 50, 100, 250, 500, 1000, 2500, 5000, 10000] // buckets for response size from 5 bytes to 10000 bytes
labelNames: metricLabels,
buckets: responseSizeBuckets || defaultSizeBytesBuckets
});

@@ -58,0 +112,0 @@

32

src/utils.js
'use strict';
function getMetricNames(metricNames, useUniqueHistogramName, metricsPrefix, projectName) {
const getMetricNames = (metricNames, useUniqueHistogramName, metricsPrefix, projectName) => {
const prefix = useUniqueHistogramName === true ? projectName : metricsPrefix;

@@ -13,11 +13,29 @@

return metricNames;
}
};
function shouldLogMetrics(excludeRoutes, route) {
return excludeRoutes.every((path) => {
return !route.includes(path);
});
}
const isArray = (input) => Array.isArray(input);
const isFunction = (input) => typeof input === 'function';
const isString = (input) => typeof input === 'string';
const shouldLogMetrics = (excludeRoutes, route) => excludeRoutes.every((path) => !route.includes(path));
const validateInput = ({ input, isValidInputFn, defaultValue, errorMessage }) => {
if (typeof input !== 'undefined') {
if (isValidInputFn(input)) {
return input;
} else {
throw new Error(errorMessage);
}
}
return defaultValue;
};
module.exports.getMetricNames = getMetricNames;
module.exports.isArray = isArray;
module.exports.isFunction = isFunction;
module.exports.isString = isString;
module.exports.shouldLogMetrics = shouldLogMetrics;
module.exports.validateInput = validateInput;