@elastic.io/maester-client - npm Package Compare versions

Comparing version 4.0.0-dev.6 to 4.0.0-dev.7

CHANGELOG.md

@@ -1,3 +0,3 @@
-# 4.0.0 (June 17, 2022)
-* new version of library 
+# 4.0.0 (June 28, 2022)
+* New version of library with braking changes, look for updates in [README](/README.md) 
 
@@ -4,0 +4,0 @@ # 3.4.3 (April 8, 2022)

package.json

 {
   "name": "@elastic.io/maester-client",
-  "version": "4.0.0-dev.6",
+  "version": "4.0.0-dev.7",
   "description": "The official object-storage client",
-  "main": "src/index.js",
-  "types": "src/index.d.ts",
+  "main": "dist/src/index.js",
+  "types": "dist/src/index.d.ts",
   "scripts": {
     "lint": "eslint '*/**/*[^.d$].ts' --quiet --fix",
-    "test": "NODE_ENV=test mocha -r ts-node/register --recursive spec/**/*.ts --timeout 10000",
+    "test": "mocha -r ts-node/register --recursive spec/**/*.ts --timeout 12000",
     "integration-test": "npm run pretest && mocha --exit --r ts-node/register spec-integration/**/*.ts --timeout 1000000",
-    "pretest": "eslint --ext .ts --quiet --fix && find src spec spec-integration -name \"*.js\" -type f -delete && find src spec spec-integration -name \"*.d.ts\" -type f -delete",
+    "pretest": "rm -rf dist && eslint --ext .ts --quiet --fix && find src spec spec-integration -name \"*.js\" -type f -delete && find src spec spec-integration -name \"*.d.ts\" -type f -delete",
     "posttest": "tsc",
     "build": "tsc",
-    "prepare": "npm run build",
+    "prepare": "rm -rf dist && npm run build",
     "prepublishOnly": "npm run lint && npm run test",
@@ -68,5 +68,5 @@ "license-check": "license-checker --excludePrivatePackages --excludePackages \"$(node ../../../.ignored-licenses.js)\" --onlyAllow \"$(node ../../../.allowed-licenses.js)\""
     "jsonwebtoken": "8.5.1",
-    "sinon": "10.0.0",
+    "sinon": "11.1.2",
     "stream-mime-type": "1.0.2"
   }
-}
+}

README.md

@@ -7,2 +7,6 @@ # Maester Client
 
+This library propose you two clients: `ObjectStorage` & `ObjectStorageWrapper`. 
+Use `ObjectStorageWrapper` to operate with data like `string`, `number`, `object`, `array`
+Use `ObjectStorage` to operate attachments (also could be used for same purpose as `ObjectStorageWrapper`)
+
 Note: All the code snippets written in Typescript
@@ -12,8 +16,9 @@
 ```
-import { ObjectStorageWrapper } from '@elastic.io/maester-client/dist/ObjectStorageWrapper';
+import { ObjectStorage, ObjectStorageWrapper } from '@elastic.io/maester-client';
 
-const objectStorage = new ObjectStorageWrapper(this);
+const objectStorageWrapper = new ObjectStorageWrapper(this);
+const objectStorage = new ObjectStorage(creds);
 ```
 
-### CRUD operations
+## ObjectStorageWrapper CRUD operations
 
@@ -27,3 +32,3 @@ ### Create object
 where
-- data - object data to create. *Required*
+- data - any data to create. *Required*
 - queryHeaders - array of objects `{ key: string, value: string }`, current maximum - 5 items. Where `key` (must be lowercase) - searchable field name (see below in `Get objects by query parameters` section), must be unique for whole array, if specified - `value` must be specified as well; `value` - searchable field value, if specified - `key` must be specified as well. *Optional*
@@ -34,5 +39,5 @@ - metaHeaders - array of objects `{ key: string, value: string }`, where `key` (must be lowercase) - meta field name, must be unique for whole array, if specified - `value` must be specified as well; `value` - meta field value, if specified - `key` must be specified as well. *Optional*
 ```
-const obj = await objectStorage.createObject(data);
-const obj = await objectStorage.createObject(data, [], [], 100000);
-const obj = await objectStorage.createObject(
+const obj = await objectStorageWrapper.createObject(data);
+const obj = await objectStorageWrapper.createObject(data, [], [], 100000);
+const obj = await objectStorageWrapper.createObject(
   data,
@@ -43,3 +48,3 @@ [{key: 'somequeriablefieldkey', value: 'somequeriablefieldvalue'}],
 );
-const obj = await objectStorage.createObject(
+const obj = await objectStorageWrapper.createObject(
   data,
@@ -53,2 +58,3 @@ [{key: 'anotherqueriablefieldkey', value: 'anotherqueriablefieldvalue'}, {key: 'anotherqueriablefieldkey2', value: 'anotherqueriablefieldvalue2'}],
 ### Read operations
+
 #### Get object by ID
@@ -61,8 +67,8 @@
 where
-- id - Maester internal id of the object to update. E.g. '76380cae-aee3-457a-9029-d971f61e3731'. *Required*
+- id - Maester internal id of the object to lookup. E.g. '76380cae-aee3-457a-9029-d971f61e3731'. *Required*
 - responseType - One of response-types [`json`, `stream`, `arraybuffer`]. Data will be returned in an appropriate format. Defaults to `json`. *Optional*
 
 ```
-const obj = await objectStorage.lookupObjectById(id);
-const obj = await objectStorage.lookupObjectById(id, 'stream');
+const obj = await objectStorageWrapper.lookupObjectById(id);
+const obj = await objectStorageWrapper.lookupObjectById(id, 'stream');
 ```
@@ -73,2 +79,3 @@ By default method returns **a raw string**, you may want to parse JSON or do any other data processing according to object's expected data type:
 ```
+
 #### Get objects by query parameters
@@ -96,3 +103,3 @@
 ```
-const obj = await objectStorage.lookupObjectsByQueryParameters([
+const obj = await objectStorageWrapper.lookupObjectsByQueryParameters([
   { key: 'somequeriablefieldkey', value: 'somequeriablefieldvalue' },
@@ -117,4 +124,4 @@ { key: 'anotherqueriablefieldkey', value: 'anotherqueriablefieldvalue' }
 ```
-const obj = await objectStorage.updateObject(id, data);
-const obj = await objectStorage.updateObject(
+const obj = await objectStorageWrapper.updateObject(id, data);
+const obj = await objectStorageWrapper.updateObject(
   id,
@@ -124,3 +131,3 @@ data,
 );
-const obj = await objectStorage.updateObject(
+const obj = await objectStorageWrapper.updateObject(
   id,
@@ -135,2 +142,3 @@ data,
 ### Delete operations
+
 #### Delete object by ID
@@ -143,6 +151,6 @@
 where
-- id - Maester internal id of the object to update. E.g. '76380cae-aee3-457a-9029-d971f61e3731'. *Required*
+- id - Maester internal id of the object to delete. E.g. '76380cae-aee3-457a-9029-d971f61e3731'. *Required*
 
 ```
-const obj = await objectStorage.deleteObjectById(id);
+const obj = await objectStorageWrapper.deleteObjectById(id);
 ```
@@ -158,3 +166,132 @@
 ```
-const obj = await objectStorage.deleteObjectsByQueryParameters([{key: 'somequeriablefieldkey', value: 'somequeriablefieldvalue'}]);
-```
+const obj = await objectStorageWrapper.deleteObjectsByQueryParameters([{key: 'somequeriablefieldkey', value: 'somequeriablefieldvalue'}]);
+```
+
+## ObjectStorage CRUD operations
+
+### Create Object
+
+The method has the following signature:
+```
+async add(dataOrFunc: uploadData | (() => Promise<Readable>), reqWithBodyOptions?: ReqWithBodyOptions)
+```
+where
+- dataOrFunc - async function returning stream OR any data (except 'undefined').
+- [reqWithBodyOptions](/src/interfaces.ts#L27).
+
+```
+const obj = await objectStorage.add(data, { override: {'x-query-somequeriablefieldkey': 'somequeriablefieldvalue'} });
+const getAttachAsStream = async () => (await axios.get('https://img.jpg', { responseType: 'stream' })).data;
+const obj = await objectStorage.add(getAttachAsStream, { retryOptions: { retriesCount: 5, retryDelay: 10000, requestTimeout: 60000 } });
+);
+```
+
+### Read operations
+
+#### Get object by ID
+
+The method has the following signature:
+```
+async getOne(objectId: string, reqOptions: ReqOptions = {})
+```
+where
+- id - Maester internal id of the object to lookup. E.g. '76380cae-aee3-457a-9029-d971f61e3731'.
+- [reqOptions](/src/interfaces.ts#L22).
+
+```
+const obj = await objectStorage.getOne(id);
+const obj = await objectStorage.getOne(id, { responseTYpe: 'stream'});
+```
+By default method returns **a raw string**, you may want to parse JSON or do any other data processing according to object's expected data type:
+```
+const parsedObject = JSON.parse(obj);
+```
+
+#### Get objects by query parameters
+
+The method has the following signature:
+```
+async getAllByParams(params: object, reqOptions: ReqOptions = {})
+```
+where
+- params - object of query params, current maximum - 5 items. 
+- [reqOptions](/src/interfaces.ts#L22).
+
+
+Examples:
+```
+const obj = await objectStorage.getAllByParams({ 'query[field]': 'value' });
+```
+The method returns an array of items. It either is empty in case no objects found or contains objects
+
+### Update Object
+
+The method has the following signature:
+```
+async update(objectId: string, dataOrFunc: uploadData | (() => Promise<Readable>), reqWithBodyOptions?: ReqWithBodyOptions)
+```
+where
+- objectId - id of the object to update.
+- dataOrFunc - async function returning stream OR any data (except 'undefined').
+- [reqWithBodyOptions](/src/interfaces.ts#L27)
+
+```
+const obj = await objectStorage.update(data, { override: {'x-query-somequeriablefieldkey': 'somequeriablefieldvalue'} });
+const getAttachAsStream = async () => (await axios.get('https://img.jpg', { responseType: 'stream' })).data;
+const obj = await objectStorage.update(getAttachAsStream);
+);
+```
+
+### Delete operations
+
+#### Delete object by ID
+
+The method has the following signature:
+```
+async deleteOne(objectId: string, reqOptions: ReqOptions = {})
+```
+where
+- id - Maester internal id of the object to delete. E.g. '76380cae-aee3-457a-9029-d971f61e3731'.
+- [reqOptions](/src/interfaces.ts#L22).
+
+```
+const obj = await objectStorage.deleteOne(id);
+const obj = await objectStorage.deleteOne(id, { retryOptions: { retriesCount: 5, retryDelay: 10000, requestTimeout: 60000 } });
+```
+
+#### Delete objects by query parameters
+The method has the following signature:
+```
+async deleteAllByParams(params: object, reqOptions: ReqOptions = {})
+```
+where
+- params - object of query params, current maximum - 5 items. 
+- [reqOptions](/src/interfaces.ts#L22).
+```
+const obj = await objectStorage.deleteAllByParams({ 'query[field]': 'value' });
+```
+
+### Additional methods
+
+#### Use method
+The method has the following signature:
+```
+async use(forward: TransformMiddleware, reverse: TransformMiddleware)
+```
+where
+- forward - transform middleware to use with `Create` and `Update` object operations (`add`, `update` methods).
+- reverse - transform middleware to use with `Get` object operations (`getOne`, `getAllByParams` methods).
+
+#### getHeaders method
+The method has the following signature:
+```
+async getHeaders(objectId: string, reqOptions: ReqOptions = {})
+```
+where
+- forward - transform middleware to use with `Create` and `Update` object operations (`add`, `update` methods).
+- reverse - transform middleware to use with `Get` object operations (`getOne`, `getAllByParams` methods).
+
+## Limitations
+
+1. Both `ObjectStorage` and `ObjectStorageWrapper` could'n process `undefined` as values for upsert value. Also value `undefined` in array will be converted to `null`, e.g
+`[1, 'str', undefined, null, { d: 2 }]` will be saved as `[1, 'str', null, null, { d: 2 }]`

spec-integration/common.ts

 /* eslint-disable import/first */
-process.env.LOG_LEVEL = 'TRACE';
+process.env.LOG_LEVEL = 'INFO';
 process.env.LOG_OUTPUT_MODE = 'short';
+process.env.NODE_ENV = 'test';
 import getLogger from '@elastic.io/component-logger';
@@ -5,0 +6,0 @@ import sinon from 'sinon';

spec-integration/ObjectStorage.spec.ts

 import chai, { expect } from 'chai';
 import axios from 'axios';
+import sinon from 'sinon';
 import fs from 'fs';
+import { creds } from './common';
+import * as utils from '../src/utils';
+import logging from '../src/logger';
 import { ObjectStorage } from '../src';
-import { creds } from './common';
-import { streamFromData } from '../src/utils';
 
@@ -34,3 +36,3 @@ chai.use(require('chai-as-promised'));
       it('should add (json)', async () => {
-        const getJSONAsStream = async () => streamFromData({ a: 4 });
+        const getJSONAsStream = async () => utils.streamFromData({ a: 4 });
         const objectId = await objectStorage.add(getJSONAsStream);
@@ -93,3 +95,3 @@ expect(typeof objectId).to.be.equal('string');
     it('should get (default responseType: json)', async () => {
-      const getJSONAsStream = async () => streamFromData({ a: 4 });
+      const getJSONAsStream = async () => utils.streamFromData({ a: 4 });
       const objectId = await objectStorage.add(getJSONAsStream);
@@ -108,3 +110,3 @@ const object = await objectStorage.getOne(objectId);
     it('should update (addAsJSON, update as stream)', async () => {
-      const dataAsStream = async () => streamFromData({ a: 2 });
+      const dataAsStream = async () => utils.streamFromData({ a: 2 });
       const objId = await objectStorage.add({ a: 3 });
@@ -124,4 +126,4 @@ const resUpdate = await objectStorage.update(objId, dataAsStream);
     it('should update (addAsStream, update as stream)', async () => {
-      const dataAsStream = async () => streamFromData({ a: 4 });
-      const dataAsStream2 = async () => streamFromData({ a: 2 });
+      const dataAsStream = async () => utils.streamFromData({ a: 4 });
+      const dataAsStream2 = async () => utils.streamFromData({ a: 2 });
       const objId = await objectStorage.add(dataAsStream);
@@ -133,3 +135,3 @@ await objectStorage.update(objId, dataAsStream2);
     it('should update (addAsStream, update as json)', async () => {
-      const dataAsStream = async () => streamFromData({ a: 4 });
+      const dataAsStream = async () => utils.streamFromData({ a: 4 });
       const objId = await objectStorage.add(dataAsStream);
@@ -152,3 +154,3 @@ await objectStorage.update(objId, { a: 2 });
     it('should deleteOne', async () => {
-      const getAttachAsStream = async () => streamFromData({ a: 4 });
+      const getAttachAsStream = async () => utils.streamFromData({ a: 4 });
       const objectId = await objectStorage.add(getAttachAsStream);
@@ -176,3 +178,3 @@ const deletedObject = await objectStorage.deleteOne(objectId);
     it('should getByParams', async () => {
-      const jsonAsStream = async () => streamFromData({ a: 4 });
+      const jsonAsStream = async () => utils.streamFromData({ a: 4 });
       const objId1 = await objectStorage.add(jsonAsStream, { override: { 'x-query-x': '123' } });
@@ -188,2 +190,40 @@ const objId2 = await objectStorage.add(jsonAsStream, { override: { 'x-query-x': '123' } });
   });
+  describe('errors handling', () => {
+    let loggingTraceSpy;
+    let loggingWarnSpy;
+    beforeEach(() => {
+      loggingTraceSpy = sinon.spy(logging, 'trace');
+      loggingWarnSpy = sinon.spy(logging, 'warn');
+    });
+    afterEach(sinon.restore);
+    describe('response with error (4xx)', () => {
+      it('should throw 400, no retries', async () => {
+        await expect(objectStorage.getOne('not-a-uuid')).to.be.rejectedWith('Request failed with status code 400');
+        expect(loggingTraceSpy.callCount).to.be.equal(1);
+      });
+      it('should throw 404, no retries', async () => {
+        await expect(objectStorage.getOne('2e084a24-e2ea-47c6-a95a-732ec8df7263')).to.be.rejectedWith('Request failed with status code 404');
+        expect(loggingTraceSpy.callCount).to.be.equal(1);
+      });
+    });
+    describe('Server error (5xx)', () => {
+      beforeEach(() => {
+        sinon.stub(utils, 'validateRetryOptions').callsFake(() => ({ retryDelay: 1, retriesCount: 2, requestTimeout: 1 }));
+      });
+      it('should throw 5xx', async () => {
+        await expect(objectStorage.getOne('some-id')).to.be.rejectedWith('Server error during request: "timeout of 1ms exceeded"');
+        expect(loggingTraceSpy.callCount).to.be.equal(3);
+        expect(loggingWarnSpy.callCount).to.be.equal(2);
+        const [{ err: err1 }, log1] = loggingWarnSpy.getCall(0).args;
+        expect(err1.toJSON().message).to.be.equal('timeout of 1ms exceeded');
+        expect(log1).to.be.equal('Error during object request, retrying (1)');
+        const [{ err: err2 }, log2] = loggingWarnSpy.getCall(1).args;
+        expect(err2.toJSON().message).to.be.equal('timeout of 1ms exceeded');
+        expect(log2).to.be.equal('Error during object request, retrying (2)');
+      });
+      xit('RUN THIS TEST WITHOUT PORT-FORWARDING', async () => {
+        await expect(objectStorage.getOne('some-id')).to.be.rejectedWith('Server error during request: "connect ECONNREFUSED 127.0.0.1:3002"');
+      });
+    });
+  });
 });

spec-integration/ObjectStorageWrapper.spec.ts

@@ -30,2 +30,26 @@ import chai, { expect } from 'chai';
   });
+  it('should createObject (number)', async () => {
+    const objectId = await objectStorageWrapper.createObject(8);
+    expect(isUUID(objectId)).to.be.equal(true);
+    const object = await objectStorageWrapper.lookupObjectById(objectId);
+    expect(JSON.parse(object)).to.be.deep.equal(8);
+    const headers = await objectStorageWrapper.getObjectHeaders(objectId);
+    expect(headers['content-type']).to.be.equal('application/json');
+  });
+  it('should createObject (array)', async () => {
+    const objectId = await objectStorageWrapper.createObject([1, 'dva', null]);
+    expect(isUUID(objectId)).to.be.equal(true);
+    const object = await objectStorageWrapper.lookupObjectById(objectId);
+    expect(JSON.parse(object)).to.be.deep.equal([1, 'dva', null]);
+    const headers = await objectStorageWrapper.getObjectHeaders(objectId);
+    expect(headers['content-type']).to.be.equal('application/json');
+  });
+  it('should createObject (string)', async () => {
+    const objectId = await objectStorageWrapper.createObject('[1, dva, null]');
+    expect(isUUID(objectId)).to.be.equal(true);
+    const object = await objectStorageWrapper.lookupObjectById(objectId);
+    expect(JSON.parse(object)).to.be.deep.equal('[1, dva, null]');
+    const headers = await objectStorageWrapper.getObjectHeaders(objectId);
+    expect(headers['content-type']).to.be.equal('application/json');
+  });
   describe('lookupObjectById', () => {
@@ -80,2 +104,10 @@ it('should lookupObjectById', async () => {
     });
+    it('should updateObjectById (string)', async () => {
+      const objectId = await objectStorageWrapper.createObject({ a: 2 });
+      expect(isUUID(objectId)).to.be.equal(true);
+      const updated = await objectStorageWrapper.updateObjectById(objectId, 'hey');
+      expect(updated.objectId).to.be.equal(objectId);
+      const object = await objectStorageWrapper.lookupObjectById(objectId);
+      expect(JSON.parse(object)).to.be.deep.equal('hey');
+    });
     it('should throw 404', async () => {
@@ -82,0 +114,0 @@ await expect(objectStorageWrapper.updateObjectById('fa208d86-6b81-408e-87f3-4b6e90be7db9', {})).to.be.rejectedWith('Request failed with status code 404');

spec/helpers.ts

 /* eslint-disable import/first */
-process.env.REQUEST_MAX_RETRY = '3';
 process.env.REQUEST_RETRY_DELAY = '0';
+process.env.NODE_ENV = 'test';
 import { Readable, Duplex } from 'stream';
@@ -5,0 +5,0 @@ import * as crypto from 'crypto';

New alerts

License Policy Violation

License

This package is not allowed per your license policy. Review the package's license to ensure compliance.

Found 1 instance in 1 package

Major refactor

Supply chain risk

Package has recently undergone a major refactor. It may be unstable or indicate significant internal changes. Use caution when updating to versions that include significant changes.

Found 1 instance in 1 package

Fixed alerts

License Policy Violation

License

This package is not allowed per your license policy. Review the package's license to ensure compliance.

Found 1 instance in 1 package

Improved metrics

Total package byte prevSize

277176

increased by52.08%

Lines of code

4937

increased by47.42%

Number of lines in readme file

283

increased by93.84%

Worsened metrics

Number of package files

57

decreased by-3.39%

Number of low supply chain risk alerts

26

increased by8.33%

Number of medium supply chain risk alerts

3

increased by50%

Dependency changes

• + Added@sinonjs/commons@3.0.1(transitive)

• + Added@sinonjs/fake-timers@11.3.1(transitive)

• + Added@sinonjs/samsam@6.1.3(transitive)

• + Addeddiff@5.2.0(transitive)

• + Addedjust-extend@6.2.0(transitive)

• + Addednise@5.1.9(transitive)

• + Addedpath-to-regexp@6.3.0(transitive)

• + Addedsinon@11.1.2(transitive)

• - Removed@sinonjs/fake-timers@6.0.1(transitive)

• - Removed@sinonjs/samsam@5.3.1(transitive)

• - Removeddiff@4.0.2(transitive)

• - Removedisarray@0.0.1(transitive)

• - Removedjust-extend@4.2.1(transitive)

• - Removednise@4.1.0(transitive)

• - Removedpath-to-regexp@1.9.0(transitive)

• - Removedsinon@10.0.0(transitive)

• Updatedsinon@11.1.2