@jupiterone/integration-sdk-core - npm Package Versions

5.7.0 - 2021-02-15

Added

• Expose the following read-only properties on IntegrationProviderAPIError
  • endpoint
  • status
  • statusText

5.6.2 - 2021-02-09

Added

• Added log statement after uploadData() successfully sends data to synchronizer.

5.6.1 - 2021-01-20

Fixed

• Allow duplicate key tracker to grow greater than V8 engine limit. See #420.

5.6.0 - 2021-01-14

Added

• Whitespace trimming of integration config values.
• Support for skipping writing graph object files when running CLI commands

Changed

• Skip logging warn message when a CredentialsError is received in graph object uploads.

5.5.0 - 2021-01-02

Added

• Support for specifying the upload batch size via the uploadBatchSize in createPersisterApiStepGraphObjectDataUploader.

5.4.0 - 2020-12-31

Added

• JobState.hasKey() as an efficient means of determining whether an entity or relationship has been added. This allows integrations to avoid doing any of their own key tracking.

Fixed

• A createIntegrationEntity bug in transferring source properties into an entity caused a memory leak.

5.3.0 - 2020-12-27

Added

• Support for omitting specific graph objects from file storage. Some entities and relationships do not need to be stored on the file system at all. We only need to store graph objects on the file system if we later intend to fetch the data from the job state or iterate over the _type.

  Usage in an integration step:

  {
    id: 'my-step',
    name: 'My step',
    entities: [
      {
        resourceName: 'The Record',
        _type: 'my_record',
        _class: ['Record'],
        indexMetadata: {
          // This _type will not be written to the file system
          enabled: false,
        },
      },
    ],
    relationships: [],
    async exeutionHandler() {
      ...
    }
  }
  

  See PR #404

5.2.1 - 2020-12-23

Fixed

• Fixed issue where the SDK could upload empty entity and relationship graph object files.

5.2.0 - 2020-12-18

Changed

• OPTIMIZATION: Buffer entities and relationships in memory and allow for fast lookups. This change allows us to skip flushing to disk anytime there is a call to findEntity, iterateEntities or iterateRelationships.

  See PR #395

• OPTIMIZATION: Allow FileSystemGraphObjectStore to specify graphObjectBufferThreshold, which defines the maximum number of graph objects that the graph object store can buffer into memory before flushing to disk. Machines with more memory should consider increasing this value as the default is 500.

  See PR #395

• OPTIMIZATION: Continuously upload integration data during collection phase. Previously, our integrations had two primary phases. The first phase was the collection phase, and the second phase was the upload phase. The integration SDK now mixes these two phases and the integrations will upload the collected data to JupiterOne during the integration execution step that it has been collected in.

  See PR #396

• OPTIMIZATION: Reduce the size of the graph object files that are stored on disk by default. Previously, all graph object files written to disk while running the integration locally and running the integration in the managed JupiterOne runtime, were created with whitespace. The file whitespace is now only created when running the integration locally via the CLI commands.

  See PR #399

Removed

• Remove BucketMap as it's no longer used in the FileSystemGraphObjectStore. BucketMap was technically exposed externally, but should not have been used externally.

5.1.0 - 2020-12-08

Added

• Added registerEventHandlers and unregisterEventHandlers exports from the runtime package. These functions will register a common set of event handlers that handle out-of-band errors that can be thrown in integration steps, such as unhandledRejection and multipleResolves. These handlers should be added to any deployment projects that invoke executeIntegrationInstance. Developers should instrument these event handlers as early as possible in the node process.
• Added registerIntegrationLoggerEventHandlers and unregisterIntegrationLoggerEventHandlers as convenience functions, since JupiterOne infrastructure will handle these event emitter failures by calling logger.error().
• Added registerEventHandlers to executeIntegrationLocally, so that events are caught when running yarn j1-integration collect.
• Made registerIntegrationEventHandlers call the integration logger onFailure function.

Changed

• Increase concurrency on integration data uploads
• Implement retry logic around integration data uploads