@wmfs/pg-delta-file

pg-delta-file

Outputs change-only-update CSV files (or “delta” files) that contain all the necessary actions required to re-synchronize rows in a cloned table.

Usage

const generateDeltaFile = require('pg-delta-file')

const deltaInfo = await generateDeltaFiles(
  since: '2017-07-16T20:37:26.847Z',
  outputFilepath: '/some/temp/dir/people-delta.csv',
  actionAliases: {
    insert: 'u',
    update: 'u',
    delete: 'd'
  },
  createdColumnName: '_created',
  modifiedColumnName: '_modified',
  transformFunction: (row, callback) => { ... } // optional data transformation
  filterFunction: (row) => { ... } // option filter predicate
  dryrun: true // optional flag, set true to return info without generating output file 
  csvExtracts: {
    '[schema.]people': [
      'PERSON', // Just output a literal
      '$ACTION', // Will output 'u' or 'd'
      '$ROW_NUM', // Row counter
      '@social_security_id', /// Column data
      '@first_name',
      '@last_name',
      '@age'
      '$DATESTAMP',
      '$TIMESTAMP',
      '$DATETIMESTAMP',
    ],
    '[schema2.]address': [
      ...
    ]
  }
)

/*
{
  "totalCount": 5,
  "people": {
    "totalCount": 5
  },
  "address": {
    "totalCount": 3
    "filteredCount": 2
  }
} 
*/

Install

$ npm install pg-delta-file --save

License

MIT

Changelog

1.67.0 (2021-01-12)

🛠 Builds

• deps: Bump pg-query-stream from 3.2.3 to 4.0.0 (8c1c3e7)