json-column-parser

json-column-parser

A JSON parser for several big data sources formatted columns.

This package provides the required functionality to process the type of a complex column to a convenient json representation.

Nested Hive Column Example

array<struct<student:struct<name:string,id:string,subjects:array<struct<sub_name:string,sub_id:bigint>>>,supply:struct<
amount:bigint,product:struct<product_id:string,product_name:string,template:struct<code:string,currency:string>>,color:string>>>

Quick Start

Installation

Install this component with NPM.

npm install --save json-column-parser

Implementation

import { parser, types } from "json-column-parser";

// Parse one or more columns
// The nestedColumn used is the one from the example 	above
const parsed = parser.parseColumns([
  { name: "test_column", type: nestedColumn, description: "test_description" },
]);

console.dir(parsed[0].colType);
console.dir((parsed[0].colType as types.JSONArray)[0]);

Output

[
  {
    student: { name: 'string', id: 'string', subjects: [Array] },
    supply: { amount: 'bigint', product: [Object], color: 'string' }
  }
]

{
  student: { name: 'string', id: 'string', subjects: [ [Object] ] },
  supply: {
    amount: 'bigint',
    product: {
      product_id: 'string',
      product_name: 'string',
      template: [Object]
    },
    color: 'string'
  }
}