qflatfile

qFlatFile is Flat File parser, writer and converter for node.js services:

var qff = require('qflatfile');
var qffp = qff.getParser({delim: ','})

var sFlatFileString = ""
// Load flat file into a sFlatFileString

var aData = qffp.parse(sFlatFileString)
// aData contains an Array containing the parsed contents from flat file

This library is written to be executed syncronously. As such, it is not recommended to be used in usecases where the data processed is very long. In such cases It would be best to use flat file services that run Asyncronously.

Current Status

This is a first release of a code base that was ported from another language and proved to be very stable in a production environment with millions of cycles. Its not to say that this code will work 100% of the time but its quite solid. If any failure modes are found, please let me know and I will be interested to get fixes done.

Installation

npm install qflatfile

Features

• read any properly formed flat file (csv, tsv, etc.)
• write properly formatted flat files
• convert json file over to a flat file (csv, tsv, etc.)

Using the parser

The parser will convert a string of a properly formed flat file in almost any format (comma, tab etc.)

var qff = require('qflatfile');
var qffp = qff.getParser({delim: ','})

var sFlatFileString = ""
// Load flat file into a sFlatFileString

var aData = qffp.parse(sFlatFileString)
// aData contains an Array containing the parsed contents from flat file

For qff.getParser({<opts>}) the following options are available

Field	Required?	Default	Description
delim	Yes	-	Deliminator for flat file. Single character only.
asJson	No	false	When true, parser will return data in Json format otherwise, data will be as an Array

Using the writer

The writer will convert a Array to a string of a properly formed flat file in almost any format (comma, tab etc.)

var qff = require('qflatfile');
var qffw = qff.getWriter({delim: ','})

var aTable = []
// Load in a 2d Array of data to be sent to a flat file

var sFlatFile = qffw.writeTable(aTable)
// sFlatFile contains a properly formed Flat file of the data

// Alternatively, you can write one record at a time
var aRecord = []
// Load in a single record into aRecord

var sFFRec = qffw.writeRecord(aRecord)
// sFFRec contains a properly formed Flat file record

For qff.getWriter({<opts>}) the following options are available

Field	Required?	Default	Description
delim	Yes	-	Deliminator for flat file. Single character only.

Using the converter

The converter will take a file containing json and will output into a flat file.

var qff = require('qflatfile');
var qffc = qff.getConverter({
	 delim: '\t'
	,jsonIn: true
	,asFileIn: true
	,asFileOut: true
	,fileInName: './testFileIn.json'
	,fileOutName: './flatFileOut.tsv'
	,nullValue: '-'
	,ignoreFields: ["accountId","errorDetails","folderId","id","broadcastMessageId","errorDetails","type"]
	,jsonProcessor: function(r){
		d=new Date(r.timeStamp);
		r.creationDate = new Date(r.creationDate);
		r.date = d.getFullYear()+"/"+d.getMonth()+"/"+d.getDay();
		r.dateTime = d.getHours()+":"+d.getDate();
		return r
	}
});

qffc.convertSync();
// After execution, output file should be generated with properly formatted 
// flat file

For qff.getConverter({<opts>}) the following options are available

Field	Required?	Default	Description
delim	Yes	-	Deliminator for flat file. Single character only.
jsonIn	yes	-	Must be true. This is for future functionality which will support other conversion paths.
asFileIn	yes	-	Must be true. This is for future functionality which will support other conversion paths.
asFileOut	yes	-	Must be true. This is for future functionality which will support other conversion paths.
fileInName	yes	-	Path to file including file name for input.
fileOutName	yes	-	Path to file including file name for output.
nullValue	no	(null)	String to insert when a null / undefined is resolved in processing of json in
ignoreFields	no	false	Array of strings for field names to ignore when processing
jsonProcessor	no	-	Function to be executed for each json record in. Must return json record.

Future

• Converter:
  • CLI tool
  • Conversion of flat file in and JSON file output
  • conversion of flat file in and JSON String out
  • Conversion of flat file string in and JSON file out
  • Conversion of flat file string in and JSON string out
• Parser:
  • Allow processing of an input file as well
• writer:
  • Allow writing to an output file

License

BSD 2 Clause. See "LICENSE.txt"