csvhelper

CSVHelper

A simple CSV Convertor, Ever

This is command line interface tool to perform some conversion operation given in a config file on any given csv file and will write the converted csv file to the given output location

Features

• Formate date column by cofigured formate using moment package.
• Filtering data by configurable dynamic predicate.
• Create new column by aggregating existing column together in the configured order.
• Create a new column value based on a predicate expression evalauted from existing column values.

Tech

CSVHelper uses a number of open source projects to work properly:

• [create-file] - file system operations!
• [csv-parser] - parsing input csv files
• [csvtojson] - convert csv to json to perform transformation easily
• [json-2-csv] - convert json back to csv
• [moment] - perform manipulation on date values
• [yargs] - configure command args for this cli tool

And of course CSVHelper itself is open source with a public repository on GitHub.

Installation

CSVHelper requires Node.js v10+ to run.

Install

npm install -g csvhelper

Usage from any where in the cmd execute below commands with mentioned args

csvhelper -s <sourceCSVpath> -c <configJSONfilePath> -o <outputFilePath>

Config JSON options

below is the total JSON config options, based on the operations concerned options should be provided to perform the transformation

{
    "operation": "",  
    // allowed operation values
    // formatter , filter, merge, formulaField
    "predicateOptions": {
        "columnName" : "mention column name base on which predicate should perform",
        "value": "threshold value",
        "operator": "operator for comparison"
    },
    // predicateOptions is required when the operation is filter
    "formateOptions": {
        "columnName": "column name on which formating should happen currently supports date column type",
        "formate" : "formate option ex MMM d, YYYY"
    },
    // formateOptions is required when the operation is formatter
    "mergeOptions": {
        "newColumnName": "new column name",
        "mergeColumns": ["array of column names in string"], ex ["col1", "col2"]
        "formate": "string inpeterpolation order based on which columns should be merged ex: ${0}, ${1} will give col1, col2"
    }
    // mergeOptions is required when the operation is merge
    "formulaFieldOptions" : {
        "columnName": "new column name",
        "predicate": {
            "field":"field on which predicate should be performed",
            "value": "threshold value",
            "operator": "comparison operator",
            "truthyValue":"what value should be added to column if predicate is true",
            "falsyValue": "what value should be added to column if predicate is fals"
        }
    }
    //formulaFieldOptions is required when the operation is formulaField
}

sample configs

for formatter operation

{
    "operation" : "formatter",
    "formateOptions" : {
        "columnName" : "dob",
        "formate": "MMM d, YYYY"
    }
}

filter operation config

{
    "operation" : "filter",
    "predicateOptions" : [{
        "columnName": "age",
        "value": "50",
        "operator": ">"
        
    }]
}

Merge operation config

{
    "operation" : "merge",
    "mergeOptions": {
        "newColumnName": "name",
        "mergeColumns": ["first_name","last_name"],
        "formate": "${0}, ${1}"
    }
}

Formula field config

{
    "operation" : "formulaField",
    "formulaFieldOptions": {
        "columnName": "eligible_for_voting",
        "predicate": {
            "field":"age",
            "value": "18",
            "operator": ">",
            "truthyValue":"T",
            "falsyValue": "F"
        }
    }
}

License

MIT

Free Software, Hell Yeah!