csv-parser

What is csv-parser?

The csv-parser npm package is a tool for parsing CSV files and streams in Node.js. It converts CSV data into readable streams of JSON objects, allowing for easy manipulation and processing of CSV data within a Node.js application.

What are csv-parser's main functionalities?

Parsing CSV Files

This feature allows you to parse CSV files into JSON objects. The code sample demonstrates how to read a CSV file using a readable stream, parse it with csv-parser, and collect the resulting JSON objects into an array.

const csv = require('csv-parser');
const fs = require('fs');
const results = [];

fs.createReadStream('data.csv')
  .pipe(csv())
  .on('data', (data) => results.push(data))
  .on('end', () => {
    console.log(results);
    // Handle the parsed data here
  });

Custom Headers

This feature allows you to specify custom headers for the CSV data if the CSV file does not contain a header row. The code sample shows how to define a custom set of headers that will be used to map the CSV columns to JSON object properties.

const csv = require('csv-parser');
const fs = require('fs');
const results = [];

fs.createReadStream('data.csv')
  .pipe(csv({ headers: ['column1', 'column2', 'column3'] }))
  .on('data', (data) => results.push(data))
  .on('end', () => {
    console.log(results);
  });

Skip Lines

This feature allows you to skip a specified number of lines at the beginning of the CSV file, which can be useful for skipping metadata or comments. The code sample demonstrates how to skip the first two lines of the CSV file before parsing the data.

const csv = require('csv-parser');
const fs = require('fs');
const results = [];

fs.createReadStream('data.csv')
  .pipe(csv({ skipLines: 2 }))
  .on('data', (data) => results.push(data))
  .on('end', () => {
    console.log(results);
  });

Other packages similar to csv-parser

papaparse

Papa Parse is a powerful CSV parser that supports browser and server-side parsing. It offers features like auto-detection of delimiters, streaming large files, and parsing local and remote files. Compared to csv-parser, Papa Parse has a broader feature set and can be used in both browser and Node.js environments.

fast-csv

Fast-csv is another CSV parsing and formatting library for Node.js. It provides a simple API for parsing and formatting CSV data and supports both streams and promises. Fast-csv is known for its performance and ease of use, and it is a good alternative to csv-parser with similar stream-based processing capabilities.

csvtojson

csvtojson is a full-featured CSV parser library that converts CSV to JSON. One of its key features is the ability to handle large files and streams efficiently. It also supports custom column parsers and has a slightly different API compared to csv-parser, offering more customization options for the parsing process.

README

csv-parser

Streaming CSV parser that aims for maximum speed as well as compatibility with the csv-spectrum CSV acid test suite

npm install csv-parser

csv-parser can convert CSV into JSON at at rate of around 90,000 rows per second (perf varies with data, try bench.js with your data).

Usage

Simply instantiate csv and pump a csv file to it and get the rows out as objects

You can use csv-parser in the browser with browserify

var csv = require('csv-parser')
var fs = require('fs')

fs.createReadStream('some-csv-file.csv')
  .pipe(csv())
  .on('data', function(data) {
    console.log('row', data)
  })

The data emitted is a normalized JSON object

The csv constructor accepts the following options as well

var stream = csv({
  raw: false,     // do not decode to utf-8 strings
  separator: ',', // specify optional cell separator
  quote: '"',     // specify optional quote character
  escape: '"',    // specify optional escape character (defaults to quote value)
  newline: '\n',  // specify a newline character
  strict: true    // require column length match headers length
})

It accepts too an array, that specifies the headers for the object returned:

var stream = csv(['index', 'message'])

// Source from somewere with format 12312,Hello World
origin.pipe(stream)
  .on('data', function(data) {
    console.log(data) // Should output { "index": 12312, "message": "Hello World" }
  })

or in the option object as well

var stream = csv({
  raw: false,     // do not decode to utf-8 strings
  separator: ',', // specify optional cell separator
  quote: '"',     // specify optional quote character
  escape: '"',    // specify optional escape character (defaults to quote value)
  newline: '\n',  // specify a newline character
  headers: ['index', 'message'] // Specifing the headers
})

If you do not specify the headers, csv-parser will take the first line of the csv and treat it like the headers

Command line tool

There is also a command line tool available. It will convert csv to line delimited JSON.

npm install -g csv-parser

Open a shell and run

$ csv-parser --help # prints all options
$ printf "a,b\nc,d\n" | csv-parser # parses input

Options

You can specify these CLI flags to control the JSON serialization output format

• outputSeparator - default \n, what to put between JSON items in the output
• beforeOutput - default empty, what to put at beginning of output
• afterOutput - default \n, what to put at end of output

For example, to produce an object with a JSON array of items as output:

--beforeOutput='{"items":[' --afterOutput=]} --outputSeparator=,

License

MIT