@alexpekurovsky/logparser

LogParser

LogParser is converter from file with single JSON object per line to CSV file.

This project is done as exam task for IronSource.

Features:

• Converting single file
• Converting all files in directory
• Converting files in directory matching pattern
• Converting files in directory and subdirectories (recursively) matching pattern
• Multiple files processing simultaneously

Installation & Usage

$ npm i @alexpekurovsky/logparser [-g]
$ logparser FILENAME  # to process single file
$ logparser DIRNAME   # to process all files in dir
$ logparser DIRNAME PATTERN  # to process all files in dir matching mattern
$ logparser DIRNAME PATTERN true  # to process all files in dir and subdirs matching pattern

Examples

$ logparser /var/log/IronSourceApp/IS_main.log
$ logparser /var/log/IronSourceApp
$ logparser /var/log/IronSourceApp '*.log'
$ logparser /var/log 'IS_*.log' true

Demo

$ ll
-rw-r--r--  1 root root 6651236352 Jun 28 01:55 big.log
-rw-r--r--  1 root root       2822 Jun 27 23:20 light.log
-rwxr-xr-x  1 root root        238 Jun 28 01:54 makebig.sh
-rw-r--r--  1 root root   51962784 Jun 27 22:40 Mundo-json-2013-01-03.log

$ logparser . '*.log'
Processing big.log ...
Processing light.log ...
Processing Mundo-json-2013-01-03.log ...
Processing light.log completed.
Processing Mundo-json-2013-01-03.log completed.
Processing big.log completed.

$ ll -h
-rw-r--r--  1 root root 6,2G Jun 28 01:55 big.log
-rw-r--r--  1 root root 3,6G Jun 28 02:18 big.log.csv
-rw-r--r--  1 root root 2,8K Jun 27 23:20 light.log
-rw-r--r--  1 root root 1,8K Jun 28 01:56 light.log.csv
-rwxr-xr-x  1 root root  238 Jun 28 01:54 makebig.sh
-rw-r--r--  1 root root  50M Jun 27 22:40 Mundo-json-2013-01-03.log
-rw-r--r--  1 root root  29M Jun 28 01:56 Mundo-json-2013-01-03.log.csv

License

Feel free to use this code, if needed

More info

Updates to code to work more efficient on big files after Job Offer :)