url-examiner

url-examiner

A small Node.js tool & library that checks URL from a given CSV input file and report the analysis results in a new CSV output file.

Usage

Prerequisites

You need a working version of:

• Node v14+ (recommended v14.18.0)
• npm 6+ (recommended v6.14.15)

As a tool

# Basic
npx url-examiner -i input_file.csv -o output_file.csv

# Advanced (fullname)
npx url-examiner \ 
  --input input_file.csv \ 
  --output output_file.csv \
  --timeout 5000 \
  --headers "Authorization: Bearer xxx.yyy.zzz"
  --headers "User-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:64.0) Gecko/20100101 Firefox/80.0"
  --from 10
  --to 200
  --bulk 50

# Advanced (shortcut)
npx url-examiner \ 
  -i input_file.csv \ 
  -o output_file.csv \
  -m 5000 \
  -H "Authorization: Bearer xxx.yyy.zzz"
  -H "User-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:64.0) Gecko/20100101 Firefox/80.0"
  -f 10
  -t 200
  -b 50

Options:

option	description	default value
-d --delay number	delay (in ms) between two URL calls	None
-b --bulk number	number of concurrent HTTP calls during analysis phase	10
-f --from number	line "from"	None
-H --headers headers...	HTTP request headers	None
(required) -i --input input_file	input file path	None
-m --timeout max-time	max time allowed to succeed an HTTP check request	1000
-o --ouput filepath	ouput file path	None
-s --separator character	column separator	;
-t --to number	line "to"	None

Input file:

Supported file extension: .csv

Supported data format (without head line):

<reference>;<url>

Output file:

As a library

npm install url-examiner

import Program from './Program';

const program = new Program();

program.run('my-app -i input_file.csv -o output_file.csv').then(() => process.exit(0));