file-fetcher

I don't want to read the README

Just show me how it works as fast as possible!

npm i -g file-fetcher
mkdir /tmp/file-fetcher-test
cd !$
echo '[
  {
    "url": "https://dummyimage.com/200x100/ddd/777.jpg&text=file-fetcher+test",
    "path": "img",
    "name": "#.jpg",
    "delay": 1
  }
]' > config.json
file-fetcher config.json

The images are being downloaded to /tmp/file-fetcher-test/img every minute.

File fetcher

file-fetcher is a simple tool that lets you download files recurrently.

Installation

$ npm install -g file-fetcher

Usage

• Create a config file with your configuration (See documentation and example below)
• Quick use:

$ file-fetcher <config file>

• Pro use:

$ file-fetcher
    [ --config | -c ]      <config file>
    [ --destination | -d   <destination folder>  ]

Default destination folder is cwd.

Config file properties

The config file must contain an array of objects. Each object corresponds to a file and can/must have the following properties:

Property	Type	Required?	Default	Description
url	String	✓		The complete URL from which to download the file
path	String	✓		The relative local path where to save the file
name	String	✓		The name pattern of the destination file (e.g.: my-file-number-#.jpg)
description	String		name	The string used to represent the file in log messages
firstIndex	Number		1	The first number used as index (#) in name pattern
digits	Number		3	The minimum number of digits for the index (#) in name pattern. Smaller numbers will be padded with 0s on the left.
cron	String	✓ or delay		The cron pattern that determines the download frequence
delay	Number	✓ or cron		The delay in minutes that determines the download frequence

File name pattern

The file name needs to be a pattern in order to not overwrite the same file after each download.

Pattern substitutions:

String	Replaced by
#	Index of the download iteration
{date}	A string representing the current time

delay vs cron

To determine the download frequence of each file, you can use either the delay property or the cron property.

• delay is simply a number of minutes between each download
  N.B.: due to the cron implementation, using a number like 29 will trigger a download at 00:00, 00:29, 00:58, 01:00, 01:29, ...
• cron is a cron expression

Config example

[
  {
    "url": "http://www.my-site.com/path/to/file.jpg",
    "name": "file_nb_#({date}).jpg",
    "path": "relative/path/to/file",
    "cron": "0 0/2 7-18 * * *",
    "description": "my file"
  },
  {
    "url": "http://www.my-site.com/path/to/other/file.jpg",
    "name": "file_nb_#.jpg",
    "firstIndex": 42,
    "digits": 4,
    "path": "relative/path/to/other/file",
    "delay": 3,
    "description": "my other file"
  }
]

A functional .sample.config.json file is provided in the sources.