4chan-downloader

4chan-downloader

Downloads all files in 4chan threads.

Install

npm install -g 4chan-downloader

CLI

Help output:

Usage: 4chan [options] <url> [<dir>]

Examples:
  4chan <url>                Download all files in a thread into {cwd}.
  4chan <url> foo            Download all files in a thread into {cwd}/foo.
  4chan <url> foo/bar        Download all files in a thread into {cwd}/foo/bar.
  4chan -w <url>             Download all and watch for new files.
  4chan -w -i 20 <url>       Download all and check for new files every 20 seconds.
  4chan -f gif,webm <url>    Download all gif and webm files.
  4chan -s 200x100 <url>     Download all files bigger than 200x100 pixels.
  4chan -s 100 <url>         Download all files bigger than 100x100 pixels.


Options:
  -w, --watch     Watch for new files.
  -i, --interval  Watching interval in seconds.     [default: 10]
  -f, --filter    Filter specific file types.       [default: "jpg,gif,png,webm"]
  -o, --override  Override existing files.
  -n, --names     Save with original file names.
  -s, --size      Smallest files to download.       [default: "100x100"]
  -t, --threads   Limit of simultaneous downloads.  [default: 1]
  -v, --version   Show version.
  -h, --help      Show help.

API

var FortuneDownloader = require('4chan-downloader');
var thread = new FortuneDownloader('http://boards...')
	.threads(4)
	.watch()
	.downloadTo('foo/bar');
// ...
thread.stop();

FortuneDownloader(url)

Consturctor. new keyword is optional. All methods are chaining.

#threads(count)

Set number of downloading threads.

#watch([interval])

Make instance watch for new files.

interval Integer Watching interval in millisecods. Default: 10000

#filter(fun)

Custom function that filters out specific files. Function returns true when the file should be downloaded and false otherwise.

Function arguments:

1. file Object File description object:

   {
   	url: 'http://i.4cdn.org/g/1407348528901.jpg',
   	prettysize: '650 KB',
   	width: 1920,
   	height: 1080,
   	name: 'Original file name.jpg',
   	idname: '1407348528901.jpg',
   	type: 'jpg',
   	size: 0, // set when file stars downloading
   	completed: 0, // updated during downloading
   	progress: 0   // updated during downloading
   }
   

#namer(fun)

Custom function that returns the name to be used when saving files.

Function arguments:

1. file Object File description object:

   {
   	url: 'http://i.4cdn.org/g/1407348528901.jpg',
   	prettysize: '650 KB',
   	width: 1920,
   	height: 1080,
   	name: 'Original file name.jpg',
   	idname: '1407348528901.jpg',
   	type: 'jpg',
   	size: 0, // set when file stars downloading
   	completed: 0, // updated during downloading
   	progress: 0   // updated during downloading
   }
   

#override(state)

Set whether instance should override already existing files. Default behavior is false.

• state Boolean

#destination(path)

Where to save files. By default files are saved into current working directory.

• path String

#download()

Starts downloading.

#downloadTo(path)

Shorthand for:

FortuneDownloader(url)
	.destination('foo/bar')
	.download();

#stop()

Clears current download queue, and cancels watching when in progress.

It does not cancel ongoing download streams.

After #stop() has been called and ongoing downloads finished, end event will be fired. If there were no ongoing downloads, end will be fired immediately.

Properties

url

Type String. Thread URL.

queue

Type Array. Array of files (file descriptor objects) to be downloaded. File is removed from here right before file:start event.

finished

Type Array. Array of downloaded file URLs. When watching is active, any subsequent download will filter out files in this array.

active

Type Integer. Number of currently active download threads in progress.

Events

FortuneDownloader instance is an event emitter.

parse

Triggered when page has been parsed, and #queue array has been populated with files soon to be downloaded.

error

Triggered when download canceling error has happened. When this event fires, end event won't.

Callback arguments:

1. error Error

end

Triggered when download instance has finished downloading all files, and watching is not enabled.

Also fires when #stop() is called and all ongoing downloads finish.

file:start

Triggered when file starts downloading.

Callback arguments:

1. file Object File descriptor object:

   {
   	url: 'http://i.4cdn.org/g/1407348528901.jpg',
   	prettysize: '650 KB',
   	width: 1920,
   	height: 1080,
   	name: 'Original file name.jpg',
   	idname: '1407348528901.jpg',
   	type: 'jpg',
   	size: 665600, // available now
   	completed: 0,
   	progress: 0
   }
   

file:chunk

Triggered when chunk of a file has been received.

Callback arguments:

1. chunk Buffer
2. file Object File descriptor object:

   {
   	url: 'http://i.4cdn.org/g/1407348528901.jpg',
   	prettysize: '650 KB',
   	width: 1920,
   	height: 1080,
   	name: 'Original file name.jpg',
   	idname: '1407348528901.jpg',
   	type: 'jpg',
   	size: 665600,
   	completed: 40253, // sum of all chunks till now
   	progress: 0.06139871873093349603416717510677 // completed / size
   }
   

file:progress

Same as chunk, but without the chunk argument:

1. file Object File descriptor object:

   {
   	url: 'http://i.4cdn.org/g/1407348528901.jpg',
   	prettysize: '650 KB',
   	width: 1920,
   	height: 1080,
   	name: 'Original file name.jpg',
   	idname: '1407348528901.jpg',
   	type: 'jpg',
   	size: 665600,
   	completed: 40253, // sum of all chunks till now
   	progress: 0.06139871873093349603416717510677 // completed / size
   }
   

file:error

Triggered when file downloading error has happened. These errors don't stop the downloading process.

Callback arguments:

1. error Error
2. file Object File descriptor object.

file:end

Triggered when file has finished with downloading.

Callback arguments:

1. file Object File descriptor object.

License

MIT