nolimit-crawl-storing

Nolimit Crawl Storing

This is how to store file and metadata based on injecting data of object. You just need to define your object of metadata and your input files. Install it with npm install nolimit-crawl-storing Just require var file = require('nolimit-crawl-storing')

How to use

file.inject(metadata,object) You need to define your metadata in object type, then input your object/content subsequently in parameter. It will automatically save file if buffer reach buffer maximum size.

file.bufferFlush(metadata) This is how to emptied your buffer, by writing it to file manually. Its useful to make sure your buffer is stored

file.setBufferSize(size) This is how you define your buffer size level, default is 50 or defined by config.json in module

file.setIncomingDir(size) This is how you define your temp dir for files, default is ../ or defined by config.json in module

file.setOutputDir(size) This is how you define your output dir for files, default is ./ or defined by config.json in module