collect-feeds
A CLI for collect feed URLs.
Installation
npm install collect-feeds
Usages
$ npx collect-feeds [COUNT]
Read URLs from the file url-queue.txt
from the current working directory, then find feed URLs for the first 10
URLs.
With COUNT
, fetch a specified number of URLs.
The found feed URLs are appended to found.yml
, the failed URLs are appended to failed.yml
.
Once a URL was fecthed, it will be appended to skip-list.txt
, so next time will fast forward these URLs.
You can modify skip-list.txt
to skip some URLs or re-fetch some URLs.
License
Copyright (c) 2020 dailyrandomphoto. Licensed under the MIT license.