Node Raspar 🕷️
Easily scrap the web for torrent and media files.
Getting Started
Raspar exports a resolver function which accepts the configuration object as an argument and returns promises for all available methods.
Installation
To use Raspar in your project, run:
npm i node-raspar
Usage
Raspar follows the latest maintenance LTS version of Node and v14.x or greater is recommended.
Configuration
NOTE: You can use any promise based cache store engine from this list, raspar uses fs-hash by default.
const options = {
driver: "1337x",
perPage: 12,
cache: { options: { ttl: 60 * 8 } },
};
Example 1 - get a list of music files (using the zippyshare
driver)
const raspar = require("node-raspar")(options);
const page = 1;
const genre = "Hip Hop";
raspar.list(page, genre).then(console.log).catch(console.error);
Example 2 - search for torrent files (using the 1337x
driver)
const raspar = require("node-raspar")(options);
(async () => {
const page = 1;
const keyword = "avengers";
const results = await raspar.search(keyword, page);
console.log(results);
})();
Available Drivers and Methods
Drivers | Type | Search (query, page) | List (page) |
---|
Zippyshare | Music | :white_check_mark: | :white_check_mark: |
1337x | Torrent | :white_check_mark: | :white_check_mark: |
Netnaija | Movie | :white_check_mark: | :white_check_mark: |
Zoro | Anime | :white_check_mark: | :white_check_mark: |
Running / Deploying the API
This is the first step if you're trying to either run it with or without using docker
In terminal run the following commands to get the code on your machine.
$ git clone https://github.com/kodjunkie/node-raspar.git raspar
$ cd raspar
$ sudo chmod -R 777 temp/
Without Docker
Run these additional commands
$ npm install
$ npm start
With Docker
Run these additional commands
$ docker build -t raspar-api .
$ docker run --name raspar -it -p 3000:3000 raspar-api
With Docker Compose
Run these additional commands
$ docker compose up --build
$ docker compose up
$ docker compose down
$ docker exec -it raspar bash
$ exit
$ npm install
$ node ./node_modules/puppeteer/install.js
Deploying on Heroku
Heroku requires some additional dependencies that aren't included on the Linux box that Heroku spins up for you.
To add the dependencies on deploy, add the Puppeteer Heroku buildpack to the list of buildpacks for your app under Settings > Buildpacks.
The url for the buildpack is https://github.com/CoffeeAndCode/puppeteer-heroku-buildpack
API Documentation
NOTE: localhost
refers to the address your server is running on. By default, it runs on port 3000
Swagger: http://localhost:3000/docs
Liked it?
Hope you liked this project, don't forget to give it a star ⭐
Tests
$ npm test
$ docker exec -it raspar npm test
License
This project is opened under the MIT 2.0 License which allows very broad use for both academic and commercial purposes.