movie-script-scraper

Movie Script Scraper

A simple scraper to retrieve movie scripts by genre or title from IMSDB.

Installation

$ npm install -S movie-script-scraper

Usage

Example Usage

Movie Script Scraper exposes a function; simply pass this function the options (see below), and it will return a promise with an array of the file paths of the scripts it saved.

const mss = require('movie-script-scraper');

const options = {
	genre: 'Action',
	total: 10,
};

mss(options)
	.then(filePaths => {
		console.log(filePaths);
	})
	.catch(err => {
		console.error('There was a problem');
	});

Options

• genre [string] - Any valid film genre, a complete list can be found here.
  • Defaults to "Action".
• total [number] - the total number of scripts you want from a given genre.
  • Defaults to 10.
• title [string] - The name of the film's script you want.
• dest [string] - Location that you want to save your scripts.
  • Defaults to ./scripts in the root directory.

Running from command line

You can run MSS directly from the CLI (if it's globally available in your PATH, e.g. by npm install -g mss) with variety of useful options.

$ movie-script-scraper  --total 10 --genre Comedy

or enter a title:

$ movie-script-scraper  --title 'american sniper'

How it Works

Conveniently IMSDB provides RSS feeds based on movie genre (ex. http://www.imsdb.com/feeds/genre.php?genre=Comedy). Using the awesome fetch module we are then able to grab that page and use a regular expression to generate an array of movie script URLs. We then visit each URL with Fetch and use another awesome module called Cheerio to select just the movie script from each page and finally output each script to a file.

Running Locally

1. Install dependencies with:

$ npm install

1. Run Tests

$ npm test

Contributing

Don't hesitate to create a pull request. Every contribution is appreciated. In development you can start the tests by calling npm test. Checkout our contribution README for more info.

1. Fork it!
2. Create your feature branch: git checkout -b my-new-feature
3. Commit your changes: git commit -am 'Add some feature'
4. Push to the branch: git push origin my-new-feature
5. Submit a pull request :D

Maintainers

Joe Karlsson

License

MIT