Huge News!Announcing our $40M Series B led by Abstract Ventures.Learn More
Socket
Sign inDemoInstall
Socket

fugotcha

Package Overview
Dependencies
Maintainers
1
Versions
1
Alerts
File Explorer

Advanced tools

Socket logo

Install Socket

Detect and block malicious and high-risk dependencies

Install

fugotcha

A web scraper for Dischord's Fugazi Live Series

  • 1.0.0
  • latest
  • Source
  • npm
  • Socket score

Version published
Maintainers
1
Created
Source

Fugotcha

Fugotcha is a command-line utility for scraping data from the Fugazi Live Series on Dischord.com.

You might be interested in this project if:

  • You're a Fugazi-obsessed friend of mine (I have many of these) working on a data-visualization project for school (with no data), and your first instinct was to collect it manually.
  • You're curious about Puppeteer, Google's Node library for controlling Chrom(e|ium) programmatically, or any of the other technologies used. This project contains relatively simple implementations to get your feet wet.
  • You'd like to teach me something about how to do it better. This was a learning project for me; create an issue if you've got something to share!

Installation

Use the package manager npm to install fugotcha.

npm install --global fugotcha

Requirements

Requires Node v7.6.0 or greater.

Usage

Scrape every (-c 0) release, starting with the first (-p washington-dc-usa-90387), and put the results in /tmp/fugotcha.csv:

fugotcha -p washington-dc-usa-90387 -c 0 -o /tmp/fugotcha.csv

Get help:

fugotcha -h

Notes

Data is dumped into a CSV-ish file specified by the -f option.

CSV-ish? RFC 4180 specifies that each line of a CSV "should contain the same number of fields throughout the file." Because the number of tracks is variable across releases, the rows of the CSV will have different numbers of fields unless I pick an arbitrary number of tracks (x) and supply empty values for releases with fewer than x tracks. Doing so, however, guarantees that Fugazi will get together for another show and play x+1 songs -- and then where would we be?

LibreOffice handles CSV-ish files with unequal numbers of fields without problem. I suspect most spreadsheet applications do.

Technologies

Why Puppeteer?

Dischord's website doesn't appear to do much DOM manipulation via JavaScript. One could argue that launching a web browser capable of interpreting JavaScript, headless though it may be, is overkill when the content could be fetched using wget or similar. That's a totally valid argument.

So, why puppeteer? Eh, I wanted to learn something new.

Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

License

AGPL-3.0

FAQs

Package last updated on 08 Dec 2018

Did you know?

Socket

Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.

Install

Related posts

SocketSocket SOC 2 Logo

Product

  • Package Alerts
  • Integrations
  • Docs
  • Pricing
  • FAQ
  • Roadmap
  • Changelog

Packages

npm

Stay in touch

Get open source security insights delivered straight into your inbox.


  • Terms
  • Privacy
  • Security

Made with ⚡️ by Socket Inc