Market Bot is a high performance Ruby scraper for Google's Android Market with a simple to use API. It is built on top of Nokogiri and Typhoeus.
A simple scraper for directed scrapes of APIs, feed or structured HTML. Plays nicely with wuclan and wukong.
А simple module for requests to Instagram without an API key.
Financial information scraper gem. Uses Yahoo Finance API.
A scraper for instapaper to provide a basic api without hacing to pay per month
a simple scraping api for espn stats and data
A instagram scraper to get user instagram data thought public api (search and media)
Scrape ArcGIS data from REST API and export to postgres db
BugGuide.net is a website for sharing insect photos from North America. Over the years it as acrued many experts and a great deal of excellent data, but not an API. This gem is little more than a scraper for conveniently extracting data from the website.
Web Scraper is a library to build APIs by scraping static sites and use data as models.
Ruby wrapper for Prompt API's Scraper Checker API
Scrape Github Organization member information. Currently, the Github API doesn't support the new Organizations feature. This CLI scrapes the list of members for an Organization.
A screen-scraper to retrieve vital information not available via the public API
A screen-scraper to retrieve your cards and transactions from the Opal Card website
Official Ruby client for the GMaps Lead Finder Agent HTTP API — scrape Google Maps places (name, phone, website, emails) via a hosted cloud pipeline.
Twitter API alternative Ruby SDK for tweet search, follower scraping, timelines, DMs, communities, lists, trending, and X automation. Not affiliated with X Corp.
Link Scraper is a website link extractor API that pulls internal and external links from any webpage URL in real time. It resolves relative paths to absolute URLs, strips empty fragments, and returns anchor text with internal and external counts.
Website Images Scraper is an image scraper API that extracts image URLs and hosting domains from any web page. Send a page URL to retrieve direct image sources, breakdown counts for internal versus external assets, and unique hosting domains.
Crawlora is a web-scraping API for structured public web data — search, marketplace, social, finance, media, reviews, and geodata — without running your own scrapers. This gem is the official, typed, batteries-included Ruby client: grouped helpers (client.bing.search) and dynamic operation calls for every endpoint, API-key and JWT auth, automatic retries with exponential backoff and Retry-After, numeric and cursor pagination, before/after middleware hooks, and client-side rate limiting.
X API & Twitter API SDK for Ruby by Xquik. Scrape Twitter search, timelines, profiles & followers as a typed Twitter scraper & tweet scraper. Manage media, webhooks, monitors & X automation through documented Xquik REST routes. Xquik is the world's fastest and cheapest X Twitter scraper service with the most complete X data. Not affiliated with X Corp.
This gems silences all of Sidekiq exceptions in retryable jobs, except the last one. Specially useful when you know you job is most likely going to fail often (scrapers, api calls, etc)
Jekyll plugin to generate a robots.txt to block AI scrapers via the Dark Visitors API (https://darkvisitors.com)