Sledgehammer is a gem which allows to crawl websites in search of email addresses. It uses Typhoeus and Sidekiq to spawn ultra-fast workers which gathers data in no-time.
Include the gem in your Gemfile
gem "sledgehammer"
Bundle the Gemfile
bundle install
Run the install script which will create a migration file and a config file.
bundle exec rails generate sledgehammer:install
Migrate your database
bundle exec rake db:migrate
You should be aware of using this gem with application with sqlite3 database. Due to multi threaded nature of gem you will be greeted with "SQLite3::BusyException: database is locked" errors. PostgreSQL, MySQL or MongoDB should be just fine.
Run sidekiq worker form your code:
Sledgehammer::CrawlWorker.perform_async ARRAY_OF_URLS, [OPTIONS]
Here is sample usage:
Sledgehammer::CrawlWorker.perform_async ['http://example.com'], { depth_limit: 3 }
[d4rky-pl] (https://github.com/d4rky-pl)
Sledgehammer is Copyright © 2014 Growth Republic. It is free software, and may be redistributed under the terms specified in the LICENSE file.
FAQs
Unknown package
We found that sledgehammer demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 2 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Socket's package search now displays weekly downloads for npm packages, helping developers quickly assess popularity and make more informed decisions.
Security News
A Stanford study reveals 9.5% of engineers contribute almost nothing, costing tech $90B annually, with remote work fueling the rise of "ghost engineers."
Research
Security News
Socket’s threat research team has detected six malicious npm packages typosquatting popular libraries to insert SSH backdoors.