BnetScraper is a Nokogiri-based scraper of Battle.net profile information. Currently this only includes Starcraft2.
Run gem install bnet_scraper or add gem 'bnet_scraper' to your Gemfile.
Say you have the URL of a Battle.net account you would like to scrape. To begin, create an instance
of BnetScraper::Starcraft2::ProfileScraper, passing it the URL. Calling the #scrape method
returns a new BnetScraper::Starcraft2::Profile object with the basic information.
scraper = BnetScraper::Starcraft2::ProfileScraper.new(url: 'http://us.battle.net/sc2/en/profile/2377239/1/Demon/')
profile = scraper.scrape
profile.class.name # => BnetScraper::Starcraft2::Profile
profile.achievement_points # => 3760
profile.account # => 'Demon'
Once you have a BnetScraper::Starcraft2::Profile object, you can easily access other information
for scraping thanks to syntactic sugar. This includes leagues, achievements, and match history.
scraper = BnetScraper::Starcraft2::ProfileScraper.new(url: 'http://us.battle.net/sc2/en/profile/2377239/1/Demon/')
profile = scraper.scrape
profile.recent_achievements # Scrapes achievement information, returns array of achievements
profile.match_history # Scrapes recent match history, returns array of matches
profile.match_history[0].class.name # => BnetScraper::Starcraft2::Match
profile.leagues[0].class.name # => BnetScraper::Starcraft2::League
profile.leagues[0].division # Scrapes the 1st league's information page for rank, points, etc
Interested in grabbing everything about a profile eagerly? You're in luck, because there's a method
just for you. Call BnetScraper::Starcraft2#full_profile_scrape with the usual options hash that
ProfileScraper would take, and it will eager-load the achievements, matches, and leagues.
profile = BnetScraper::Starcraft2.full_profile_scrape(url: 'http://us.battle.net/sc2/en/profile/2377239/1/Demon/')
profile.class.name # => 'BnetScraper::Starcraft2::Profile'
profile.leagues.first.name # => 'Changeling Bravo'
Alternatively, these scrapers can be accessed in isolation.
There are several scrapers that pull various information. They are:
All of the scrapers take an options hash, and can be created by either passing a URL string for the profile URL or passing the account information in the options hash. Thus, either of these two approaches work:
scraper1 = BnetScraper::Starcraft2::ProfileScraper.new(url: 'http://us.battle.net/sc2/en/profile/2377239/1/Demon/')
scraper2 = BnetScraper::Starcraft2::ProfileScraper.new(bnet_id: '2377239', account: 'Demon', region: 'na')
All scrapers have a #scrape method that triggers the scraping and storage. The #scrape method will return an
object containing the scraped data result.
This pulls basic profile information for an account, as well as an array of league URLs. This is a good starting point for league scraping as it provides the league URLs necessary to do supplemental scraping.
scraper = BnetScraper::Starcraft2::ProfileScraper.new(url: 'http://us.battle.net/sc2/en/profile/2377239/1/Demon/')
profile = scraper.scrape
profile.class.name # => BnetScraper::Starcraft2::Profile
Additionally, the resulting BnetScraper::Starcraft2::Profile object has methods to scrape additional
information without the need of creating another scraper. For example, if you need to pull league information up
on a player, you may call BnetScraper::Starcraft2::Profile#leagues and it will scrape and store the information
for memoized access.
scraper = BnetScraper::Starcraft2::ProfileScraper.new(url: 'http://us.battle.net/sc2/en/profile/2377239/1/Demon/')
profile = scraper.scrape
profile.leagues.map(&:division) #=> ['Bronze']
This pulls information on a specific league for a specific account. It is best used either in conjunction with a profile scrape that profiles a URL, or if you happen to know the specific league_id and can pass it as an option.
scraper = BnetScraper::Starcraft2::LeagueScraper.new(league_id: '12345', account: 'Demon', bnet_id: '2377239')
scraper.scrape
# => #<BnetScraper::Starcraft2::League:0x007f89eab7a680
@account="Demon",
@bnet_id="2377239",
@division="Bronze",
@name="Changeling Bravo",
@random=false,
@season="2013 Season 4",
@size="3v3">
This pulls achievement information for an account. Note that currently only returns the overall achievements, not the in-depth, by-category achievement information.
scraper = BnetScraper::Starcraft2::AchievementScraper.new(
url: 'http://us.battle.net/sc2/en/profile/2377239/1/Demon/'
)
achievement_information = scraper.scrape
achievement_information[:recent].size # => 6
achievement_information[:recent].first
# => #<BnetScraper::Starcraft2::Achievement:0x007fef52b0b488
@description="Win 50 Team Unranked or Ranked games as Zerg.",
@earned=#<Date: 2013-04-04 ((2456387j,0s,0n),+0s,2299161j)>,
@title="50 Wins: Team Zerg">
achievement_information[:progress]
# => {:liberty_campaign=>1580,
:swarm_campaign=>1120,
:matchmaking=>1410,
:custom_game=>120,
:arcade=>220,
:exploration=>530}
achievement_information[:showcase].size # => 5
achievement_information[:showcase].first
# => #<BnetScraper::Starcraft2::Achievement:0x007fef52abcb08
@description="Finish a Qualification Round with an undefeated record.",
@title="Hot Shot">
This pulls the 25 most recent matches played for an account. Note that this is only as up-to-date as battle.net is, and will likely not be as fast as in-game.
scraper = BnetScraper::Starcraft2::MatchHistoryScraper.new(
url: 'http://us.battle.net/sc2/en/profile/2377239/1/Demon/'
)
matches = scraper.scrape
matches.size # => 25
wins = matches.count { |m| m.outcome == :win } # => 15
losses = matches.count { |m| m.outcome == :loss } # => 10
matches.first
# => #<BnetScraper::Starcraft2::Match:0x007fef55113428
@date="5/24/2013",
@map_name="Queen's Nest",
@outcome=:win,
@type="3v3">
Scraping is only possible if the site is up. Use this if you want to verify the failed scrape is because the site is down:
BnetScraper::Starcraft2::Status.na # => 'Online'
BnetScraper::Starcraft2::Status.fea # => 'Offline'
BnetScraper::Starcraft2::Status.cn # => nil (China is unsupported)
BnetScraper::Starcraft2::Status.fetch # => [
{:region=>"North America", :status=>"Online"},
{:region=>"Europe", :status=>"Online"},
{:region=>"Korea", :status=>"Online"},
{:region=>"South-East Asia", :status=>"Online"}
]
This pulls the list of 200 Grandmasters for a given region. Each player is returned as a hash.
scraper = BnetScraper::Starcraft2::GrandmasterScraper.new(region: :na)
players = scraper.scraper
players.size # => 200
players[0].keys # => [:rank, :name, :race, :points, :wins, :losses]
I would love to see contributions! Please send a pull request with a feature branch containing specs (Chances are excellent I will break it if you do not) and I will take a look. Please do not change the version as I tend to bundle multiple fixes together before releasing a new version anyway.
Written by Andrew Nordman, see LICENSE for details.
FAQs
Unknown package
We found that bnet_scraper demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Malicious Nx npm versions stole secrets and wallet info using AI CLI tools; Socket’s AI scanner detected the supply chain attack and flagged the malware.
Security News
CISA’s 2025 draft SBOM guidance adds new fields like hashes, licenses, and tool metadata to make software inventories more actionable.
Security News
A clarification on our recent research investigating 60 malicious Ruby gems.