github.com/google/go-github/scrape
Package scrape provides a client for interacting with GitHub using screen scraping. This is intended to be used as a supplement to the standard go-github library to access data that is not currently exposed by either the official REST or GraphQL APIs. Because of the nature of screen scraping, this package should be treated as HIGHLY EXPERIMENTAL, and potentially unstable. We make no promises relating to compatibility or stability of the exported API. Even though this package is distributed as part of the go-github library, it is explicitly exempt from any stability promises that my be implied by the library version number.
Readme
The scrape package provides an experimental client for accessing additional GitHub data via screen scraping. It is designed to be a client of last resort for data that cannot be retrieved via the REST or GraphQL APIs.
Add only what you need. Whereas the main go-github library attempts to implement the entire GitHub REST API, there is little point in trying to do that here. Certainly, feel free to contribution patches to get data you actually need, but I'd rather not try and provide exhaustive coverage of all GitHub data here.
Add only what can't be accessed elsewhere. If the data can be retrieved through the REST or GraphQL API, use the appropriate libraries for that.
Prefer read-only access. For now, I'm only focusing on reading data. It might be that writing works fine as well, but it is of course much riskier.
See apps.go for examples of methods that access data. Basically,
fetch the contents of the page using client.get, and then use goquery to dig
into the markup on the page. Prefer selectors that grab semantic ID or class
names, as they are more likely to be stable.
FAQs
Package scrape provides a client for interacting with GitHub using screen scraping. This is intended to be used as a supplement to the standard go-github library to access data that is not currently exposed by either the official REST or GraphQL APIs. Because of the nature of screen scraping, this package should be treated as HIGHLY EXPERIMENTAL, and potentially unstable. We make no promises relating to compatibility or stability of the exported API. Even though this package is distributed as part of the go-github library, it is explicitly exempt from any stability promises that my be implied by the library version number.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Cyber extortion in the US and Canada hit record levels in 2023, with ransomware attacks surging and median ransom demands skyrocketing, though fewer companies are choosing to pay ransoms.
Security News
Ecma TC39 is meeting this week and has moved key ECMAScript proposals forward, advancing Deferred Import Evaluation, Error.isError(), RegExp Escaping, and Promise.try to the next stages.
Security News
Researchers have demonstrated that teams of LLM agents can exploit zero-day vulnerabilities with a 53% success rate, and the costs of using AI to do so are rapidly becoming more affordable than hiring a human penetration tester.
