similarweb_scraperis is a python library for scraping similarweb with proxycrawl api and it can bypass the distil projection so far. It also provides some functionality for transforming scraped data into pd dataframe.
Use the package manager pip to install foobar.
pip install similarweb-scraper
## Usage
from similarweb_scraper import scraper
### get the website html
web_scrape = scraper()
web_scrape.login(#api key from proxycrawl.com)
web_scrape.webpage_scrape(#websit e.g: hk.yahoo.com)
### get the html code
soup = web_scrape.og_soup
### get the html code as json format
web_json = web_scrape.json_storage
### get data into json format
df = web_scrape.metrics_to_df(#str(metrics_type))
##metrics_type name :
#'country_share',
#'traffic_share',
# engagement',
#'monthly_traffic_data'
# more function will be available soon
Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.
Please make sure to update tests as appropriate.
FAQs
Using proxycrawl api to scrape similarweb data
We found that similarweb-scraper demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
Security News
Socket’s threat research team has detected six malicious npm packages typosquatting popular libraries to insert SSH backdoors.
Security News
MITRE's 2024 CWE Top 25 highlights critical software vulnerabilities like XSS, SQL Injection, and CSRF, reflecting shifts due to a refined ranking methodology.
Security News
In this segment of the Risky Business podcast, Feross Aboukhadijeh and Patrick Gray discuss the challenges of tracking malware discovered in open source softare.