Security News
Introducing the Socket Python SDK
The initial version of the Socket Python SDK is now on PyPI, enabling developers to more easily interact with the Socket REST API in Python projects.
This is a light crawling tool.
High performance.
Easy to use.
python>=3.7
setup.py
git clone https://github.com/lthoangg/shopee-crawler
python/python3 setup.py install
pip
pip install shopee-crawler
Examples:
from shopee_crawler import Crawler
crawler = Crawler()
crawler.set_origin(origin="shopee.vn") # Input your root Shopee website of your country that you want to crawl
data = crawler.crawl_by_shop_url(shop_url='shop_url')
data = crawler.crawl_by_cat_url(cat_url='cat_url')
data = crawler.crawl_by_search(keyword='keyword')
data = crawler.crawl_cat_list()
# print(data)
About 12.000 rows (products) in 2-3 seconds (32 workers)
About 12.000 rows (products) in 5-6 seconds (16 workers)
import time
from shopee_crawler import Crawler
crawler = Crawler()
crawler.set_origin(origin="shopee.vn")
keyword = 'điện thoại samsung'
# Start time
start = time.time()
# Crawl
data = crawler.crawl_by_search(keyword=keyword)
# End time
end = time.time()
# Time crawling
print("Time : ",end - start, "seconds")
FAQs
Crawler for Shopee from all country
We found that shopee-crawler demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
The initial version of the Socket Python SDK is now on PyPI, enabling developers to more easily interact with the Socket REST API in Python projects.
Security News
Floating dependency ranges in npm can introduce instability and security risks into your project by allowing unverified or incompatible versions to be installed automatically, leading to unpredictable behavior and potential conflicts.
Security News
A new Rust RFC proposes "Trusted Publishing" for Crates.io, introducing short-lived access tokens via OIDC to improve security and reduce risks associated with long-lived API tokens.