
Research
/Security News
Toptal’s GitHub Organization Hijacked: 10 Malicious Packages Published
Threat actors hijacked Toptal’s GitHub org, publishing npm packages with malicious payloads that steal tokens and attempt to wipe victim systems.
github.com/winszheng/crawler
本爬虫爬取了珍爱网所有城市第一页用户的信息。信息存储在docker运行的elasticsearch。
开发流程:单任务版->并发版->分布式版。
用docker跑elasticsearch,版本号需要根据自己的docker进行修改:
docker run -d -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" elasticsearch:7.11.1
直接运行single下的main.go以及resources中的main.go。
例如,开一个存储server和三个爬虫server:
// 开存储server
cd ./distributed/persist/server
go run itemsaver.go --port=1234
// 开爬虫server,数量任意
cd ./distributed/worker/server
go run worker.go --port=9000
go run worker.go --port=9001
go run worker.go --port=9002
// 开爬虫client
go run main.go -itemsaver_host=":1234" -worker_hosts=":9000,:9001,:9002"
搜索结果如下,实际上,爬取的数据种类比前端显示出来的更多,但是前端不是本项目的重点(所以就随便整整得了吧)。
FAQs
Unknown package
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
/Security News
Threat actors hijacked Toptal’s GitHub org, publishing npm packages with malicious payloads that steal tokens and attempt to wipe victim systems.
Research
/Security News
Socket researchers investigate 4 malicious npm and PyPI packages with 56,000+ downloads that install surveillance malware.
Security News
The ongoing npm phishing campaign escalates as attackers hijack the popular 'is' package, embedding malware in multiple versions.