Huge News!Announcing our $40M Series B led by Abstract Ventures.Learn More
Socket
Sign inDemoInstall
Socket

espider

Package Overview
Dependencies
Maintainers
1
Alerts
File Explorer

Advanced tools

Socket logo

Install Socket

Detect and block malicious and high-risk dependencies

Install

espider

easy spider

  • 1.7.3
  • PyPI
  • Socket score

Maintainers
1

Easy Sipder


Easy Spider 主要有四大模块:

  • Spider 负责推送请求到请求线程池
  • Downloader 负责启动请求与数据,请求在启动前会经过请求处理程序,响应在下载后会经过响应处理程序
  • Pipeline 负责清理数据,数据的持久化等工作

流程图如下

epsider流程图


TODO

  • 2020-04-06

    • 修复 start_requests 错误提示
    • 自动设置请求优先级
    • 请求和响应扩展合并为下载中间件
    • settings 像 scrapy 看齐
  • 2020-04-07

    • 优化 setting
    • 下载器开始停止问题

FAQs


Did you know?

Socket

Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.

Install

Related posts

SocketSocket SOC 2 Logo

Product

  • Package Alerts
  • Integrations
  • Docs
  • Pricing
  • FAQ
  • Roadmap
  • Changelog

Packages

npm

Stay in touch

Get open source security insights delivered straight into your inbox.


  • Terms
  • Privacy
  • Security

Made with ⚡️ by Socket Inc