Security News
tea.xyz Spam Plagues npm and RubyGems Package Registries
Tea.xyz, a crypto project aimed at rewarding open source contributions, is once again facing backlash due to an influx of spam packages flooding public package registries.
Readme
利用生成器、协程等流式处理数据。
对数据按多个函数的组合进行过滤、去重,对数据列进行转换,对数据进行计算。
对数据进行质量控制(QC)检查,确保最后的数据与预期相符。
将数据写入csv等。
编程主要要到了生成器,各个类用for循环从上游抽取数据,用yield给下游提供数据。通过改写or规则,利用|操作符将各个组建组合起来。
同时支持协程,构建广播、路由等数据节点。
特点: - 高拓展性。 - 低内存占用。 - 近依赖python基础包。
整体思路主要参考了这个项目:https://github.com/sandabuliu/python-stream。
pip install file-stream
从CSV文件读取数据,按条件筛选后输出到屏幕。
::
reader = CsvReader('/home/hetao/Data/p5w/数据分析/IPO_RoadShow.txt', delimiter='\t', encoding='gbk')
fit = Filter(lambda x: True)
writer = ScreenOutput(end='\r')
p = reader | fit | writer
p.output()
更多范例参见test文件夹。
FAQs
process data as stream.
We found that file-stream demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Tea.xyz, a crypto project aimed at rewarding open source contributions, is once again facing backlash due to an influx of spam packages flooding public package registries.
Security News
As cyber threats become more autonomous, AI-powered defenses are crucial for businesses to stay ahead of attackers who can exploit software vulnerabilities at scale.
Security News
UnitedHealth Group disclosed that the ransomware attack on Change Healthcare compromised protected health information for millions in the U.S., with estimated costs to the company expected to reach $1 billion.