Research
Security News
Malicious npm Packages Inject SSH Backdoors via Typosquatted Libraries
Socket’s threat research team has detected six malicious npm packages typosquatting popular libraries to insert SSH backdoors.
Python port of open source text processing library for Turkish, zemberek-nlp
Python implementation of Natural Language Processing library for Turkish, zemberek-nlp. It is based on zemberek 0.17.1 and is completely written in Python meaning there is no need to setup a Java development environment to run it.
Source Code
https://github.com/Loodos/zemberek-python
Dependencies
Currently, following modules are supported.
Core (Partially)
TurkishMorphology (Partially)
Tokenization
Normalization (Partially)
You can install the package with pip
pip install zemberek-python
Example usages can be found in examples.py
There are some minor changes in codes where original contains some Java specific functionality and data structures. We used Python equivalents as much as we could but sometimes we needed to change them. And it affects the performance and accuracy a bit.
In MultiLevelMphf class, in the original Java implementation, there are some integer multiplication operations which I tried to reimplement using vanilla Python 'int', but the results were not the same. Then I tried it with numpy.int32 and numpy.float32, since default java int and float types are 4 byte. The results were the same with Java, however, oftenly these operations produced RuntimeWarning as the multiplication caused overflow. In Java there were no overflow warnings whatsoever. I could not find a reasonable explanation to this situation, nor I could find a better way to implement it. So I suppressed overflow warnings for MultiLevelMphf. Therefore, please be aware that, this is not a healthy behaviour, and you should be careful using this code.
This project is Python port of zemberek-nlp.
FAQs
Python port of open source text processing library for Turkish, zemberek-nlp
We found that zemberek-python demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 2 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
Security News
Socket’s threat research team has detected six malicious npm packages typosquatting popular libraries to insert SSH backdoors.
Security News
MITRE's 2024 CWE Top 25 highlights critical software vulnerabilities like XSS, SQL Injection, and CSRF, reflecting shifts due to a refined ranking methodology.
Security News
In this segment of the Risky Business podcast, Feross Aboukhadijeh and Patrick Gray discuss the challenges of tracking malware discovered in open source softare.