Research
Security News
Quasar RAT Disguised as an npm Package for Detecting Vulnerabilities in Ethereum Smart Contracts
Socket researchers uncover a malicious npm package posing as a tool for detecting vulnerabilities in Etherium smart contracts.
A typescript implementation of the Rapid Automated Keyword Extraction (RAKE) algorithm with Part-of-Speech post-processing
The goal of this library was to create a Typescript translation of the javacsript implementation, which in itself was a translation of the python implementation.
Extraction defaults to processing the input text as if it is written in the English language. Stop words are provided by default using stopwords-iso
In addition, Part-of-Speech post-processing is used to further filter keywords. This is done by using an implementation of brill tagging. For usage details, see the list of brill tag descriptions
let keywords: string[] = [];
console.log(keywords);
> []
const input: string = 'I have some apples and bananas here for the table';
keywords = extractWithRakePos({ text: input });
console.log(keywords);
> ['table', 'bananas', 'apples']
const stop: Set<string> = new Set(['apples']);
keywords = extractWithRakePos({ text: input, additionalStopWordSet: stop });
console.log(keywords);
> ['table', 'bananas']
Differences in regular expressions and stopword lists have big impacts on this algorithm and sticking close to the python means that the code was easy to compare to ensure that it was in the ballpark.
This algorithm is described in Text Mining: Applications and Theory and also in this excellent blog post by Alyona Medelyan.
It operates using only the text you give it and produces surprisingly good results. There are likely better results possible but these mostly seem to involve a combination of Python, Machine Learning and a corpus of data.
The appeal of RAKE is of the "bang for the buck" variety.
Currently this library produces subtly different results than either the paper or the original Python implementation. While the results (especially the top scoring ones) line up nicely, these little deviations represent something to understand and resolve.
After hammering out differences in the results, plans are to focus on
The stopword list used by the python version is here. It has a comment as the first line which might break the world...
Links to other stopword lists can be found here
Any file with one word per line should be fine.
FAQs
A typescript implementation of the Rapid Automated Keyword Extraction (RAKE) algorithm with Part-of-Speech post-processing
We found that rake-pos demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
Security News
Socket researchers uncover a malicious npm package posing as a tool for detecting vulnerabilities in Etherium smart contracts.
Security News
Research
A supply chain attack on Rspack's npm packages injected cryptomining malware, potentially impacting thousands of developers.
Research
Security News
Socket researchers discovered a malware campaign on npm delivering the Skuld infostealer via typosquatted packages, exposing sensitive data.