Product
Introducing SSO
Streamline your login process and enhance security by enabling Single Sign-On (SSO) on the Socket platform, now available for all customers on the Enterprise plan, supporting 20+ identity providers.
retext-keywords
Advanced tools
Readme
retext plugin to extract keywords and key phrases.
This package is a unified (retext) plugin to extract keywords and key phrases from a document, and exposes that metadata on the file.
You can use this plugin any time you’re dealing with unified or retext already, and are interested in keywords and key phrases. Importantly, keywords extraction in NLP is a rather heavy and sometimes fragile process, so you might be better off manually providing a list of keywords.
This package is ESM only. In Node.js (version 16+), install with npm:
npm install retext-keywords
In Deno with esm.sh
:
import retextKeywords from 'https://esm.sh/retext-keywords@8'
In browsers with esm.sh
:
<script type="module">
import retextKeywords from 'https://esm.sh/retext-keywords@8?bundle'
</script>
Say our document example.txt
contains (from Wikipedia):
Terminology mining, term extraction, term recognition, or glossary extraction, is a subtask of information extraction. The goal of terminology extraction is to automatically extract relevant terms from a given corpus.
In the semantic web era, a growing number of communities and networked enterprises started to access and interoperate through the internet. Modeling these communities and their information needs is important for several web applications, like topic-driven web crawlers, web services, recommender systems, etc. The development of terminology extraction is essential to the language industry.
One of the first steps to model the knowledge domain of a virtual community is to collect a vocabulary of domain-relevant terms, constituting the linguistic surface manifestation of domain concepts. Several methods to automatically extract technical terms from domain-specific document warehouses have been described in the literature.
Typically, approaches to automatic term extraction make use of linguistic processors (part of speech tagging, phrase chunking) to extract terminological candidates, i.e. syntactically plausible terminological noun phrases, NPs (e.g. compounds "credit card", adjective-NPs "local tourist information office", and prepositional-NPs "board of directors" - in English, the first two constructs are the most frequent). Terminological entries are then filtered from the candidate list using statistical and machine learning methods. Once filtered, because of their low ambiguity and high specificity, these terms are particularly useful for conceptualizing a knowledge domain or for supporting the creation of a domain ontology. Furthermore, terminology extraction is a very useful starting point for semantic similarity, knowledge management, human translation and machine translation, etc.
…and our module example.js
contains:
import {toString} from 'nlcst-to-string'
import {retext} from 'retext'
import retextKeywords from 'retext-keywords'
import retextPos from 'retext-pos'
import {read} from 'to-vfile'
const file = await retext()
.use(retextPos) // Make sure to use `retext-pos` before `retext-keywords`.
.use(retextKeywords)
.process(await read('example.txt'))
console.log('Keywords:')
if (file.data.keywords) {
for (const keyword of file.data.keywords) {
console.log(toString(keyword.matches[0].node))
}
}
console.log()
console.log('Key-phrases:')
if (file.data.keyphrases) {
for (const phrase of file.data.keyphrases) {
console.log(toString(phrase.matches[0].nodes))
}
}
…then running node example.js
yields:
Keywords:
term
extraction
Terminology
web
domain
Key-phrases:
terminology extraction
terms
term extraction
knowledge domain
communities
This package exports no identifiers.
The default export is retextKeywords
.
unified().use(retextKeywords[, options])
Extract keywords and key phrases.
The results are stored on file.data.keyphrases
(Array<Keyphrase>
) and file.data.keywords
(Array<Keyword>
).
options
(Options
, optional)
— configurationTransform (Transformer
).
Keyphrase
Info on a key phrase (TypeScript type).
matches
(Array<PhraseMatch>
)
— all matchesscore
(number
)
— score of phrase, for one matchstems
(Array<string>
)
— stems of phraseweight
(number
)
— score of phrase, for all matchesKeyword
Info on a keyword (TypeScript type).
matches
(Array<WordMatch>
)
— all matchesscore
(number
)
— score of word, for all matchesstem
(string
)
— stems of wordOptions
Configuration (TypeScript type).
maximum
(number
, default: 5
)
— try to detect at most maximum
words and phrases; actual counts may
differ, for example, when two words have the same score, both will be
returned; when too few words exist, less will be returnedPhraseMatch
Match (TypeScript type).
nodes
(Array<Node>
)
— matched nodesparent
(Node
)
— parentWordMatch
Match (TypeScript type).
This package is fully typed with TypeScript.
It exports the additional types
Keyphrase
,
Keyword
,
Options
,
PhraseMatch
, and
WordMatch
.
It also registers the file.data
fields with vfile
.
If you’re working with the file, make sure to import this plugin somewhere in
your types, as that registers the new fields on the file.
/**
* @typedef {import('retext-keywords')}
*/
import {VFile} from 'vfile'
const file = new VFile()
console.log(file.data.keywords) //=> TS now knows this is `Array<Keyword> | undefined`.
Projects maintained by the unified collective are compatible with maintained versions of Node.js.
When we cut a new major release, we drop support for unmaintained versions of
Node.
This means we try to keep the current release line, retext-keywords@^8
,
compatible with Node.js 16.
See contributing.md
in retextjs/.github
for ways
to get started.
See support.md
for ways to get help.
This project has a code of conduct. By interacting with this repository, organization, or community you agree to abide by its terms.
FAQs
retext plugin to extract keywords
The npm package retext-keywords receives a total of 1,484 weekly downloads. As such, retext-keywords popularity was classified as popular.
We found that retext-keywords demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Product
Streamline your login process and enhance security by enabling Single Sign-On (SSO) on the Socket platform, now available for all customers on the Enterprise plan, supporting 20+ identity providers.
Security News
Tea.xyz, a crypto project aimed at rewarding open source contributions, is once again facing backlash due to an influx of spam packages flooding public package registries.
Security News
As cyber threats become more autonomous, AI-powered defenses are crucial for businesses to stay ahead of attackers who can exploit software vulnerabilities at scale.