Huge News!Announcing our $40M Series B led by Abstract Ventures.Learn More
Socket
Sign inDemoInstall
Socket

retext-keywords

Package Overview
Dependencies
Maintainers
1
Versions
28
Alerts
File Explorer

Advanced tools

Socket logo

Install Socket

Detect and block malicious and high-risk dependencies

Install

retext-keywords

Keyword extraction with Retext

  • 0.0.1
  • Source
  • npm
  • Socket score

Version published
Weekly downloads
667
decreased by-48.05%
Maintainers
1
Weekly downloads
 
Created
Source

retext-keywords Build Status Coverage Status

browser support


Keyword extraction with Retext.

Installation

NPM:

$ npm install retext-keywords

Component.js:

$ component install wooorm/retext-keywords

Usage

var Retext = require('retext'),
    keywords = require('retext-keywords'),
    root;

var root = new Retext()
    .use(keywords)
    .parse(
        /* First three paragraphs on Term Extraction from Wikipedia:
         * http://en.wikipedia.org/wiki/Terminology_extraction */
        'Terminology mining, term extraction, term recognition, or ' +
        'glossary extraction, is a subtask of information extraction. ' +
        'The goal of terminology extraction is to automatically extract ' +
        'relevant terms from a given corpus.' +
        '\n\n' +
        'In the semantic web era, a growing number of communities and ' +
        'networked enterprises started to access and interoperate through ' +
        'the internet. Modeling these communities and their information ' +
        'needs is important for several web applications, like ' +
        'topic-driven web crawlers, web services, recommender systems, ' +
        'etc. The development of terminology extraction is essential to ' +
        'the language industry.' +
        '\n\n' +
        'One of the first steps to model the knowledge domain of a ' +
        'virtual community is to collect a vocabulary of domain-relevant ' +
        'terms, constituting the linguistic surface manifestation of ' +
        'domain concepts. Several methods to automatically extract ' +
        'technical terms from domain-specific document warehouses have ' +
        'been described in the literature.' +
        '\n\n' +
        'Typically, approaches to automatic term extraction make use of ' +
        'linguistic processors (part of speech tagging, phrase chunking) ' +
        'to extract terminological candidates, i.e. syntactically ' +
        'plausible terminological noun phrases, NPs (e.g. compounds ' +
        '"credit card", adjective-NPs "local tourist information office", ' +
        'and prepositional-NPs "board of directors" - in English, the ' +
        'first two constructs are the most frequent). Terminological ' +
        'entries are then filtered from the candidate list using ' +
        'statistical and machine learning methods. Once filtered, ' +
        'because of their low ambiguity and high specificity, these terms ' +
        'are particularly useful for conceptualizing a knowledge domain ' +
        'or for supporting the creation of a domain ontology. Furthermore, ' +
        'terminology extraction is a very useful starting point for ' +
        'semantic similarity, knowledge management, human translation ' +
        'and machine translation, etc.'
    );

root.keywords();
/*
 * Array[5]
 * ├─ 0: Object
 * |     ├─ stem: "terminolog"
 * |     ├─ score: 1
 * |     └─ nodes: Array[7]
 * ├─ 1: Object
 * |     ├─ stem: "term"
 * |     ├─ score: 1
 * |     └─ nodes: Array[7]
 * ├─ 2: Object
 * |     ├─ stem: "extract"
 * |     ├─ score: 1
 * |     └─ nodes: Array[7]
 * ├─ 3: Object
 * |     ├─ stem: "web"
 * |     ├─ score: 0.5714285714285714
 * |     └─ nodes: Array[4]
 * └─ 4: Object
 *       ├─ stem: "domain"
 *       ├─ score: 0.5714285714285714
 *       └─ nodes: Array[4]
*/

API

retext-keywords depends on the following plugins:

Parent#keywords({minimum=5}?)

Extract keywords, based on how many times they (nouns) occur in text.

// **See above for an example, and output.**

// Do not limit keyword-count.
root.keywords({'minimum' : Infinity});

Options:

  • minimum: Return at least (when possible) minimum keywords.

Results: An array, containing match-objects:

  • stem: The stem of the word (using retext-porter-stemm);
  • score: A value between 0 and (including) 1. the first match always has a score of 1;
  • nodes: An array containing all matched word nodes.

Parent#keyphrases({minimum=5}?)

Extract keywords, based on how many times they (nouns) occur in text.

// Do not limit phrase-count.
root.keywords({'minimum' : Infinity});

// Default values:
root.keyphrases();
/*
 * Array[6]
 * ├─ 0: Object
 * |     ├─ stems: Array[2]
 * |     |         ├─ 0: "terminolog"
 * |     |         └─ 1: "extract"
 * |     ├─ score: 1
 * |     └─ nodes: Array[3]
 * ├─ 1: Object
 * |     ├─ stems: Array[1]
 * |     |         └─ 0: "term"
 * |     ├─ score: 0.46153846153846156
 * |     └─ nodes: Array[3]
 * ├─ 2: Object
 * |     ├─ stems: Array[2]
 * |     |         ├─ 0: "term"
 * |     |         └─ 1: "extract"
 * |     ├─ score: 0.4444444444444444
 * |     └─ nodes: Array[2]
 * ├─ 3: Object
 * |     ├─ stems: Array[2]
 * |     |         ├─ 0: "knowledg"
 * |     |         └─ 1: "domain"
 * |     ├─ score: 0.20512820512820512
 * |     └─ nodes: Array[2]
 * └─ 5: Object
 *       ├─ stems: Array[1]
 *       |         └─ 0: "commun"
 *       ├─ score: 0.15384615384615385
 *       └─ nodes: Array[3]
*/

Options:

  • minimum: Return at least (when possible) minimum phrases.

Results: An array, containing match-objects:

  • stems: An array containing the stemms of all matched word nodes inside the phrase(s);
  • score: A value between 0 and (including) 1. the first match always has a score of 1;
  • nodes: An matrix containing array-phrases, each in turn containing word nodes.

Benchmark

Run the benchmark yourself:

$ npm run benchmark

On a MacBook Air, keywords() runs about 3,041 op/s on a section / small article.

            Finding keywords in English
 3,041 op/s » small (10 paragraphs, 20 sentences, 300 words)
   349 op/s » medium (100 paragraphs, 200 sentences, 3000 words)

            Finding keyphrases in English
   738 op/s » small (10 paragraphs, 20 sentences, 300 words)
    47 op/s » medium (100 paragraphs, 200 sentences, 3000 words)

License

MIT

Keywords

FAQs

Package last updated on 16 Jul 2014

Did you know?

Socket

Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.

Install

Related posts

SocketSocket SOC 2 Logo

Product

  • Package Alerts
  • Integrations
  • Docs
  • Pricing
  • FAQ
  • Roadmap
  • Changelog

Packages

npm

Stay in touch

Get open source security insights delivered straight into your inbox.


  • Terms
  • Privacy
  • Security

Made with ⚡️ by Socket Inc