trigram-utils

trigram-utils

A few language trigram utilities.

Contents

• What is this?
• When should I use this?
• Install
• Use
• API
  • clean(value)
  • trigrams(value)
  • asDictionary(value)
  • asTuples(value)
  • tuplesAsDictionary(tuples)
• Types
• Compatibility
• Security
• Related
• Contribute
• License

What is this?

This package contains a few utilities that can help when working with trigram (an n-gram where each slice is 3 characters) based natural language detection.

When should I use this?

Probably not often, except when you want to create something like franc, but build it in something other than UDHR.

Install

This package is ESM only. In Node.js (version 12.20+, 14.14+, or 16.0+), install with npm:

npm install trigram-utils

In Deno with Skypack:

import * as trigramUtils from 'https://cdn.skypack.dev/trigram-utils@2?dts'

In browsers with Skypack:

<script type="module">
  import * as trigramUtils from 'https://cdn.skypack.dev/trigram-utils@2?min'
</script>

Use

import {clean, trigrams, asDictionary, asTuples, tuplesAsDictionary} from 'trigram-utils'

clean(' t@rololol ') // => 't rololol'

trigrams(' t@rololol ')
// => [' t ', 't r', ' ro', 'rol', 'olo', 'lol', 'olo', 'lol', 'ol ']

asDictionary(' t@rololol ')
// => {'ol ': 1, lol: 2, olo: 2, rol: 1, ' ro': 1, 't r': 1, ' t ': 1}

const tuples = asTuples(' t@rololol ')
// => [
//   ['ol ', 1],
//   ['rol', 1],
//   [' ro', 1],
//   ['t r', 1],
//   [' t ', 1],
//   ['lol', 2],
//   ['olo', 2]
// ]

tuplesAsDictionary(tuples)
// => {olo: 2, lol: 2, ' t ': 1, 't r': 1, ' ro': 1, rol: 1, 'ol ': 1}

API

This package exports the following identifiers: clean, trigrams, asDictionary, asTuples, tuplesAsDictionary. There is no default export.

clean(value)

Clean a value (string). Strips some (for language detection) useless punctuation, symbols, and numbers. Collapses white space, trims, and lowercases.

trigrams(value)

From a value (string), make clean, padded trigrams (see n-gram) (Array<string>).

asDictionary(value)

From a value (string), get clean trigrams as a dictionary (Record<string, number>): keys are trigrams, values are occurrence counts.

asTuples(value)

From a value (string), get clean trigrams with occurrence counts as a tuple (Array<[string, number]>): first index (0) the trigram, second (1) the occurrence count.

tuplesAsDictionary(tuples)

Turn trigram tuples (Array<[string, number]>) into a dictionary (Record<string, number>).

Types

This package is fully typed with TypeScript. It exports an additional Gemoji type that models its respective interface.

Compatibility

This package is at least compatible with all maintained versions of Node.js. As of now, that is Node.js 12.20+, 14.14+, and 16.0+. It also works in Deno and modern browsers.

Security

This package is safe.

Related

• words/trigrams — trigrams for 400+ languages based on UDHR
• words/n-gram — get n-grams from text
• wooorm/franc — natural language detection

Contribute

Yes please! See How to Contribute to Open Source.

License

MIT © Titus Wormer