cld3-asm

cld3-asm

cld3-asm is isomorphic javascript binding to google's compact language detector v3 based on WebAssembly cld3 binary. This module aims to provide thin, lightweight interface to cld3 without requiring native modules.

Install

npm install cld3-asm

Usage

Loading module asynchronously

cld3-asm relies on wasm binary (or asm.js where wasm is not supported) of cld3, which need to be initialized first.

import { loadModule } from 'cld3-asm';

const cldFactory = await loadModule();

loadModule loads wasm binary, initialize it, and returns factory function to create instance of cld3 language identifier.

loadModule(binaryEndpoint?: string): Promise<CldFactory>

It accepts binaryEndpoint as optional parameter for mainly browser environment. Unlike node, browser can't access wasm / asm binary directly in filesystem but have to fetch. Provide endpoints for paths to dist/src/lib/**/*.(wasm|mem) and it'll be fetched runtime. On node, this endpoint can be used to override physical path to binaries.

Creating language identifier

create(minBytes?: number, maxBytes?: number): LanguageIdentifier

LanguageIdentifier exposes minimal interfaces to cld3's NNetLanguageIdentifier.

• findLanguage(text: string): Readonly<LanguageResult> : Finds the most likely language for the given text.
• findMostFrequentLanguages(text: string, numLangs: number): Array<Readonly<LanguageResult>> : Splits the input text into spans based on the script, predicts a language for each span, and returns a vector storing the top num_langs most frequent languages
• dispose(): void : Destroy current instance of language identifier. It is important to note created instance will not be destroyed automatically.

There are simple examples for each environments. In each example directory do npm install && npm start.

Building / Testing

Few npm scripts are supported for build / test code.

• build: Transpiles code to ES5 commonjs to dist.
• test: Run cld / cld3-asm test both. Does not require build before execute test.
• test:cld: Run integration test for actual cld3 wasm binary, using cld's test case.
• test:cld3-asm: Run unit test against cld3-asm interface
• lint: Run lint over all codebases
• lint:staged: Run lint only for staged changes. This'll be executed automatically with precommit hook.
• commit: Commit wizard to write commit message

License

• cld3: original license
• cld3-asm: MIT

Changelog

0.0.1 (2017-08-27)

Bug Fixes

• findmostfrequentlanguages: update type definition (ecdf032)
• findmostfrequentlanguages: utf8 encode input (b7c0b48)

Features

• cld3: add initial cld3 binaries (3825273)
• cld3: bump up cld3 (398eb97)
• cldasmmodule: export asm module interfaces (9c2f119)
• cldfactory: export cldfactory interfaces (bb4e308)
• cldloader: implement cldloader (c1b061c)
• index: export loadModule (3435c69)
• index: setup index export (ea05fe5)
• loadmodule: implement loadmodule (1ead2ec)
• util: add utility implementation (cca4c79)