A machine learning framework for Node.js. Supports multi-level classification and online learning.
Limdu is a machine-learning framework for Node.js. It supports multi-label classification, online learning, and real-time classification. Therefore, it is especially suited for natural language understanding in dialog systems and chat-bots.
Limdu is in an "alpha" state - some parts are working (see this readme), but some parts are missing or not tested. Contributions are welcome.
Limdu currently runs on Node.js 0.12 and later versions.
npm install limdu
You can run the demos from this project: limdu-demo.
Table of Contents generated with DocToc
var limdu = require('limdu');
var colorClassifier = new limdu.classifiers.NeuralNetwork();
colorClassifier.trainBatch([
{input: { r: 0.03, g: 0.7, b: 0.5 }, output: 0}, // black
{input: { r: 0.16, g: 0.09, b: 0.2 }, output: 1}, // white
{input: { r: 0.5, g: 0.5, b: 1.0 }, output: 1} // white
]);
console.log(colorClassifier.classify({ r: 1, g: 0.4, b: 0 })); // 0.99 - almost white
Credit: this example uses brain.js, by Heather Arthur.
var birdClassifier = new limdu.classifiers.Winnow({
default_positive_weight: 1,
default_negative_weight: 1,
threshold: 0
});
birdClassifier.trainOnline({'wings': 1, 'flight': 1, 'beak': 1, 'eagle': 1}, 1); // eagle is a bird (1)
birdClassifier.trainOnline({'wings': 0, 'flight': 0, 'beak': 0, 'dog': 1}, 0); // dog is not a bird (0)
console.dir(birdClassifier.classify({'wings': 1, 'flight': 0, 'beak': 0.5, 'penguin':1})); // initially, penguin is mistakenly classified as 0 - "not a bird"
console.dir(birdClassifier.classify({'wings': 1, 'flight': 0, 'beak': 0.5, 'penguin':1}, /*explanation level=*/4)); // why? because it does not fly.
birdClassifier.trainOnline({'wings': 1, 'flight': 0, 'beak': 1, 'penguin':1}, 1); // learn that penguin is a bird, although it doesn't fly
birdClassifier.trainOnline({'wings': 0, 'flight': 1, 'beak': 0, 'bat': 1}, 0); // learn that bat is not a bird, although it does fly
console.dir(birdClassifier.classify({'wings': 1, 'flight': 0, 'beak': 1, 'chicken': 1})); // now, chicken is correctly classified as a bird, although it does not fly.
console.dir(birdClassifier.classify({'wings': 1, 'flight': 0, 'beak': 1, 'chicken': 1}, /*explanation level=*/4)); // why? because it has wings and beak.
Credit: this example uses Modified Balanced Margin Winnow (Carvalho and Cohen, 2006).
The "explanation" feature is explained below.
Using Javascript's binding capabilities, it is possible to create custom classes, which are made of existing classes and pre-specified parameters:
var MyWinnow = limdu.classifiers.Winnow.bind(0, {
default_positive_weight: 1,
default_negative_weight: 1,
threshold: 0
});
var birdClassifier = new MyWinnow();
...
// continue as above
Some classifiers can return "explanations" - additional information that explains how the classification result has been derived:
var colorClassifier = new limdu.classifiers.Bayesian();
colorClassifier.trainBatch([
{input: { r: 0.03, g: 0.7, b: 0.5 }, output: 'black'},
{input: { r: 0.16, g: 0.09, b: 0.2 }, output: 'white'},
{input: { r: 0.5, g: 0.5, b: 1.0 }, output: 'white'},
]);
console.log(colorClassifier.classify({ r: 1, g: 0.4, b: 0 },
/* explanation level = */1));
Credit: this example uses code from classifier.js, by Heather Arthur.
The explanation feature is experimental and is supported differently for different classifiers. For example, for the Bayesian classifier it returns the probabilities for each category:
{ classes: 'white',
explanation: [ 'white: 0.0621402182289608', 'black: 0.031460948468170505' ] }
While for the winnow classifier it returns the relevance (feature-value times feature-weight) for each feature:
{ classification: 1,
explanation: [ 'bias+1.12', 'r+1.08', 'g+0.25', 'b+0.00' ] }
WARNING: The internal format of the explanations might change without notice. The explanations should be used for presentation purposes only (and not, for example, for extracting the actual numbers).
In addition to Winnow and NeuralNetwork, version 0.2 includes the following binary classifiers:
This library is still under construction, and not all features work for all classifiers. For a full list of the features that do work, see the "test" folder.
In binary classification, the output is 0 or 1;
In multi-label classification, the output is a set of zero or more labels.
var MyWinnow = limdu.classifiers.Winnow.bind(0, {retrain_count: 10});
var intentClassifier = new limdu.classifiers.multilabel.BinaryRelevance({
binaryClassifierType: MyWinnow
});
intentClassifier.trainBatch([
{input: {I:1,want:1,an:1,apple:1}, output: "APPLE"},
{input: {I:1,want:1,a:1,banana:1}, output: "BANANA"},
{input: {I:1,want:1,chips:1}, output: "CHIPS"}
]);
console.dir(intentClassifier.classify({I:1,want:1,an:1,apple:1,and:1,a:1,banana:1})); // ['APPLE','BANANA']
In addition to BinaryRelevance, version 0.2 includes the following multi-label classifier types (see the multilabel folder):
This library is still under construction, and not all features work for all classifiers. For a full list of the features that do work, see the "test" folder.
// First, define our base classifier type (a multi-label classifier based on winnow):
var TextClassifier = limdu.classifiers.multilabel.BinaryRelevance.bind(0, {
binaryClassifierType: limdu.classifiers.Winnow.bind(0, {retrain_count: 10})
});
// Now define our feature extractor - a function that takes a sample and adds features to a given features set:
var WordExtractor = function(input, features) {
input.split(" ").forEach(function(word) {
features[word]=1;
});
};
// Initialize a classifier with the base classifier type and the feature extractor:
var intentClassifier = new limdu.classifiers.EnhancedClassifier({
classifierType: TextClassifier,
featureExtractor: WordExtractor
});
// Train and test:
intentClassifier.trainBatch([
{input: "I want an apple", output: "apl"},
{input: "I want a banana", output: "bnn"},
{input: "I want chips", output: "cps"},
]);
console.dir(intentClassifier.classify("I want an apple and a banana")); // ['apl','bnn']
console.dir(intentClassifier.classify("I WANT AN APPLE AND A BANANA")); // []
As you can see from the last example, by default feature extraction is case-sensitive. We will take care of this in the next example.
Instead of defining your own feature extractor, you can use those already bundled with limdu:
limdu.features.NGramsOfWords
limdu.features.NGramsOfLetters
limdu.features.HypernymExtractor
You can also make 'featureExtractor' an array of several feature extractors, that will be executed in the order you include them.
//Initialize a classifier with a feature extractor and a case normalizer:
intentClassifier = new limdu.classifiers.EnhancedClassifier({
classifierType: TextClassifier, // same as in previous example
normalizer: limdu.features.LowerCaseNormalizer,
featureExtractor: WordExtractor // same as in previous example
});
//Train and test:
intentClassifier.trainBatch([
{input: "I want an apple", output: "apl"},
{input: "I want a banana", output: "bnn"},
{input: "I want chips", output: "cps"},
]);
console.dir(intentClassifier.classify("I want an apple and a banana")); // ['apl','bnn']
console.dir(intentClassifier.classify("I WANT AN APPLE AND A BANANA")); // ['apl','bnn']
Of course you can use any other function as an input normalizer. For example, if you know how to write a spell-checker, you can create a normalizer that corrects typos in the input.
You can also make 'normalizer' an array of several normalizers. These will be executed in the order you include them.
This example uses the quadratic SVM implementation svm.js, by Andrej Karpathy. This SVM (like most SVM implementations) works with integer features, so we need a way to convert our string-based features to integers.
var limdu = require('limdu');
// First, define our base classifier type (a multi-label classifier based on svm.js):
var TextClassifier = limdu.classifiers.multilabel.BinaryRelevance.bind(0, {
binaryClassifierType: limdu.classifiers.SvmJs.bind(0, {C: 1.0})
});
// Initialize a classifier with a feature extractor and a lookup table:
var intentClassifier = new limdu.classifiers.EnhancedClassifier({
classifierType: TextClassifier,
featureExtractor: limdu.features.NGramsOfWords(1), // each word ("1-gram") is a feature
featureLookupTable: new limdu.features.FeatureLookupTable()
});
// Train and test:
intentClassifier.trainBatch([
{input: "I want an apple", output: "apl"},
{input: "I want a banana", output: "bnn"},
{input: "I want chips", output: "cps"},
]);
console.dir(intentClassifier.classify("I want an apple and a banana")); // ['apl','bnn']
The FeatureLookupTable takes care of the numbers, while you may continue to work with texts!
Say you want to train a classifier on your home computer, and use it on a remote server. To do this, you should somehow convert the trained classifier to a string, send the string to the remote server, and deserialize it there.
You can do this with the "serialization.js" package:
npm install serialization
On your home machine, do the following:
var serialize = require('serialization');
// First, define a function that creates a fresh (untrained) classifier.
// This code should be stand-alone - it should include all the 'require' statements
// required for creating the classifier.
function newClassifierFunction() {
var limdu = require('limdu');
var TextClassifier = limdu.classifiers.multilabel.BinaryRelevance.bind(0, {
binaryClassifierType: limdu.classifiers.Winnow.bind(0, {retrain_count: 10})
});
var WordExtractor = function(input, features) {
input.split(" ").forEach(function(word) {
features[word]=1;
});
};
// Initialize a classifier with a feature extractor:
return new limdu.classifiers.EnhancedClassifier({
classifierType: TextClassifier,
featureExtractor: WordExtractor,
pastTrainingSamples: [], // to enable retraining
});
}
// Use the above function for creating a new classifier:
var intentClassifier = newClassifierFunction();
// Train and test:
var dataset = [
{input: "I want an apple", output: "apl"},
{input: "I want a banana", output: "bnn"},
{input: "I want chips", output: "cps"},
];
intentClassifier.trainBatch(dataset);
console.log("Original classifier:");
intentClassifier.classifyAndLog("I want an apple and a banana"); // ['apl','bnn']
intentClassifier.trainOnline("I want a doughnut", "dnt");
intentClassifier.classifyAndLog("I want chips and a doughnut"); // ['cps','dnt']
intentClassifier.retrain();
intentClassifier.classifyAndLog("I want an apple and a banana"); // ['apl','bnn']
intentClassifier.classifyAndLog("I want chips and a doughnut"); // ['cps','dnt']
// Serialize the classifier (convert it to a string)
var intentClassifierString = serialize.toString(intentClassifier, newClassifierFunction);
// Save the string to a file, and send it to a remote server.
On the remote server, do the following:
// retrieve the string from a file and then:
var intentClassifierCopy = serialize.fromString(intentClassifierString, __dirname);
console.log("Deserialized classifier:");
intentClassifierCopy.classifyAndLog("I want an apple and a banana"); // ['apl','bnn']
intentClassifierCopy.classifyAndLog("I want chips and a doughnut"); // ['cps','dnt']
intentClassifierCopy.trainOnline("I want an elm tree", "elm");
intentClassifierCopy.classifyAndLog("I want doughnut and elm tree"); // ['dnt','elm']
CAUTION: Serialization was not tested for all possible combinations of classifiers and enhancements. Test well before use!
// create a dataset with a lot of input-output pairs:
var dataset = [ ... ];
// Decide how many folds you want in your k-fold cross-validation:
var numOfFolds = 5;
// Define the type of classifier that you want to test:
var IntentClassifier = limdu.classifiers.EnhancedClassifier.bind(0, {
classifierType: limdu.classifiers.multilabel.BinaryRelevance.bind(0, {
binaryClassifierType: limdu.classifiers.Winnow.bind(0, {retrain_count: 10})
}),
featureExtractor: limdu.features.NGramsOfWords(1),
});
var microAverage = new limdu.utils.PrecisionRecall();
var macroAverage = new limdu.utils.PrecisionRecall();
limdu.utils.partitions.partitions(dataset, numOfFolds, function(trainSet, testSet) {
console.log("Training on "+trainSet.length+" samples, testing on "+testSet.length+" samples");
var classifier = new IntentClassifier();
classifier.trainBatch(trainSet);
limdu.utils.test(classifier, testSet, /* verbosity = */0,
microAverage, macroAverage);
});
macroAverage.calculateMacroAverageStats(numOfFolds);
console.log("\n\nMACRO AVERAGE:"); console.dir(macroAverage.fullStats());
microAverage.calculateStats();
console.log("\n\nMICRO AVERAGE:"); console.dir(microAverage.fullStats());
Use this option to get the list of all samples with a given class.
var intentClassifier = new limdu.classifiers.EnhancedClassifier({
classifierType: limdu.classifiers.multilabel.BinaryRelevance.bind(0, {
binaryClassifierType: limdu.classifiers.Winnow.bind(0, {retrain_count: 10})
}),
featureExtractor: limdu.features.NGramsOfWords(1),
pastTrainingSamples: [],
});
// Train and test:
intentClassifier.trainBatch([
{input: "I want an apple", output: "apl"},
{input: "I want a banana", output: "bnn"},
{input: "I really want an apple", output: "apl"},
{input: "I want a banana very much", output: "bnn"},
]);
console.dir(intentClassifier.backClassify("apl")); // [ 'I want an apple', 'I really want an apple' ]
The native svm.js implementation takes a lot of time to train - quadratic in the number of training samples. There are two common packages that can be trained in time linear in the number of training samples. They are:
The limdu.js package provides wrappers for these implementations. In order to use the wrappers, you must have the binary file used for training in your path, that is:
Once you have any one of these installed, you can use the corresponding classifier instead of any binary classifier used in the previous demos, as long as you have a feature-lookup-table. For example, with SvmPerf:
var intentClassifier = new limdu.classifiers.EnhancedClassifier({
classifierType: limdu.classifiers.multilabel.BinaryRelevance.bind(0, {
binaryClassifierType: limdu.classifiers.SvmPerf.bind(0, {
learn_args: "-c 20.0"
})
}),
featureExtractor: limdu.features.NGramsOfWords(1),
featureLookupTable: new limdu.features.FeatureLookupTable()
});
and similarly with SvmLinear.
See the files classifiers/svm/SvmPerf.js and classifiers/svm/SvmLinear.js for a documentation of the options.
Some advanced features are working but not documented yet. If you need any of them, open an issue and I will try to document them.
LGPL
Code contributions are welcome. Reasonable pull requests, with appropriate documentation and unit-tests, will be accepted.
Do you like limdu? Remember that you can star it :-)
FAQs
A machine learning framework for Node.js. Supports multi-level classification and online learning.
The npm package limdu receives a total of 195 weekly downloads. As such, limdu popularity was classified as not popular.
We found that limdu demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
In this segment of the Risky Business podcast, Feross Aboukhadijeh and Patrick Gray discuss the challenges of tracking malware discovered in open source softare.
Research
Security News
A threat actor's playbook for exploiting the npm ecosystem was exposed on the dark web, detailing how to build a blockchain-powered botnet.
Security News
NVD’s backlog surpasses 20,000 CVEs as analysis slows and NIST announces new system updates to address ongoing delays.