1

Some context: Node.js, Bot, natural module.

I would like to build a Bot and I am using the natural module in order to parse and overall classify the user input.

var classifier = new natural.BayesClassifier();
classifier.addDocument('Hi', 'welcome');
classifier.addDocument('Hello', 'welcome');
classifier.addDocument('Hey', 'welcome');
classifier.addDocument('Good', 'welcome');
...
//back to home
classifier.addDocument('go back to home', 'back2home');
classifier.addDocument('go back home', 'back2home');
classifier.addDocument('return',  'back2home');
classifier.addDocument('return to home', 'back2home');
...
classifier.train();
...
classifier.classify(text);

Those tests work fine:

  "I would like to go back home" => back2home
  "Hi" => welcome

All good, but what if the user text contains something such as: "bla bla bla", I want to get a way to know that the that the text not fits enough in any of the above cases. "bla bla bla" returns me => welcome, but actually i would like it return something such "unknown"/not understood.

It is a way to "train" the classifier in such a way? Thanks.

Community
  • 1
  • 1
Vincenzo
  • 85
  • 12

1 Answers1

2

You can use the getClassifications() method to get a list of classifications and the associated score, or 'confidence', with it. From that list you can determine which, if any, it best matches. Ex:

console.log(classifier.getClassifications('blah blah blah'));

Output:

[ { label: 'welcome', value: 0.5 },
  { label: 'back2home', value: 0.5 } ]

This example isn't a great one but you can see that it doesn't match any one label very well. The higher the value the higher the confidence.

You can check the value of it to ensure that it is above a certain level. I like to use 0.8 as my check value. Loop through the results.

const results = classifier.getClassifications('blah blah blah');
let intents = [];

// Check for confidence greater than 8
results.forEach((result) => {
    if(result.value > 0.8) {
        intents.push(result);
    }
});

// Sort intents array by object.value
intents.sort((a,b) => {
    if(a.value < b.value) {
        return -1;
    }
    if(a.value > b.value) {
        return 1;
    }
    return 0;
});

You now have an array of intents with confidence greater than 0.8, sorted descending by their confidence score.

More information at https://github.com/NaturalNode/natural#classifiers
Credit for sorting function Sort array of objects by string property value in JavaScript

Community
  • 1
  • 1
Ding
  • 3,065
  • 1
  • 16
  • 27
  • it looks like the totals for this algorithm always will add up to 1.0, so if you you only have two categories you're forcing it to make a choice. Add more than N categories, and then a 1/N confidence threshold should give valuable results. Also note that natural.Bayes doesn't use word vectors so you won't get any semantic interpolation. – dcsan Nov 21 '20 at 20:03