2

The company where I work needs to choose one of the available NLP API's providers. For the POC I've integrated Api.Ai and I must say it works really good. Console UI is smooth and easy to work with. The same goes for REST API's.

Though I have to check and compare also other providers (Luis, Wit.ai, etc.)

Is there any recent "study" how the major AI providers compare nowadays (support, quality, etc.)? Everything info available I've found (about comparisons) was kind of outdated.

I've checked Microsoft's Luis -> looks like a copy (UI, etc.) of Api.Ai.

People are saying Wit.ai (Facebook) console is a mess to work with, etc.

There are +50 NLP API providers out there - it's hard to check every single one of them.

I would be very thankful if anyone has some recent comparison-infos about major NLP providers.

EDIT: We have decided for API.AI (DialogFlow since Google purchased it). I must say DialogFlow works amazingly. Latest V2 API works over gRPC.

sabiland
  • 2,526
  • 1
  • 25
  • 24
  • 1
    Interesting question, but it may be closed due as it's not the purpose of StackOverflow – Nicolas R Jun 12 '17 at 15:47
  • 1
    Possible duplicate of [Comparison between luis.ai vs api.ai vs wit.ai?](https://stackoverflow.com/questions/37215188/comparison-between-luis-ai-vs-api-ai-vs-wit-ai) – Ezequiel Jadib Jun 12 '17 at 16:23
  • Yes I have read "Comparisons..." but most of the comments are +1year old & there is almost zero information about console UI (quality), support, maintenance, work-flow, etc. – sabiland Jun 13 '17 at 05:20
  • You have a good subject to make an article :) – Nicolas R Jun 13 '17 at 07:50
  • After 3 years in production, I can say about DialogFlow that it performs fantastically. Superb web-portal for configuring intents + fast gRpc API. – sabiland Mar 15 '21 at 09:37

1 Answers1

2

We have recently published an evaluation study of seven NLU API-enabled services: API.ai, Amazon Lex, Microsoft LUIS, IBM Watson Conversation, wit.ai, Recast.ai or Snips.ai.

Briefly speaking:

  • IBM Watson intent detection is the best one, especially on smaller training datasets (although when trained on over 2000 samples the difference is indistinguishable).
  • API.AI is free, the performance on big enough training set matches IBM Watson and Microsoft LUIS.
  • Microsoft LUIS works significantly faster than others in our tests.
  • wit.ai has somewhat worse performance and response time than the three above, but it’s free and it provides the best language coverage (some 50 languages).
  • Amazon Lex has quite strict API limits (the training set size is limited to 200K symbols, which may be insufficient to reach a good intent detection quality for a multi-intent assistant; also it requires all training utterances to be labeled by entities, which complicated preparation of the dataset.
savenkov
  • 658
  • 8
  • 13