NLP is the backbone of any Conversational AI engine and the most frequently used decision-making factor in choosing a Digital Transformation partner. Intent Detection further plays a vital role in the NLU pipeline for any platform.
Haptik got recognized alongside Google Brain, Open AI and IBM Research at EMNLP 2020 for introducing three new datasets for the NLP community that has the potential to emerge as a major benchmark for Intent Detection globally. The paper also benchmarks Intent detection accuracy of Haptik along with Google’s DialogFlow, Microsft LUIS and Rasa in which Haptik’s performance emerges to be at par and in some cases better than the other tech giants.
This research paper has these 3 major contributions for the NLP community:
- Open-source 3 new datasets to accurately benchmark intent detection performance of task-oriented dialogue systems in the real world.
- Evaluation of 4 NLU platforms: Dialogflow, LUIS, Rasa NLU and Haptik on HINT3, highlighting gaps in language understanding and discussing tradeoffs between performance on in-scope vs out-of-scope queries.
- 'Subset Approach' for evaluation by extracting one representative sample for each natural 'class' of examples based on mutual entailment scores
For more insights on how HINT3 raises the bar for Intelligent Virtual Assistants, fill in the form and download the paper now!
