Understanding Bot Breaks, and How to Handle Them

This article has been authored by Swapan Rajdev, Co-Founder & CTO of Haptik

One of the primary factors behind the growing appeal of Conversational AI is its ability to converse with people naturally. A lot of people expect to be able to speak to a chatbot or virtual assistant the same way they would with friends, family or co-workers. In an enterprise context, the expectation is that a customer should be able to count on the assistance of a virtual assistant, just as they would rely on an in-store sales clerk or customer service representative.

The technology has evolved to a point where such seamless customer interactions have become possible to a large extent. But it is not perfect. Sometimes the assistant is simply unable to respond to a particular user query with the correct reply, or unable to complete the task requested by the user – and is  therefore unable to proceed with the conversation. As a result, a message is sent to the user, informing them of this break in the conversation. This can be referred to as a fallback message or bot break.

Bot breaks are, to a certain extent, inevitable. After all, virtual assistants cannot, at present, be trained to respond to every possible user input. Enterprise virtual assistants in particular need to restrict conversations to the scope of their specific domains, and bot breaks serve as a mechanism to close conversations involving irrelevant inputs and/or out-of-scope queries.

If bot breaks are handled correctly, they significantly improve user experience. But if not handled correctly, especially in an enterprise context, they have the potential to negatively impact a customer’s perception of the brand.

The good news is that there are a number of ways to effectively handle bot breaks and minimize any potential negative impact. At Haptik, we aim to build Intelligent Virtual Assistant (IVA) solutions that offer the best possible customer experience. To that end, we take several measures, preemptively at the design & development stage, as well as post-implementation, to either a) anticipate and prevent situations that lead to bot breaks, or b) identify common causes of bot breaks and take corrective measures to prevent their recurrence, or to manage the bot break as smoothly as possible.

But before we delve into that, let us take a step back and get a better understanding of precisely what constitutes a bot break, and what causes them.

Understanding Bot Breaks

A ‘bot break’ occurs when a chatbot or virtual assistant is not trained to respond to a particular user input. This, in turn, leads to a premature break in the conversational flow.

There are a number of factors that can cause a bot break. 

#1 “No Match” Error

A “No match” error is one of the most common causes of bot breaks. It occurs when the user asks the IVA something which it does not understand, or which is simply outside its scope.

#2 Ambiguity in User Input

Sometimes the IVA is unable to proceed further in the conversation due to ambiguity in the user input. This leaves the IVA confused between two or more different intents.  For instance, if the user is chatting with an IVA for an airline and simply enters the word Flight, the IVA might be confused about the specific intent of the user (for e.g. whether the user wants to book a flight, or has a query about a previously booked flight). As a result, it will be unable to proceed with the next step of the conversation, resulting in a bot break.

#3 Failure to Complete Task

Bot breaks are also caused by the IVA’s failure to complete the task a user requested. This can be due to various factors. The IVA may simply lack the capability to carry out the requested task, or the task may be beyond the intended scope of the solution. Alternatively, the IVA may be capable of carrying out the task, but the user does not qualify based on the input provided. 

For e.g. consider an IVA for an insurance brand. The IVA’s purpose is to answer customer queries around insurance policies. So if a user were to enquire about career opportunities with the insurance, the IVA will send a bot break message, as carrying out a hiring-related task is beyond its scope.

#4 User Inaction

This is a scenario which may or may not be considered a ‘bot break’ as such, but is nonetheless important to keep in mind. If a user does not respond to the IVA for a certain length of time, it’s a good practise to send a timed reminder or follow-up message to try to resume the conversation.At Haptik, our IVAs send a follow-up message after 8 minutes of user inactivity to check if the user still needs help. If there is no response, the conversation is automatically closed.

Why it’s important to handle bot breaks correctly

Brands implement IVA solutions in order to enhance customer experience by offering a seamless conversational interface that helps people get things done swiftly and accurately. If not handled properly, bot breaks undermine this goal.

A poorly designed bot break can hinder customers by stopping them from completing a task, or preventing them from resolving their queries without an explanation. This leads to a frustrating customer experience that can contribute to a negative perception of the brand.

At a more fundamental level, frequent bot breaks affect the perception of the IVA being ‘intelligent’. One of the key factors that distinguish Intelligent Virtual Assistants from traditional chatbots is their ability to go beyond a rigid ‘company script’ to have more free-flowing and ‘natural’ conversations with customers, truly understand their requirements and fulfill them with minimal friction. A poorly-handled bot break signals that the IVA is unable to handle such ‘off-script’ conversations.

Let us now explore methods of handling bot breaks, based on our own experience at Haptik. A lot of these methods are built into the Haptik platform and are available on new IVAs out-of-the-box. We also provide our enterprise partners with the option to customize these experiences, for e.g. by offering the flexibility to change the content of bot breaks messages.

Preventing Bot Breaks Before Going Live

As they say, “Prevention is better than cure” and Conversational AI is no exception to the old adage. 

To ensure that our Intelligent Virtual Assistants perform to a high standard from Day 1, we go through an intensive process of “solutioning” with our enterprise partners before the solution is implemented. During this time, we thoroughly understand our partner’s use cases and desired objectives for the solution, which enables us to start designing the most suitable conversational flows for the purpose. As part of this exercise, we also take into account the likely user inputs that would result in a bot break for this particular IVA, and build preventive measures into the conversational design to ensure that the bot break is handled correctly.

A key aspect of this process is identifying ‘error cases’ or ‘edge cases’ for a particular IVA, and then equipping the solution with suitable responses to deal with them. For instance, consider once again the example of the insurance brand’s IVA designed to answer queries around policies, term plans etc. There might be users who will ask the IVA questions regarding career opportunities, which is outside the purview of the solution. But if the IVA is trained to clarify to the user that it does not handle such queries, then a bot break is avoided. After all, a person who politely tells you that he/she cannot answer your question (or redirects you to a place where you can find the answers, as seen above) is a lot more pleasant than someone who freezes up and refuses to respond! The same holds true for virtual assistants.

Another step that can be taken is ensuring that the IVA has validations in place for vetting user input. For instance, suppose the conversational flow requires the user to enter a 10 digit cellphone number, and the user enters 9 digits instead. The IVA would point out the error to the user and allow them the chance to re-enter the number correctly.  Haptik’s technology has such validations built-in for a number of common expressions and scenarios.

Adding an autocomplete feature to the IVA is another great way to prevent bot breaks. The suggested responses that appear even as a user types will help them understand the purpose of the IVA. It will also discourage them from entering an out-of-scope query or incorrect input in freeform.

Last, but not least, our robust NLU (Natural Language Understanding) capabilities also help prevent bot breaks. We had discussed earlier how ambiguity in user input can confuse an IVA about a user’s intent. With the Disambiguation logic embedded in our NLU core, the IVA is able to ask the user follow-up questions to clarify their intent, enabling the conversation to continue.

Preventing Bot Breaks Post Implementation

Over time, the Haptik platform, not to mention our team, has become quite adept at pre-emptively tackling bot breaks and delivering a truly seamless conversational experience. But no system is 100% foolproof and there is always scope for improvement.

At Haptik, we pride ourselves on our undying commitment to customer success. To that end, we constantly work with our partners after a solution has gone live, to optimize the IVA’s performance or tackle any issues that may have slipped through the cracks.

Our Analytics Dashboard spares us the human effort of sifting through the conversational data and identifying the causes of bot breaks, instead offering a more efficient and sophisticated solution. The ‘Bot Break Topics’ features analyzes all the messages which resulted in bot breaks, and categorizes them into distinct topics that can be easily acted upon. We then take corrective measures to prevent future bot breaks from occurring within those particular conversational flows.

If the bot breaks are being caused by in-scope queries, the break messages can be designed in a manner that prompts the user to ask the query in the correct format, or guide them through a set of tasks that will help resolve the query. Thus, what would have been the premature end of the conversation instead becomes a query successfully resolved or a task successfully completed.

In the case of an out-of-scope query, an appropriate message will be sent clarifying the scope of the IVA. In the case of persistent out-of-scope queries, new conversational flows might be created to address those queries, in order to prevent the negative experience of a bot break

Haptik recently developed its own industry-first metric for evaluating the performance of Intelligent Virtual Assistants – the Intelligence Satisfaction Score (ISAT). Over a 6 month period, we were able to successfully optimize an IVA we developed for one of our partners, to raise its ISAT score from 36% to 76% (with a 10 point increase in NPS)! This massive improvement in performance can be attributed to a large extent to our ability to effectively identify the causes of bot breaks and nip them in the bud.

To sum up

It is important to be aware of bot breaks, what causes them, and how to prevent them. But it is equally important not to let bot breaks dampen our belief in the power of Conversational AI to transform the way people get things done.

As we discussed right at the start, bot breaks are simply representative of the reality that virtual assistants, particularly enterprise virtual assistants, cannot cover absolutely ‘everything’. In order to be able to focus on solving key business problems, it is necessary for IVAs to have some limitations on their scope. And since users have no limitations in terms of what they can potentially ask an IVA, the possibility of bot breaks is always present. 

In our role as Conversational AI leaders, we at Haptik aim to not only manage bot breaks properly, but use our learnings from them to design and develop even more seamless, engaging and effective conversational experiences for our partners, and their customers. I hope this article has given you a good insight into how we accomplish this each and every day.