An Introductory Guide to Chatbots
What is all the commotion around chatbots this year? Are chatbots just a fad or are they here to stay? And if they are, what can they do for you? We shall try to answer some of those questions and more.
Let’s start with a little taste of what chatbots can do. The CNN chatbot helps users retrieve news. You tell the CNN bot what you are interested in, be it the summer Olympics or US presidential election and it will happily oblige with a selection of news stories. Disney keeps releasing bots along with its movies, the Zootopia bot, for instance, lets you discover and solve crimes. You can talk to the Miss Piggy bot to know more about her! The Hyatt hotels bot can talk to you about your reservation status, check-in times, available facilities and price. Hello Jarvis bot is a personal assistant and Healthtap answers questions about medical issues (yikes!). There are chatbots to talk to about weather (Poncho), stock markets (Finance bot), shopping (Nordstrom), travel (Kayak) and almost anything under the sun. You can even use a Chatbot for random chats to create stories or just to kill time.
This is just the tip of the iceberg though. The above examples are only a few of the chatbots hosted on one messenger, Facebook. Along with thousands of bots already on Facebook, scores are added daily and new messengers are adding the capability.
The sheer quantity of bots available today is proof that businesses see tremendous potential in them.
How did it all start?
Some of the earliest Chatbot like software were ELIZA and SmarterChild. Eliza was coded with a therapist like chat response, asking users how they felt about what they were saying. Moving forward, SmarterChild was the first chat program created in the spirit of today’s chatbots. It showcased the possibility of accessing database information through a messenger interface. Chris Messina is credited with contributing a great deal to the Chatbot boom. He is currently the Developer Experience Lead at Uber. His value proposition is simple: make Uber available on all platforms so users can book a car on the medium of their choice. Although it seems like the Chatbot story is beginning now, the Chatbot story has come a long way starting from ELIZA to the modern day conversational interfaces like Siri and Cortana.
What is a good bot?
Just like the name indicates, a bot chats with people. Most people are comfortable chatting with other humans through messenger services. Thus, the best bot is the one that behaves most like a human. A commonly accepted benchmark test for bots is the Turing test, where there are 3 players. The first player talks with 2 other players, one of which is a human and the other a chatbot. Within 5 minutes, the first player must choose between them and say which was the human and which was the bot. Alan Turing dictated that a good chatbot is said to receive the accolade of being human 70% of the time.
Why now?
Messenger services are beginning to look like the next generation UI, replacing apps and websites, existing giants are responding. To shore up its position in the messenger environment, Facebook recently dropped $22 billion to acquire WhatsApp. With the incredible amount of money spent, we can assume that Facebook is seeing enormous potential in the industry.
We always hear those millennials are the social media generation. Yet, in recent years their addiction to messaging apps has far surpassed their addiction to social media. Business Insider Intelligence reports, the top 4 messaging apps have turned the tables on the top 4 social media platforms in recent years and have become the more popular of the two with smartphone users. As we grow forward the gap between social media and messenger service adoption will continue to grow.
Since people spend so much time in messaging apps, these apps are seen as excellent mediums to advertise and as of late, for doing business and providing customer support. The time of the chatbot has well and truly come!
The science behind the Chatbot
A Chatbot is a computer program that behaves like a human, that is a classic definition of artificial intelligence. That classic definition is actually an oversimplification as there are a couple of AI technologies at the heart of the chatbot:
Natural Language Processing (NLP)
While natural language processing covers a plethora of topics, we can understand it best with the analogy of a person learning a new language. A person who has just been introduced to a language can understand the meaning of words they know when they read or hear them. Similarly, a Chatbot understands only the words that are fed into its lexicon. While feeding the entire English dictionary into a Chatbot program is very easy, telling it how different words can combine to make sentences is much trickier. Making a machine learn grammar is even tougher (not that it’s easy for humans to learn grammar!). A bot, unlike humans, can’t understand different contextual usages of a word if they are not programmed into it. Natural language processing is the name given to the ability of computer programs to make sense of common written and spoken language. The better a program understands sentences, the higher the quality of the replies it can provide to the user.
ne of the best examples of NLP today is its application in Google Search. Type something like, “How is the weather in Seattle” in Google and Google will provide the weather information in a nice card at the top of the page. Type, “what are the chances of rain in Seattle” and it will give you the same result. This is an excellent example of natural language processing in practice. The NLP programming of the Google bot is mature enough to understand that although the question and wording are slightly different, both cases require it to pull up the same set of information.
To achieve good NLP, chatbots are fed all thinkable variations of the frequently asked questions and answers manually. Algorithms are then written to help the bot fit in synonyms and scope is left for fuzzy matching as people generally ask the same questions in a number of ways.
Machine Learning (ML)
Machine learning is the ability of computers to learn by themselves without any hard coding. It’s worth mentioning that it is likely the vast majority of chatbots on the market today are not using machine learning. Most are using a retrieval based model where possible questions and answers are hard-coded into a flowchart-like structure. Thus most of today’s chatbots are not really learning to answer questions themselves using machine learning but are still learning to understand questions better with the help of NLP. For chatbots that are using the machine learning like IBM’s Watson or Google Assistant, the difference is they have a lot more data to work with and train. Some chat bot today and those developed in the near future will use a process called “supervised learning” for machine learning.
To put it into simple terms, answers given by the bots for different questions are rated on accuracy, usually by a human operator and fed back to the bot. The bot thus keeps learning about the right and wrong answers to questions and continuously keeps improving its answers. It stores the correct answers in its memory and rejects the wrong ones.
Machine learning is perhaps the most important aspect of Chatbots, without machine learning the bot will remain as smart or as stupid as it was initially until its code is changed manually.
Sentiment Analysis
Although machine learning is essential for a chatbot to expand its response lexicon over time and NLP assists in understanding the questions and statements entered by the user, these two alone are not enough for a chatbot to have full contextual understanding in a conversation.
For anyone that has worked any length of time in customer service, the way you respond to customers depends vastly on their emotional state at the time. We live in an age where consumers can easily air their grievances with companies over public channels, so understanding when customers are overheating and offering that extra concession is essential to successfully managing customer relationships.
While NLP takes a sentence and breaks it down into codified elements to allow the chatbot to understand the content, sentiment analysis seeks to rate the context of a sentence or series of sentences on an emotional scale from perfectly positive to perfectly negative. When a user adds the sentence, everyone is going to hear about the kind of service your company has provided me, it is of no consequence in NLP. However, sentiment analysis will take the statement into context with the preceding conversation and decide whether to respond with thank you or go into damage control mode.
Testing
Testing is a cornerstone of any application development, and the same goes for the post-app era. No matter where you intend to use chatbots, testing is a must have before any chatbot can be launched in its live version. Intents in a chatbot without machine learning could very well do the job, if intents had enough incumbent messages that matched the sample data given to the intent, and if you didn’t require to remember past data given to the bot. But in all likelihood, your objectives will require some machine learning, as human input will come in ways you may not intent, spelling mistakes, and data that if not remembered by the bot earlier in the conversation, could very well break up the flow of the conversation if the human user needs to recite past data. This would put them in a conversational loop that would be very difficult to get out of without forcing a change of conversation topic from the user’s side.
Testing includes 3 main methods after you feel satisfied with your current intents, which are:
Training
Confidence Level Adjustment
Conversational Testing
To really use a chatbot to meet an objective, you could, of
course, build your own, or hire someone else to do it for you. But there are pros and cons for each method. If you build it yourself, the NLP you make, or the third party NLP you use, will require testing modules to allow you to train the bot. If you’ve hired a third party to build your chatbot for you, ask them beforehand what their testing methods include. Any real chatbot agency or contract developer would tell you about the training of the bot in sample data intent, and ML Confidence Levels.
