June 22nd, 2020
Demonstration of TED Policy in Rasa Dialogue Management
Vincent Warmerdam
This guide accompanies a video on our algorithm whiteboard playlist. You can see the video below, but we figured that having a written guide would make it easier for our community members to reproduce our experiment.
The goal of this exercise will be to build a chatbot that is robust against interruption such that we can see how TED is able to handle the interrupt.
The Task
If you want to follow along you'll need to clone this github repository. The repository contains a Rasa project with a very particular set of actions. Here's a snippet from the stories.md file.
## interrupt path 1
* greet
- utter_greet
* start_count
- utter_10_countdown
* confirm
- utter_9_countdown
* confirm
- utter_8_countdown
* confirm
- utter_7_countdown
* confirm
- utter_6_countdown
* bot_challenge
- utter_iamabot
- utter_5_countdown
* confirm
- utter_4_countdown
* confirm
- utter_3_countdown
* confirm
- utter_2_countdown
* confirm
- utter_1_countdown
* confirm
- utter_0_countdown
* goodbye
- utter_goodbye
The goal of this chatbot is to be able to perform a count-down. Once the countdown starts the user needs to confirm before the countdown can continue. This also allows the user to interrupt the countdown. If the user asks the chatbot "are you a human?" the chatbot needs to respond appropriately but we'd also like it to continue the countdown immedaitely afterwards. To do this appropriately we'll need to have a system that does more than just map the intent to an action, we'll also need to include the history of the conversation.
We have more examples of stories like this one but only a few. The hope is that just a few stories are enough to have an algorithm generalise.
Dialogue Understanding
Taking the history into account is exactly what our TED Policy is good at, there's even a hyperparameter that we can specify that will tell us how far the algorithm will look back. If we zoom in on the policies section in our config.yml file then we can see it explicitly.
policies:
- name: MemoizationPolicy
- name: TEDPolicy
epochs: 200
max_history: 5
- name: MappingPolicy
This configuration file gives us an opportunity to try out different settings for max_history to see what the effect is in the conversation. We'll not run a quantitative benchmark though. Instead we'll just look at the types of mistakes that the assistant makes given a configuarion. We'll change the configuration, run rasa train and then check what responses we get via rasa shell.
TED with max_history=1
Here's a snippet of conversation given that we only have a lookback of 1 dialogue step.
Your input -> start counting
Countdown! ETA 10 🎵
Your input -> ok
Countdown! ETA 5 🎵
Your input -> ok
Countdown! ETA 5 🎵
Your input -> ok
Countdown! ETA 5 🎵
Your input -> ok
Countdown! ETA 5 🎵
Not a great experience. Let's increase the history.
TED with max_history=3
We'll repeat the exercise but now we allow TED to look back three timesteps.
Your input -> count
Countdown! ETA 10 🎵
Your input -> ok
Countdown! ETA 9 🎵
Your input -> ok
Countdown! ETA 8 🎵
Your input -> ok
Countdown! ETA 7 🎵
Your input -> are you a bot?
I am a bot, not a human, powered by Rasa.
Countdown! ETA 6 🎵
Your input -> ok
Countdown! ETA 5 🎵
Your input -> are you a bot?
I am a bot, not a human, powered by Rasa.
Your input ->
Countdown! ETA 10 🎵
The conversation is a whole lot better, but we seem to be able to fool it when we interrupt. It seems to correctly utter that it is not a human but it is not consistently recovering the countdown.
TED with max_history=10
We'll repeat the exercise but now we allow TED to look back ten timesteps.
Your input -> count
Countdown! ETA 10 🎵
Your input -> ok
Countdown! ETA 9 🎵
Your input -> ok
Countdown! ETA 8 🎵
Your input -> are you a bot?
I am a bot, not a human, powered by Rasa.
Countdown! ETA 7 🎵
Your input -> ok
Countdown! ETA 6 🎵
Your input -> are you a bot?
I am a bot, not a human, powered by Rasa.
Countdown! ETA 5 🎵
Your input -> ok
Countdown! ETA 4 🎵
Your input -> are you a bot?
I am a bot, not a human, powered by Rasa.
Countdown! ETA 3 🎵
Your input -> ok
Countdown! ETA 2 🎵
Your input -> are you a bot?
I am a bot, not a human, powered by Rasa.
Countdown! ETA 1 🎵
Your input -> ok
End of countdown!
This is a whole lot better! It can count down properly even when we interrupt it.
Conclusion
It deserves mentioning that our usecase here is a bit artificial. Odds are that being able to count down with interruptions is not the use-case that you or your company will have in mind. That's why we recommend experimenting with this max_history size yourself on your own projects.
That said, this exercise does highlight that you can allow your digital assistant to take more information into account by allowing it to look back more. So feel free to experiment with this setting to see if it makes an improvement.
Happy hacking!