Intuit Smart Assistant
This is the tale of a smart assistant that wanted to spend its days helping humans.
My role
As lead product designer, I designed, prototyped, and researched the whole experience, working with customers in the research lab.
We are teaching computers to talk to humans, not the other way around.
— Conversational UI team, Google
The Challenge
While working with the QuickBooks team at Intuit, I was invited to participate in a Design Thinking workshop to reimagine the future of Customer Care. The goal was to address these questions:
How might we make customer support advisory and consultative?
How might we make customer support available any time or place?
The company wanted to make a bold, innovative leap.
Conversational UI was the highest voted concept
In the workshop, we formed small teams, brainstormed ideas, and then narrowed them. The team I joined chose 3 ideas to test with customers:
A QuickBooks personal advisor, to handle your needs and connect you within the company
Video chat, by web and smartphone
A conversational UI you can use as a messaging app on your smartphone
The sketches were quick, rough, 5-minute concepts that we showed to customers in 10-minute feedback sessions. Sometimes rough sketches that can be iterated on rapidly are better than beautiful ones.
(These are not the actual sketches, to respect NDAs, but you get the idea.)
The customer feedback was:
The personal advisor was not appealing (rated 3-4 of 10). Customers wanted to feel independent.
The video chat was moderately appealing (rated 5-6 of 10). Customers felt self-conscious and concerned about mobile bandwidth.
The chat messaging was hugely appealing (rated 9-10 of 10). Customers loved quick chat messages, especially when the messages were proactively helpful.
Within a few weeks, I was asked to design and prototype a chatbot for Facebook Messenger for the Intuit QuickBooks Facebook page.
We analyzed data about customer messages
To get started with the chatbot, I wanted a better understanding of the customers who contact Intuit through Facebook. I worked with Intuit's social media team to uncover data about the messaging contacts for the previous month. This is the first part of a huge spreadsheet of data (with names redacted, for privacy).
Six clear personas emerged
The team received about 1700 messages during May 2017. About 1200 were for technical questions on QuickBooks products, about evenly split between QuickBooks Online and QuickBooks Desktop.
Next, I met with a team of 3 others (in total, two product designers and two product managers) to analyze the data and find persona patterns.
We discovered six personas, which we verified with the social media team:
First Timers, visiting for the first time and looking around
Info Seekers, asking technical questions about their QuickBooks product
Last Chancers, frustrated customers giving the company one last chance
Past Last Chancers, past their last chance and threatening legal action
Global Citizens, from other countries who want to talk in their language
Outage Affected, needing an online outage resolved
Most customer requests came from mobile
We also found that 75% of traffic to the QuickBooks Facebook page was from mobile devices. The chatbot needed to be effective on mobile. About half the time during chat, users have the onscreen keyboard open, so they're interacting with the chatbot in the top half of their mobile screen.
So, I made rapid prototypes of conversations, in ChatFuel
I realized that to concept the chatbot and get customer feedback, I needed a fully interactive prototype. I chose ChatFuel to build that prototype. ChatFuel is an intuitive, drag-and-drop chatbot builder tool for Facebook Messenger.
You can build an entire chatbot and launch it from ChatFuel. In my case, I built a prototype in ChatFuel, which was later converted to api.ai for launch, to use natural language processing.
I built my first test chatbot in about 10 minutes. Next, I quickly started prototyping a chatbot for the team, working with the use cases the product manager had defined for the MVP release:
15 How Do I topics for QuickBooks Online (described here)
3 inside sales use cases for QuickBooks Desktop
Creating quick prototypes in ChatFuel was incredibly fun and easy, and allowed me to really get at the heart of what customers wanted.
Showing the prototypes to customers revealed their reactions
I then worked with customers in the design lab, in 30-minute sessions. I showed them chatbot features, and then asked them to tell me how they felt about the experiences, using simple emojis (green, yellow, and red). Because the emojis are easy to relate to, customers were willing to open up and share their feelings.
This is a summary of customer emotions I saw, across various chatbot features.
Customers didn’t want long articles or blocks of text
From the customer sessions, I learned very early that:
Customers are wary of chatbots, even in the tech-savvy San Francisco area
They're willing to use a chatbot if they can speak to a human at any time
They definitely don't want to read long articles, or be given links
We had to create an interaction model that worked well for Intuit’s customers. These are two design patterns that worked especially well for the How Do I questions.
First, the summary card (on the bottom row, center).
Customers loved having the chatbot perform actions for them
But there was something that our customers wanted even more: the ability to use QuickBooks Online by giving simple natural language commands, rather than its graphical interface.
This is another flow I showed customers, allowing them to easily create and send an invoice. (Here, assume the customer is securely logged in to their QuickBooks Online account; we definitely thought of that, and it's in plan for a future release.)
This personalized flow in the chatbot saves 15 clicks in the simplest QuickBooks Online product. Customers responded with comments like:
Wow!
Can you really do that?
I love that!
That's inspired
To understand the impact this might have on QuickBooks, keep in mind that Nir Eyal, author of Hooked, says:
When the effort required to accomplish an action decreases, usage tends to explode.
As a team, we recognized that this feature is essentially a new, disruptive interface to QuickBooks Online. This was out of scope for the MVP, but definitely planned for future releases.
Users sent the chatbot very human messages
With this customer feedback, it was time for testing sessions, where Intuit staff could play with the chatbot hands on, and we could learn from what they did. These were really fun jam sessions.
I set up testing sessions twice a week, for three weeks. We'd have 5-25 people attend, according to their schedules. As people used the chatbot, we asked them to log issues.
Surprisingly, in our first week of testing, one of the top 3 utterances users entered was I love you. Like this:
Over these three weeks, I made about 100 fixes to the chatbot prototype each week, including:
About 100 responses to human utterances (such as How are you? Are you bot or human?)
Answers to frustrated customers
Replies to marketing type questions about the product
It felt like the chatbot grew up about 10 years each week.
Users see the chatbot as a human friend
From live testing, I learned that:
Human conversations are very, very nuanced.
Users see your chatbot (or conversational UI) as a human friend.
They send emojis, stickers, images, animated GIFs, video, audio, and locations.
A given question can be expressed in many different ways. Your chatbot needs to be able to clarify the question and respond within context.
An answer can also be expressed in many different ways, depending on context.
All this work on interaction flows generated multiple conversation paths: a welcome message with multiple options, default answer with multiple options, How Do I conversations, and entry and exit points.
The conversation paths were shared with a content writer, who polished the voice and tone, according to brand guidelines.
We defined more use cases for the future
During the design process, I was asked to make regular video demos that were shown to Senior VPs and other executives. The chatbot became very popular within Intuit.
As I completed interaction design and customer testing for the MVP, the development team began converting my ChatFuel prototype to api.ai, a more sophisticated platform with natural language processing.
From the testing sessions, which emulated customer interaction, the future path became clear:
Add more How Do I questions and answers (instead of 15, the chatbot needs at least 5000, and eventually about 50,000)
Add conversation contexts, so the chatbot can respond appropriately when a question is asked at different conversation points
Add more sales use cases (this will be described another post)
Meanwhile, my friends ...
