How Chatbot Metrics Influence Customer Service Outcomes

If you get a chatbot on your website, how will it affect your overall customer service performance? Do chatbots even improve customer service? What metrics are there to help you tie chatbots to your customer service outcomes?

A pressing concern when leveraging new technology for your team is to measure its performance as it affects your desired outcomes or KPIs in customer service. 

Let's choose three indicators of customer service performance, and we'll break down the metrics that matter under each of these qualitative outcomes.

Here's a handy way to navigate this article based on those three indicators:

• Customer Satisfaction
• Customer Service Team Capacity
• Customer Service Agent Satisfaction 
• Summary

Customer Satisfaction

Results of a Springer-published study by Chatura Ranaweera on customer satisfaction finds that it has a direct link with retention and word-of-mouth brand referral. Your CSAT performance is a significant revenue factor. 

Here's how chatbots contribute to and help you more accurately measure your customer satisfaction from a micro and macro perspective. 

Unanswered Messages

Chatbots don't answer 100% of customer inquiries, especially on a message level. When it comes to conversations however, it's not unusual for a chatbot to satisfy customers 60-80% of the time.

But individual messages and their responses give context to the overall customer experience during each particular chatbot touchpoint. 

While you can easily see a percentage of unanswered messages in your dashboard, it's also important to drill down particular conversations and review individual messages. You can examine specifically:

• what the customer wants
• how the chatbot responds
• and how it fits into the overall conversation

This, combined with message feedback and an end-of-conversation Net Promoter Score, gives you valuable insight about your customer service chatbot performance. 

It also helps you identify the specific pain-points your customers experience. Prioritise and take action to improve performance based on this data. 

Message Feedback

Many chatbots include a feature where customers can leave feedback on specific messages using a simple thumbs-up or down. 

This helps you zero in on underperforming chatbot replies. Use it with the contextual help of the conversation view to improve content in chatbot replies for specific intents, topics or questions.

It could simply be improving the copy of the reply, adding images, gifs or videos for more detailed guidance, or linking to a dedicated page on that topic. 

NPS and Conversation Feedback

Net Promoter Score is a short two-part survey, usually used at the end of a customer service conversation. It specifically quantifies customer satisfaction by asking if the customer is likely to recommend the product/service to their friends/colleagues. The second part is a qualitative free-form text, where the customer can give comments about their experience.

As you offer NPS surveys on more service touchpoints, like in chatbot conversations, you get a more accurate representation of average customer experience on a referral level. The other advantage of using NPS on multiple touchpoints is that you can pinpoint which individual services need extra attention to improve overall performance.

Further Reading: What is a Chatbot? Get the overview of defined chatbot types and how to measure their most common metrics.

Customer Service Team Capacity

Customer Service Team capacity is a significant factor in customer experience. A capacity planning solution for Amazon.com found over a million dollars in savings for their customer service department, partly from creating a model for diversifying messages answered by the team, and those tackled by other services. 

“These advancements in planning our capacity and optimizing our contact allocation plans have significantly improved our ability both to respond to customers quickly, which improves customer experience, and also to lower our costs, which increases corporate flexibility.” - Kim Rachmeler, Former VP of Worldwide Customer Service at Amazon.com

How can you measure increased capacity with chatbot implementation?

More Conversations Addressed by AI

How many more customers you serve in your conversational support. 

Any brand that implements chatbots can expect an increase in conversation volume capability. After all, an AI engine which talks to your customers with no regard for queues is going to be able to manage more conversations. 

Brands that extend service hours to 24/7 conversational support typically see their conversation volume in chat increase by about 40%. The metrics from all-day conversational support will quickly give you insight about the ideal time-of-day availability for your customers. 

When chatbots are only active during live chat service hours, an increase in conversation capacity is expected, about 10-15% (based on companies with around 300,000 annual customers).

Queue Length and First Response Time

Do customers wait more, or less time for service when you have a chatbot? 

As Kim Rachmeler says, reducing the amount of time a customer has to wait to speak to a representative has a positive impact on customer experience.

Chatbots are extremely efficient at answering frequent customer issues, and do so concurrently. So a chatbot can answer 100 customer questions in the same time a customer service advisor could answer one. 

However, this only works for frequent questions and automated processes you want handled by AI chatbots. The rest of the questions are still going to your human customer service agents. 

In general, we see around 60-90% of all chat issues solved by a chatbot, where 10-40% get transferred to a human agent, either through live chat or when the chatbot opens a ticket with your customer service ticketing system. 

Although chatbots are regularly measured against total chat conversations, it makes sense to look at the numbers across your entire customer service offering. 

• do you see fewer frequent issues in your phone or email support?
• is there a reduction in calls, email or form submissions as chat numbers increase?
• do your agents have more time to pursue extra activities that can help improve your overall customer service?

For example, Varma saved enough time per month (330 hours) to give two of their service agents new responsibilities. Here's what their Senior Vice-President of Pension Services and IT, Tiina Kurki had to say:

"Our GetJenny chatbot, Helmi, complements our customer service department, the quality of our telephone customer service has changed; common issues are reduced, while calls requiring human expertise are dominating.

Further Reading: Find out how an unemployment fund handled a 3x surge in customer requests while improving customer satisfaction.

Customer Service Agent Satisfaction

While there are many factors to agent satisfaction, from team dynamics to management quality, one of the key elements contributing to agent stress and turnover is repetitive and continuous human interaction coupled with strict quantitative targets (Keser & Yilmaz).

A significant detractor can be the quality of their interactions tied to a sense of accomplishment: 

"Agents report that the most satisfying part of the job is when the call ends in the customer's needs having been met. If the issue was a particularly challenging one, the emotional satisfaction of a job well done and a customer satisfied are factors that keep call center employees coming back for more of what can be a very stressful role."
- Susan Leah, "Increasing Customer Satisfaction Through Employee Satisfaction in a Call Center Environment" - Master's Research Paper, University of Wisconsin-Stout. 

What chatbot metrics can elevate agent satisfaction in the workplace? 

Deflection and Automation

When a chatbot works at the same time as agents (during live chat or phone) and it reduces the number of conversations they need to handle, it is normally called deflection. 

When the chatbot works outside of service hours and answers customer questions successfully, this is automation. 

AI chatbots generally deflect or automate from 60-90% of incoming customer inquiries. The number depends on:

• how good your chatbot is
• the types of question you allow your chatbot to answer
• industry, use case and complexity of your product

Deflection and Automation reduce the number of simple frequent questions agents encounter in their everyday work. It's not just that they no longer get the same question over and over and over again, it's mainly because they have extra time to concentrate on more complex issues customers would contact customer service about. 

Robotic Process Automation (RPA) Adoption

Reducing frequent issues is most often the first step in a strategy to improve customer service operations, whether in customer or agent satisfaction. It's a simple and fast way to reduce the volume of conversations that agents have to address. 

The next stage of this strategy may be to introduce RPA into your AI chatbot implementation. 

This is when you have a more complicated procedure than answering a simple question, but it still happens enough for it to become a chore for customer service agents. 

It could be something like relaying tracking and delivery information, informing a customer if a certain product is in stock, or cancelling an order for a customer. These don't just require "knowledge", they require the ability to perform "actions". Typically this sort of action might need to:

• authenticate the customer
• check a number of internal systems
• relay the information back to the customer
• take action based on the customer's need

When you successfully use RPA to automate these actions enough like A-Lehdet did, it removes the burden from human agents having to perform this action repeatedly. 

Create a Working Environment Where Knowledge and Critical Thinking is Valued

Deflection, Automation and Robotic Process Automation all feed directly into Susan Leah's point: that more challenging customer issues give agents a greater sense of accomplishment. 

It transforms a repetitive job into a career where subject matter expertise, critical thinking and empathy are valued. 

Further Reading: Do chatbots even work? Here's how 9 companies use chatbots and what they get out of it.

TL;DR

AI Chatbots can transform your customer service, internally and externally, significantly improving your customer and employee experience. 

Customer Satisfaction can be measured more frequently and improved with the help of chatbots. Bring these chatbot-provided metrics into your overall CSAT score improvement plans:

• Unanswered Messages
• Message Feedback
• NPS and Conversation Feedback

Customer Service Team Capacity planning is a challenging part of operations, and chatbots can boost your capacity while giving you data to plan your strategy into the future. Check these metrics and contrast them with your other channels:

• Conversation Count
• Queue Length and First Response Time

Agent Satisfaction is essential to a healthy, committed customer service team. Any brand that cares about customer experience should be promoting knowledge and skill development in their teams, reducing agent churn and creating an environment that builds confidence and celebrates achievement. After all, your front-line support is the face of your brand. Focus on improving these metrics to give your agents a more rewarding workplace:

• Deflection and Automation Rate
• Actions Automated by RPA

GetJenny helps brands connect with their customers 24/7 using AI chatbots tailored to improve customer experience. The user friendly approach handles all the technical stuff behind the scenes, while you can focus on serving customers at a whole new scale.

Customer service agents love JennyBot because it deflects the repetitive, tedious tasks and helps them do what they do best: give amazing service.

If you’d like to leverage AI chatbots to improve your customer service capacity and customer and agent satisfaction, talk to one of our chatbot experts first.