The call center numbers said the old system did better. Agents were busier than before, taking calls that came out of the new one. Staffing plans were blown, and the project was costing more than what it replaced.
I was brought in to advise. Sixty days later we’d beaten their original goal.
The first problem wasn’t in the system. Nobody had agreed on what “working” meant. So we started there: one primary metric, which calls counted toward it, and how to compare it with the old system’s numbers. The metric that mattered was resolving the primary intent: opening a service request. The old system resolved 20% of those calls. The goal was 30%. The new one was at 18%.
To figure out how to get there, we needed data. The application hadn’t been instrumented to measure its own goal, so I specified what to capture and engineering added it. Within a couple of weeks we had it.
We stitched together whole calls: the start, authentication, intent capture, the task steps and wrap-up. We also listened to more than a hundred calls, mostly ones that failed in or right after intent capture. (Listening is key. Transcripts leave a lot out. They drop pauses and side speech, and they pick up background noise as if someone had answered.)
Three things showed up. Callers were frustrated at having to authenticate up front. The system failed to capture what a lot of people were calling about. And even when it did, finishing the request took too many steps.
It was tempting to fix all three. I ran the numbers first. Improving the completion path wouldn’t get us much closer to 30%, because not enough calls were reaching it. The fix had to be at the front: authentication and intent capture.
More data and more recordings told us why. Callers heard a generic greeting and then a string of questions before they learned anything about what the system could do for them. When it finally asked why they were calling, they got several instructions and an open-ended question at once. People hesitated, gave partial answers, stammered. The system treated that as errors, which added to the bad experience.
So I redesigned most of the opening. The greeting got specific. The prompt asking why they called got about 60% shorter, and I rebuilt it as “Why” + “Action” + “Method” in place of confusing instructions and unnecessary examples.
We wrote two alternate versions and tested all three against the original, live. All three beat the original. One beat the others by more than 10 points. (Interestingly, it wasn’t the one I’d have bet on.)
With that prompt and some clean-up before and after it, more calls reached the part of the system that could resolve them. Resolution went from 18% to 33%.
If your assistant is underperforming:
- Find and agree on one goal metric, how it’s calculated, which sessions count and what the baseline is, before you judge anything.
- Make sure the system records what you need to measure that goal. Audible and digital data. Test the data before launch.
- Read or listen to whole conversations, especially the failures.
- Do the math on the traffic and where you’re losing people before you fix anything. Fix the step that moves the goal.
- When people hesitate or stumble, look at when and how you asked the question.
- Test alternatives in production. Your favorite might not win.
Where are your conversations falling apart?