Former Google Exec on Building AI Agents that Customers Actually Love

Nearly 95% of enterprise AI deployments fail to deliver the outcomes they promise. The reason for these failures often isn’t a flaw in the technology itself, but a fundamental misunderstanding of its purpose. Too many AI initiatives focus on automation and efficiency rather than what truly drives success: customer trust and delight.

This problem is at the center of Cassie Kozyrkov's work as CEO of Kozyr and former Chief Decision Scientist at Google. In a recent conversation with Cresta's VP of Product Marketing, Devon Mychal, Kozyrkov explained that for decades, frustrating customer experiences were an accepted byproduct of technological limitations. But today, she argues, those excuses are gone.

"What I'm pretty sure we hate is being dehumanized by shoddy design choices, by technology that is designed by someone who clearly does not have our best interests at heart," Kozyrkov says. One of the reasons we keep making these poor design choices, she explains, is a deep-seated cognitive mismatch.

• An unfair standard: We hold AI to a different standard of perfection than we do humans because our mental models are based on much simpler tools, Kozyrkov explains. "Our tools have been so simple. A hammer does the same thing every time. The complicated thing is its wielder. With digital tools like spreadsheets, if there’s an issue, we still tend to look for human error. But when we try to automate intractable things with countless possibilities, we have to rely on probability and make the best guesses based on data. Making that mental change from a deterministic hammer to a probabilistic system is going to be an interesting leap for society."

That "hammer-first" mindset helps explain why many enterprises still measure AI performance with outdated KPIs, Kozyrkov says. When building a truly flexible system was "impossible or expensive," old metrics like call deflection reflected that reality. Now, teams must adapt to a new way of managing customer experience.

• Zombie virus metrics: "When we use metrics like 'containment' and 'deflection,' it's the wrong way to think. It's like treating your precious customers like a zombie virus that needs to be quarantined," Kozyrkov says. Instead, the real innovation comes from focusing on experience. Eventually, that might mean upending old metrics entirely, she predicts. "I expect that call time for human agents will go up. Because humans are the only ones who can take responsibility, the gnarly, difficult situations that require a lot of human judgment will ultimately end up flowing to them. The hard functions are for the humans."

Adopting this new mindset also means building systems that can be trusted, Kozyrkov explains. To make her case, she tells the story of a dealership chatbot that accidentally sold a car for $1. For her, trust is a two-step process: rigorous testing before launch, followed by ongoing observability because, as she says, "the world is allowed to change."

• No magic in the mix: Building resilient AI means doing the hard work of preparing data, she says. "People underestimate how hard it is to get their data ready. They imagine you can just stir some digital soup and good things will come out, but it's actually hard work to structure and put meaning into that data so that what you're feeding in isn't just noise."

• The forehead tattoo rule: Kozyrkov recalls how she often gave Google engineers advice on how to trust AI systems: "Consider tattooing this on your forehead: 'The world represented by your data is the only world that you can expect to succeed in, and the world represented by the test data is the only world that you can expect to trust.'"

• Problem over platform: In a market where businesses are constantly rebranding as AI companies, leaders are sorting through the hype. Here, she recommends prioritizing their own needs over the technology. "AI will become the boring bit, and that's good. The interesting bit should be that a real problem was there and got solved."

But the real value of AI in customer service comes from its partnership with human agents, Kozyrkov explains. By lifting human agents out of repetitive drudgery to focus on more creative, higher-level problem-solving, "AI is promoting us all,” she says. Now, the most success lies in creating systems that deliver "superhuman context and superhuman memory with human-to-human interaction and human-to-human care."

Ultimately, the goal of this partnership is to give customers control and personalized experiences that feel natural, Kozyrkov concludes. "A lot of power is going to go back to the customer, and the customer will have the opportunity to dictate how they want to receive their help. That personalization is magical. It will feel to the customer like they were cared for. That's our future."

**Watch the full webinar recording here** to learn how to identify the right success metrics for AI, design empathetic interactions, and build systems that earn trust at scale.