Deep: How ElevenLabs built an AI User Research Voice Agent
AI Voice in Practice: How voice AI is transforming product activation flows, demos, user research, internationalization and more. Examples from world leading companies.
🔒DoP Deep goes deeper into the concepts and ideas that are covered in the Weekly Briefing to help you learn lessons from the experiences of top tech companies. If you’d like to upgrade to receive these in-depth pieces of analysis you can upgrade below. New reports are added every month.
Product discovery activities that used to take weeks can now be completed in just a few days.
The multi-billion dollar AI startup ElevenLabs recently revealed that they built their own user research assistant that conducted over 230 user interviews in less than 24 hours. The research was part of a discovery process for its new reader app, the Eleven Reader, and here’s a video of it in action:
The agent was trained to ask open-ended questions and avoid leading or biased questions. It generated adaptive follow-ups when responses were vague, and automatically extracted structured insights from unstructured dialogue. Once each research session was complete, Engineers, PMs, and researchers got aggregated feature requests, sentiment indicators, and competitor comparisons without having to manually review hundreds of transcripts.
Needless to say, voice AI capabilities like this are starting to have a transformative effect on how we build products.
In this DoP Deep, we’ll take a closer look at exactly how ElevenLabs did this including the models they chose, the prompts they used, essential guardrails and more so that you’re able to build your own AI user researcher to augment your own product discovery processes.
Plus, as well as ElevenLabs, we’ll explore how Voice AI features are impacting product teams more broadly with a closer look at some of the latest new AI voice features shipped by some of the world’s leading companies.
These examples are not your traditional “replace the customer service phone lines with AI”. Instead, we’ll look at some new AI voice features that directly impact product strategy and metrics that product teams care about. For example, one company recently found that introducing AI during product onboarding led to an 80% activation rate and another found that AI voice features boosted engagement rates by 55%.
Coming up:
Real examples of how voice AI is transforming core product areas including: user research, product activation, internationalization, and more - with strategic frameworks you can apply to your own product
Why one product’s users are spending 26 minutes (some over 90 minutes) in AI-powered voice onboarding conversations - and how this led to a seriously impressive 80% product activation rate
The productivity app AI voice feature that increased engagement rates by 55% and made users 5x more likely to upgrade to paid plans
How an $8 billion tech startup is using AI voice in 32 languages as its secret weapon for international expansion
The technologies that are powering this new AI voice expansion - with links to further reading, research and resources you can use to get up to speed quickly
AI voice in practice: How this analysis is structured
We will take a look at several different examples of AI voice features and their impact on the product development process across different areas including user research, activation, internationalization and more.
Each AI voice application example featured in the Deep dive is broken down into the following:
Company - the Deep dive includes a mix of B2B and B2C product use cases.
AI voice example - the example of how AI voice is used in this case including, where necessary, the specific name of the AI voice feature.
How it works - an explanation of how the AI voice feature works in practice.
Product area - where relevant, the product area that this AI voice feature is intended to help with. The areas include user research, onboarding and activation, growth, engagement and more.
Links to find out more - links to learn more about the specific example included.
1. How ElevenLabs built an AI user researcher with key principles and takeaways for product teams
For this first example, we’ll explore how ElevenLabs built their AI user researcher along with some of the important takeaways for product teams to consider. Here’s a snapshot of how it works:
In one sense, the ElevenLabs case study is a wonderfully smart example of dogfooding as a PR exercise; ElevenLabs is, after all, an AI voice company so why not scream about how good it is by building a real world example of it in action like this.
But if we look beyond the PR exercise, it’s still a very strong example of just how powerful AI voice and agent technologies are - and how they can transform the product development process. We’ll dig more into some of the technological breakthroughs later, with some research you can read for yourself to get up to speed too.
Back to this example and the tl;dr here is that ElevenLabs built a sophisticated AI-powered customer interview agent to scale qualitative user research for a single specific product, their ElevenReader app, conducting over 230 interviews in just 24 hours.
How the agent was designed
The agent was designed to investigate four specific areas through a blend of 10 fixed questions and open-ended questions focused predominantly on:
Feature requests and product improvements
Primary product usage patterns
Competitor comparisons (text-to-speech apps and audiobook services)
Pricing perception and brand value
To do this, they hand picked a natural voice called Hope that was selected specifically because of its natural, neutral warm conversational tone.