Synthetic users are not user research

Recently there was a big announcement: they had defined core user personas for each of their brands, then created a “synthetic user” for each one. The idea is that you can ask the synthetic user how they might react to a hypothetical problem, UX copy, or potential product feature.

At first, I thought this was interesting. It seemed like a lightweight way to test an idea before investing too much time in it. But the more I thought about it, the more skeptical I became.

Useful for Hypotheses, Not Validation

NN/g is pretty blunt here: synthetic users can support desk research and hypothesis generation, but they should not replace real-user research. Their feedback can be shallow, overly favorable, or unreliable. A recent 2026 paper also found that simulated users often miss the messy communication frictions real users introduce, which can make synthetic testing feel more confident than it should.

I would call this kind of work an AI heuristic pass or a synthetic research pass, not user testing. It can help you pressure-test an early idea, but when the decision matters, it still needs to be paired with real users — even just five of them.

What Users Say Is Not Enough

My biggest concern is that LLMs mostly capture what users say. And what users say is only one layer of understanding.

To understand customers, we need to look beyond stated preferences. Hannah Shamji’s four levels of customer understanding is a useful framework here because it pushes us toward the deeper reasons behind user behavior: hidden motivations, unspoken needs, and the messy reality that often gets lost in simplified research outputs.

People do not always understand their own motivations. We bring our own context and interpretation to every question. And in product decisions, people often behave differently from what they say they will do.

Emotion also plays a major role in decision-making. I do not think AI can fully mimic the emotional state of a person trying to use a confusing feature, evaluate product copy, or complete a task under real constraints.

“Our work is about others — their problems, their pain, their mess. Our job is to make sense of it and then do something about it. Not to emote or perform but to act on and solve it. There is a flawed belief that to build great things, you first need to emotionally fully absorb someone else’s experience.” — Alin Buda

The Real Opportunity for AI in Research

That said, I do see huge potential in making research easier to access.

As the cost of prototyping drops closer to zero, companies will want to test more ideas, more often. But the bottleneck is not just writing better research questions or generating faster summaries. In my own research work, the hardest part was finding the right users and scheduling time with them.

That is why tools like Listen Labs are interesting. They help recruit users and use AI to assist with interview questions and follow-ups. To me, the most powerful part is not the synthetic feedback. It is access to real people who are willing to answer the questions you have.

Better Research Does Not Always Mean More Research

But I also wonder if we are over-optimizing for scale.

Maybe what teams need is not more automated research, but more intentional time with the right users — and more discipline around what we are trying to learn.

We do not need expensive tools to uncover user needs. David Travis has written about lightweight ways to build customer understanding, such as exposure hours, live UX testing, co-design sessions, support-ticket reviews, and listening in on customer service calls or community conversations.

The core idea is simple: create spaces where customers’ struggles are visible across the company.

That might mean sharing short video clips from user sessions, sending a monthly research newsletter, or regularly surfacing patterns from support and sales. The format matters less than the habit of making real customer pain visible to the people making product decisions.

Wrapping Up 

Synthetic users may be useful for early thinking, but they are not a replacement for real research.

To make better product decisions, we need to go beyond feedback and observe what customers actually do. We need to understand the context around their behavior, their goals, and their motivations.

Most importantly, we need to be clear about what questions we are trying to answer. Not what “validation” we need in order to move forward, but what we genuinely do not know yet.

Without that, everything else is just a hunch. And hunches can be wrong, expensive, and easy to mistake for evidence.