Research that changes the roadmap
Every product team says it is user-centred. Fewer let research change what ships. The usual pattern: a researcher runs interviews and presents a report, then the team builds what was already planned.
AI products raise the stakes. Their behaviour varies from one user to the next, and teams now have new data streams, such as traces and eval scores, that look like answers. The research that earns its budget from here on ties evidence to a decision before the decision is made. This guide covers how to choose and combine methods, and how to get findings acted on.
Start from the decision
Before you pick a method, write down the decision the study will inform, who makes it and by when. If you can't name the decision, don't run the study. If the decision is already made, don't pretend the research will change it.
A one-page research brief should cover:
- The decision. For example: "Do we launch the AI summary switched on by default, or as opt-in?"
- The decision-maker and the deadline. A name and a date.
- The current bet. What the team will do if research finds nothing new.
- The evidence that would change it. Agree on this before you collect data, so nobody moves the goalposts afterwards.
Match the method to the decision
- Exploring a new problem space. Contextual inquiry, diary studies and in-depth interviews. You need to see behaviour in context, not hear predictions about it.
- Choosing between design directions. Moderated usability tests on prototypes. Test the specific interaction, not the general concept.
- Validating before launch. Unmoderated task-based tests with a larger sample, when you need to know how often a problem occurs.
- Measuring after launch. Product analytics plus short intercept surveys. The numbers show what happens, and the survey and follow-up interviews show why.
- Testing an AI feature before it exists. A Wizard of Oz study, where a person plays the AI behind a prototype. NN/g's guide to the method explains how it lowers the cost of testing before the model is built. For a bigger question, run it inside a five-day design sprint.
Combine methods on purpose
Single methods mislead. Interviews over-represent articulate users. Analytics show what happened with no reason attached. NN/g's Rachel Banawa describes three mixed-methods designs (July 2025):
- Explanatory sequential. Quantitative first, then qualitative to explain it. Analytics show that half of users discard the AI draft, and interviews show why.
- Exploratory sequential. Qualitative first, then quantitative to size it. Interviews surface five reasons people distrust a feature, and a survey measures how common each one is.
- Convergent parallel. Both strands at once, compared at the end.
If the strands disagree, resist picking the one you like. NN/g's guide to interpreting contradictory findings recommends checking whether the methods measured the same thing with the same people before you decide which to believe.
AI products add a new quantitative stream: traces and eval results. An explanatory sequential study fits well. Cluster the eval failures first, then interview and observe the users who hit them. We walk through a full example in mixed-methods UX research in the AI era.
Measure behaviour as well as belief
METR's 2025 randomised trial with experienced open-source developers shows why. The 16 developers in the study took 19% longer to finish tasks with AI tools, while believing they had been about 20% faster. METR's 2026 update found the follow-up study hard to run, partly because many developers would no longer work without AI, and called its new estimates inconclusive.
The research lesson holds either way. Self-report and measured behaviour can point in opposite directions. When a decision depends on speed, accuracy or effort, measure it.
Use AI in research without losing the evidence
AI is already in most researchers' workflows. User Interviews' State of User Research 2025 (485 respondents) found that 80% of researchers use AI, up 24 points on the previous year. The useful question is where it helps and where it hurts.
- Good uses. Transcription, tidying notes, first-pass tagging, drafting discussion guides and desk research. NN/g's advice on research with AI supports these.
- Use with care. AI-moderated interviews work for structured feedback and screening at scale. NN/g's Maria Rosala finds them weak for discovery work and for anything that depends on observing behaviour.
- Keep out of decisions. Synthetic users. NN/g's review of synthetic users accepts them for hypothesis generation and desk research, and rejects them for validation. Use them to rehearse a study, and use people to decide.
Whatever AI does in your synthesis, keep every insight traceable to a quote, a clip or a data point. If you can't trace it, you can't defend it in the room.
Make findings hard to ignore
A 40-page report gets skimmed. Findings that show up where the team already works get acted on.
- Put clips in design reviews. Two minutes of a user struggling beats any chart.
- Link findings to tickets. Attach each insight to the backlog item it affects.
- Keep a decision log. What the research showed, what was decided and why.
- Share weekly. Short updates during the study beat one presentation at the end.
- Read traces together. For AI features, a weekly session where a PM and a designer read real conversations surfaces problems no dashboard shows. We explain how in LLM error analysis is user research.
The researcher's job is decision support
The most effective researchers see their job as reducing the risk of building the wrong thing. That framing changes which methods you choose, how early you get involved and how you report. It also makes research easier to fund, because every study ties to a decision someone already cares about.
FAQ
Which UX research method should I use? Start from the decision. Exploration needs observation and interviews. Choosing between designs needs usability tests. Sizing a problem needs quantitative data. Most real decisions benefit from two methods combined.
How many participants do I need? For qualitative usability testing, around five people per distinct user group will surface most major problems, as Jakob Nielsen's analysis shows. To measure how common something is, you need a larger sample and a quantitative method.
Can AI replace user interviews? Not for decisions. AI helps with transcription, tagging and drafting. Synthetic users and AI moderators have narrow uses. Decisions about real users need evidence from real users.
How do I get stakeholders to act on research? Agree on the decision and the evidence that would change it before the study starts. Then bring findings to them in short, visual form, linked to the work they own.
If you want research set up around the decisions your team faces, including research for AI features, our UX research and product design consulting can run the study with you or train your team to run it.