Blog / 2026
When you actually need a usability study, not just session replay
Most of this site’s content about usability testing makes the case that session replay covers more ground than people assume — see session replay vs. usability testing and the cost comparison. That case is real, but it isn’t “session replay replaces usability testing.” It doesn’t, for a specific set of situations. Here’s the honest list of when a study is the right call and recorded sessions won’t get you there.
The feature doesn’t have real usage yet
This is the clearest case. Session replay records what real users do — if nobody has used the thing yet, because it’s a prototype, behind a flag, or genuinely pre-launch, there is no data to record. A usability test is the only method here that doesn’t require the feature to already exist in production.
You need to know why, not just where
Session data is very good at surfacing where a problem is — a rage-click cluster on a specific button, a form field people keep re-typing, a page loop that shows up across sessions (see finding usability issues in session recordings for the patterns to look for). It is much weaker at telling you why, because nobody’s explaining their reasoning in a recording. If you’ve looked at the where and still can’t form a confident hypothesis about the why, that’s a five-participant moderated test on that one screen, not more replay analysis.
The user base is too small or too new for a pattern to emerge
Session replay’s advantage is volume — patterns across dozens or hundreds of sessions are trustworthy in a way one session isn’t. If your actual traffic on the flow in question is a handful of people a week, you’ll wait months for enough sessions to say anything with confidence. Five scheduled participants get you an answer this week, on a timeline replay can’t match at low volume.
You’re testing against a competitor’s product, not your own
Session replay only sees your own site. If the question is “how does our signup flow compare to [competitor]’s” or “would people prefer our version of this pattern or the standard one,” that’s a comparative or preference test — recruiting participants to try both — not something recorded sessions of your product alone can answer.
The decision is expensive enough that “probably” isn’t good enough
If a redesign is going to cost weeks of engineering time, the confidence level a study gives you — directly asking people, watching them reason through it — is worth the recruiting cost even when replay data already points in the same direction. Use replay to build the hypothesis and narrow the study to the parts that actually need validating; that’s usually cheaper and faster than a broad study run from scratch, but it doesn’t replace running one when the stakes justify it.
What this means in practice
The realistic workflow for most teams isn’t “pick one.” It’s: watch what’s actually happening with session replay by default, because it’s continuous and effectively free at the margin, and reach for a usability study specifically when one of the cases above applies — pre-launch, a why-shaped question replay can’t answer, too little traffic to see a pattern, a competitive comparison, or a decision big enough to want direct confirmation.
Treating replay as the default and the study as the escalation, rather than running both in parallel out of habit, is usually what keeps research budget spent on the questions that actually needed a study.
UserTapes records 100 sessions free with no card, so building the initial hypothesis before you scope a study costs nothing to try. If what you find points to a clear “why” question a recording can’t answer, that’s your signal to schedule the study — not a reason not to have looked first.