“That’s easy to use,” says a participant. Shortly before, she tried to open the lid of a new food container in the wrong place. It only worked on the third attempt. In a questionnaire, the product might still have received a good rating.
The example is fictitious. But it shows why a positive rating and easy handling are two different findings. Anyone testing a prototype wants to know what happens during use and what can be improved before the next development round. Accordingly, Nielsen Norman Group distinguishes between observed behaviour and subjective assessment in usability testing. [1]
Three questions, three different tests
Do people understand the idea, and is it relevant to them? Then you need a concept test. This does not necessarily require a physical sample. A description, sketch or visualisation can be enough to find out which benefits people recognise and which expectations the concept raises. A standardised online test can also be useful here, for example when several concepts are to be compared against the same criteria. [2]
Can people use the product as intended? Then a prototype test with specific tasks helps. With a physical sample, a moderated on-site session can make sense, for example if only a few units exist, the sample is fragile or it needs to remain confidential. A digital click prototype, by contrast, can often be tested in a moderated session via screen sharing. What matters is which parts of use can realistically be observed in the given test environment. [1]
How does the product hold up in everyday life? If the sample is safe to use and easy to ship, a home-use test is a good option. A cleaning product, for example, can be tried out over several days under the conditions in which it will actually be used later. An online questionnaire provides comparable feedback; a short diary can add when and why the product was used. Here, everyday life is part of the research question. [3]
The value of moderation shows in the unexpected
With the food container, the moderator could wait until the participant has finished her attempt. Then she asks:
“How did you recognise where to open the lid?”
This helps to narrow down whether the shape, a marking or an expectation from previous use determined the first grip. Anyone who helps straight away may only find out whether the explanation works – and overlook the problem with the product. [1]
AI can support clearly defined tasks, such as structuring feedback. For early prototypes, we recommend human moderation especially when it is unclear which actions, misunderstandings or detours will turn out to be relevant at all. A comparison in Quirk’s shows that AI moderation missed nuances in the interview setting studied. That was a study on brand trust, not a prototype test. It is therefore no blanket proof against AI, but it makes clear why the specific task matters. [4]
Observe first, then change, then test again
In the example, the product team could make the opening point clearer. In a second round, WMM would check with new participants from the same target group whether they find the spot without help. A few well-observed sessions can uncover specific weaknesses. How often they occur across the market as a whole cannot be derived from them. [1]
This is where the advantage for clients lies: WMM combines research design, targeted recruitment, moderation and analysis. This gives the product team a basis for the next decision: What should we change, and how will we tell in the next test whether it works better?
Sources & context
- Nielsen Norman Group: Usability Testing 101 and 6 Dimensions for Assessing Usability Data in Analysis. Methodological articles.
- marktforschung.de: Online concept testing, using YouGov as an example (in German). Provider description.
- Research Live: In-Home Product Testing. Industry news about a provider example.
- Quirk’s: Comparison of human and AI moderation. Study on brand trust.











