Experiment Recipe: Exit Poll
The ‘Exit Poll’ experiment aims to get a reaction from test subjects just at the time they have experienced the problem.
Erik van der Pluijm - Aug 21 2019 - 8 min read
|Time||± 45 minutes|
|Difficulty||3 / 5|
|People||3 - 5|
|License||CC BY SA 4.0|
The purpose of the experiment canvas is to design the right experiment at the right time, facilitating a team to have the right conversation. With the experiment canvas, it is easy to design a well-defined experiment.
Tip! Track the data immediately and write everything down, so that you can check afterward if you interpreted the results correctly.
Start with identifying the current Riskiest Assumption and describing it in a way that you want to test. What will it mean for your idea if this fails? How can you tell?
Specify a clear, falsifiable hypothesis and experiment setup. After running the experiment, check the results and plan your next steps.
Your hypothesis is a statement you believe to be true about your riskiest assumption. Write it down before you run the experiment. It is too easy to change the conditions afterward to make the data fit, and this robs you of valuable insight.
For this version of the canvas, the original hypothesis formula is extended so it is easier to make it quantifiable.
Hypothesis: 'We believe, that (specific testable action) with at least (minimum number of respondents) selected from our (target audience) results in at least (percentage) responses like (what counts as a positive result) within (time frame).'
Each of the elements in brackets need to be quantified.
It’s okay to use a bandwidth for this, as long as you specify it upfront. The metrics you define need to be actionable (i.e., they need to directly relate to the hypothesis) and accessible (i.e., you need to be able to see the results).
Try to find benchmarks to define percentages, and allow for the fact that if you have small numbers of respondents, you will need quite large measurements to be sure. Look for large percentages.
Having a good protocol to run the experiment is key to quality results. It's a complete science to do this right, but for our purposes it is sufficient to just avoid the biggest mistakes.
Here, you'll need to come up with the questions or prototypes you need in detail.
The materials you'll create fall broadly in three categories: interviews, offline prototypes, and online prototypes.
Tip! You rarely need to test a technical prototype at this stage. In most cases, if it actually works in reality only becomes interesting after you have made sure people are waiting for it. So when we say 'prototype' we really mean something that looks ‘just real enough’ that customers are able to react to it in a meaningful way. And when we say 'just real enough' that is really telling you to do the bare minimum. No difficult branding exercises, technical setups, or scalable solutions!
Warning! Never forget the golden rule: KEEP IT SIMPLE!
Ok, so you have an experiment setup. Now it's time to get out there and collect the data. Use the protocol to your advantage! Make sure you record everything of value (and use for instance a google sheet to fill in the data).
Once all the data is in, it's time to go over it. Get your team together and score the data. For qualitative experiments, have a look at the Experiment Outcome Canvas also included.
Once you have your data all interpreted and scored, it's time for conclusions. Get your scores together and compare what you got with the hypothesis you setup.
If you can answer 'yes' to all of them, then you'll have a validated assumption. If you can answer 'yes' to the first three points, but didn't get enough positive scores (the last point), you'll have invalidated the assumption. If you can't answer points 1-3 with 'yes', you'll likely have a botched experiment, and the result is inconclusive.
Ok, you've got your conclusions. Time to act on them. There are three paths forward:
Sign up for free now and receive a weekly email with one new innovation tool straight to your inbox. 🚀