No results found for your search
What are the best practices for writing a hypothesis for a growth experiment?
Any thoughts on how long these should be and how specific? Maybe best place to start is explaining why it's important to have a hypothesis.
I like the hypothesis kit setup by Craig Sullivan with the help of Colin Mcfarland (SkyScanner), Lukas Vermeer (Booking.com), Rik Higham, Doug Hall, Michael Aagard (Unbounce) and others. It helps me focus on the right things when setting up an experiment. Your current situation, your proposed changes and your expected outcome. With outcome I mean, the impact and the metrics you are going to use. Because you are forced to think of this upfront you will create a good experiment.
The simple variant is:
1. Because we saw (data/feedback)
2. We expect that (change) will cause (impact)
3. We’ll measure this using (data metric)
And the advanced way is:
1. Because we saw (qual & quant data)
2. We expect that (change) for (population) will cause (impact(s))
3. We expect to see (data metric(s) change) over a period of (x business cycles)
Here is Justin's Cliffnotes Version:
Writing a hypothesis is like creating a mini business/marketing plan for an experiment. It establishes what you are trying to learn, what are the factors that can alter the test, and how the test will be conducted. Creating a hypothesis keeps you focused on the goal and helps you determine if an experiment was successful. Here's an example:
Hypothesis: Mothers will be more likely to click on a Facebook ad featuring children.
Factors: Image, Ad copy, Audience, Ad Format, Campaign goal, Facebook auto-optimization
1. Create a Facebook website traffic (clicks) campaign
2. Create ten separate ad sets with one ad in each set - ten total ads
3. Each ad will have a different image; five ads will have images of children - the other five ads will have images of other cute subjects relevant to the company/product (e.g. puppies, kittens, male models, etc.)
4. Each ad will have the same ad copy, targeted audience, and daily budget.
5. Run ads for five days and/or until there is a statistically significant result.
This didn't take me long to build because Facebook advertising is something I do on a regular basis. However, if I am conducting a new experiment on a new advertising platform creating the hypothesis helps me plan my experiment so I do not waste time or money.
Great answer, thanks for taking the time to write it!
I'll tackle the part about why it's important to have a solid hypothesis for tests... I had a conversation with someone today who said that several people in his company were arguing over whether an experiment worked or not. If they had agreed on a hypothesis up front, it would have been a lot easier to say whether it worked based on the hypothesis or it didn't work. The core of the problem at his company was that both groups had different hypotheses that were never stated before the test was run. This means lot of wasted time and often leads to frustration with testing in general.
Right back at ya!
This is a great question. When I talk a startup or company wants to use digital channels efficiently, they generally focus on "solutions" and "channels". Everyone can create a solution and get success at some level. I believe that before considering solutions or channels working on growth problem creates the difference between copy/paste webmaster tactics and real growth process.
First and short answer of "Why it is important to have a hypothesis?" is because hypothesis creates growth process. Tactics or channels don't. Long answer;
Focusing on problem needs a mindset shift. When you start to write hypothesis for your growth project you'll realise that actually you don't have much deeper understanding of your north star metric, your business growth model and your potential growth problems. When you realise this truth everything start to change.
i. You can start to evaluate your analyse process and channels. If you don't have enough user or beginning of growth process, you can consider user research techniques. With the right questions, interviews and surveys helps you to find out main growth obstacles.
ii. Try to switch your perspective. Some of growth hypothesis are for local maximum optimisation and some of them hypothesis for long term growth opportunities. Be prepared to look at both with telescope and microscope.
When you gather information about business model, growth process and your user motivations, you are ready to create hypothesis. We use these templates;
Based on the insight that (where do you take this idea) we predict that (minimum testable idea) will cause (which metrics will be good fit for this idea).
When you start to backlog all of your growth ideas like this, most of your ideas and perspectives start to shift within 1 to 3 months and instead of copying tactics you start to make real progress. We share some of our case studies and our principles at http://growthhacking.studio
A solid hypothesis should contain a Problem, Proposed Solution and Predicted Outcome.
This approach helps me get everyone aligned with why we are running a test and sets out a clearly defined success metric. I have attempted to support tests in the past with a half hearted hypothesis and I've ended up negotiating the success metric(s) after the test has been run - not ideal.
Join over 70,000 growth pros from companies like Uber, Pinterest & Twitter
Use the feedback box below if you have a question, comment or general feedback.
Your feedback has been sent.
Sweet! The link has been copied to your clip boardy board!
Flash isn't supported. Please copy the link manually.