This is going to be a slight departure from our typical growth study. Our other growth studies generally cover the entire growth history of a company. But this one begins in January of this year after we’d experience three frustrating months of stagnating growth. Up until this point we had largely grown on the back of Twitter, reaching around 90,000 MAU (Monthly Active Users) in the first year. But increasingly the hands-on execution of our Twitter playbook was preventing us from generating and testing many new ideas.
This is ironic because I generally advise other growth teams to “test, test, and test some more.” I explain that rapid testing is the most effective way to predictably grow a business. But it took the frustration of stagnating growth to cause us to fully embrace a High Tempo Testing program at GrowthHackers.com. Starting in January we set the goal of launching at least three new experiments per week. We use the term experiment broadly to include new initiatives, product feature releases, and yes, A/B tests. When we’re running high tempo, we’re pushing on all vectors to find growth in a systematic, measurable way.
The graph below shows what happened:
We grew from 90,000 MAU to 152,000 MAU in about eleven weeks without spending a dollar on advertising or increasing the size of our growth team. The only thing that changed was the velocity of our experimentation. We achieved that velocity improvement by adopting the high tempo testing framework to help us prioritize and close the loop on the growth experiments that we thought would move the needle.
What is High Tempo Testing?
Most growth professionals understand that growth hacking isn’t about a specific tactic, but rather it’s about a process of discovering which tactics will be effective for growing your business. This discovery happens through testing potential growth drivers ranging from new channels to new sharing and engagement features deep within a product. The more tests you run, the more you learn about how to grow your business. So it’s only natural to want to run as many tests per period of time as possible. That’s the idea behind high tempo testing.
My inspiration for high tempo testing comes from American football and the “high tempo offense.” Perhaps the best example of this is the Oregon Ducks’ college football team. From the moment they hit the field they are go, go, go… The opposing teams often end up on their heels and Oregon is able to find weaknesses in their defense. It’s exciting and frankly kind of exhausting to watch. High tempo testing is approached with a similar energy and urgency to quickly uncover growth opportunities to exploit.
Requirements for High Tempo Testing
High tempo testing starts with identifying experiments worth running. If one person is responsible for all ideation, they generally run out of ideas within a few weeks (at least ideas worth testing). Even within a dedicated team, ideation can become ad hoc and stagnate without a process in place that acts as a springwell of new ideas. To jump-start fresh ideas for growing GrowthHackers.com we opened the ideation process up to a much broader team. Naturally, this includes everyone on the growth team, but we also invited our interns, engineers, salespeople, support people, and more. Essentially any employee or contractor in the company that wanted to participate. But we didn’t stop there. We included advisors, friends, and even some of the most active community members. We collected hundreds of ideas from this group.
Interestingly, different groups contributed ideas based on insights that were unique to them. Some people gravitated toward ideas to improve retention, while others focused on acquisition or referral. The ideas themselves ranged from ways to improve community engagement to product enhancements. For example, one of the engineers suggested we should use Oembed to make it easier to embed highly engaging multimedia content. Most of the rest of us didn’t even know that was possible and had suggested a more manual media upload process.
After the ideation started to slow down, we decided to add a leaderboard to celebrate the people who were generating the most ideas. The leaderboard re-energized the ideation process so that we are now adding about 2 - 3X more ideas per week than we can test. Therefore, we are continuing to grow our backlog of hundreds of ideas.
Eventually, we faced the challenge of sorting through all of the ideas. With hundreds of ideas, it was tough to know where to start. We scored every new idea based on its potential impact on growth, our confidence that it will be successful, and how easy it is to implement. In other words, using an ICE score. Once the ideas were scored, it was easy to sort them and find the best ideas in the areas we thought we should prioritize first (like activation).
In the beginning, we often got bogged down trying to launch a big idea. Each time our tempo slowed down (missing our goal of launching three tests per week), our growth slowed down. The solution was a Weekly Growth Master meeting. This meeting is not for ideation. It is for processing the backlog of ideas that we already have, prioritizing, and assigning the list of ideas for the next week. Think of it as a sprint kick-off in agile development. The backlog has already been groomed, the user stories are already written, it’s time to build the weekly growth agenda (or sprint in agile terms).
Some growth experiments can be implemented by the marketing team, others by product managers and others require deep engineering skills. Balancing the workload of top priority experiments across different teams makes it much easier to hit our tempo goal. While our goal is to launch at least three tests per week, we generally shoot for five tests per week. That way if we hit roadblocks on some tests, we still hit the tempo goal. Many weeks we are able to get all five tests out the door. Our weekly growth master meeting is also used to review the tests from previous weeks. Were we able to implement all of the tests planned? Have any tests concluded and if so what were the key lessons learned? This accountability and planning meeting is an essential part of keeping the growth process on track.
All of the completed experiments are ultimately stored in a knowledge base so we ensure that we capture this learning and don’t keep repeating the same tests. The knowledge base is also very helpful when adding new members to the team, so they can understand what has been tested, what worked, and what didn’t. The analysis has started to become a bottleneck for us, so we recently added a role that is responsible for analyzing the completed tests and managing the knowledge base. Previously this responsibility was shared by the team.
High Tempo Testing Drives Results
Using a High Tempo Testing approach we’ve been able to add as many new MAUs in the last 11 weeks as we did in the first 32 weeks following the launch of GrowthHackers.com. High tempo testing isn’t easy, but it is extremely effective. It requires both a rigorous process and a system to manage that process. Much like lean manufacturing, it focuses on throughput and small batch sizes. By adopting high tempo testing, we eliminated the bottlenecks at the ideation and experimentation steps to unlock more growth. If your team is truly committed to driving growth, High Tempo Testing is the most predictable way we’ve found to hit aggressive targets.
If you’d like to run high-tempo testing experiments across your whole customer journey, get a Free Trial of Experiments.