Leave a comment
Get the GH Bookmarklet

Ask GH

1. # of experiments running every month 2. What is the % of them that are successful? Super interested to know the speed and efficacy of experimentation on a team/company level. Extra: What is the % that require development?

  • MT

    Muhammed Tüfekyapan

    almost 3 years ago #

    As Growth Hacking Studio, we run A/B tests for our clients. First of all, number of experiment is kind of vanity metric for CRO process. It is deeply related with your conversion stage, traffic and conversion problems.

    We have some projects that we can run only one test per month because of website traffic, low conversion and takes time to determine test winner. On the other hand we run 5+ tests for month for another project. It is completely based on clients needs and situation.

    Your second question is also can't be answered clearly. Based on our last three months our success rate is bigger than 50%. This is also highly related with current performance and clients conversion problems.

    Technical development typically takes 10%-15% of our whole test process. Research and analysis takes more than 60% efforts and I believe it should ;)

  • AA

    Anuj Adhiya

    almost 3 years ago #

    1. We run approx 12-15 tests/month (minimum 3/week)

    2. Talking about the % of them being successful can be misleading because each one has a different impact. Most of the ones that are wins are small wins and every now and then you get an outsized win, but this is how growth works, its (mostly) small wins that compound over time.
    Also on an overall basis, as long as your testing, your learning - this is more important than win %

    3. The % that require development varies greatly and is not really a major concern. Our current objective(s) determine which tests we pick to run. If that means they need development, then so be it.
    Having said that, if it so happens that engineering resource is not available during a certain period, we'll pick tests that don't need that resource, even if that test has a low potential impact score because its more important to keep testing and learning than wait.