Leave a comment
Get the GH Bookmarklet

Ask GH

Daniel Tunkelang from Linkedin during the AMA said: "The most important investment you can make as a company is deploying a platform to makes experiments as cheap to deploy as possible. Doing so reduces the cost of experimentation to the bare bones: developing experiments and running them." So, my question is do you have such a platform and can you describe it for us?

  • YS

    yassin shaar

    over 4 years ago #

    I came across this when I was looking for tools to run in product split tesrs...

    This tool was developed at Facebook to run experiments: http://facebook.github.io/planout/

    Well be deploying this in the coming weeks.

  • EJ

    Evan J. Andriopoulos

    about 4 years ago #

    Yes... we developed our license management and distribution system "Miits" prior to our actual gen.1 video conferencing service. We needed to be able to deploy / distribute licenses quickly so our customers can BETA and eventually buy / trial our services in seconds. We like to say sign up to in a video meeting in 2 minutes or less, no registration for anyone but the original meeting originator.

    No as we embark on gen.2 we will build off of that model and hopefully have you from sign up to a meeting in a few seconds. Our platform can be deployed quickly to anyone that works in the SaaS space. Check out gen.1 www.visiplevc.com

  • AD

    Ali Dinani

    about 4 years ago #

    Our team built a custom one in-house using google experiments.
    Essentially you create your different variations in independent files with a suffix (ex: payment_plan_variant_horizontal.html.haml) and run a single line of code.
    This line of code consistently and evenly distributes the variations to our users and sends the data to Google as needed.

    • HQ

      Hila Qu

      about 4 years ago #

      Very cool, Ali, curious to learn at which point a team should decide to move to custom platform? What's the biggest benefit comparing to using commercial solutions?

  • JG

    jordan gutierrez

    over 4 years ago #


    • ES

      Edo Sadikovic

      about 4 years ago #

      I was thinking more on the custom made model for a particular company (where Optimizely could be part of it). Thanks

      • HQ

        Hila Qu

        about 4 years ago #

        Hi @edosadikovic

        Can you share your thinking around at which point a team should decide to move to custom platform? What's the biggest benefit comparing to using commercial solutions?

  • DG

    Dwight Gunning

    about 4 years ago #

    Try find a good "feature flip" plugins/extension for your framework of choice. I've had success with django-waffle.

  • TM

    Ty Magnin

    about 4 years ago #

    Try appcues.com you can build experiences for user engagement without writing any code.

  • SJ

    Sebastian Johansson

    about 4 years ago #

    Daniel Tunkelangs AMA was awesome.

    But I havent heard anyone really define what a platform is. An experiment is, I guess, testing an idea against two groups. But I guess the groups wouldnt have to be people only. You could for instance run a test on two servers, where the metric is simply to achieve faster site speed.

    So I guess a platform has two purposes. To decide a winner and to somehow make the implementation of the experiment faster/less costly.

    But every company runs different kinds of tests. So I dont see how anyone can create an all purpose SaaS platform that makes tests quicker for everyone.

    Vwo and optimizely are good to run AB-test. Google has a tool as well. Some frameworks have similar gems/add-ons. I think Ive seen tools for doing AB-tests on apps as well. But those are focused only on a certain type of experiments.

    And could you really call them platforms? They are just AB-testing tools. Feels like platforms is some kind of hype word. And if the goal is just to get more tests done, you could achieve that in many other ways as well.

    I started to try the high temp testing framework six months ago or so. And each week I failed to reach the number of tests I wanted, I wrote down the reason. And tried to fix the bottleneck.

    Most of the time it boiled down to either not enough traffic (all pages/channels etc. already had a test running) or it was too hard to code the feature we wanted to test or I had to recruit help which took a lot of time. Never did I have a "platform" issue.

    Im guessing for most small business, starting to build your own platform is just going to steal valuable time. A reason to procrastinate doing tests instead. But I do see how some companies with a lot of data for example, can build in automatic ab-tests that are constantly running in the background to make decisions. Or if you have special type of ab-test that you know will be performed over and over again, and you can build software to automatically create those tests. Not just testing them, but some algorithm to create the actual tests as well - like moving around elements in ads etc.

    GH should have Daniel Tunkelang return for an AMA2 focused only platforms for experiments :)