Leave a comment
Get the GH Bookmarklet

Ask GH

I typically focus on user acquisition and bringing in traffic, so I haven’t had as much experience with optimizing web pages. I’d like to know specifically what tools you use and what things you look for when doing this.

Is using just Optimizely sufficient? Do you have some sort of mental checklist that you go through to determine what tests should be made and where?

I appreciate all answers, and will gladly respond to everyone’s comments and will clarify anything if needed.

  • SE

    Sean Ellis

    about 6 years ago #

    Hi Brandon, Prior to my current CEO role, I focused nearly 100% of my time on conversion rate optimization. Figuring out the right tests to run is a huge part of the process.

    The tools I use are A/B testing (Optimizely), analytics (KISSmetrics), and qualitative insights (UserTesting.com and not surprisingly Qualaroo)

    Below are the four key ways that I figure out what tests to run:

    1) Identify must-have experience - Ultimately you want to maximize the number of people that experience the product in the right way. It's a pretty long answer, so I recommend you read this blog post to help you identify the must have experience: http://www.startup-marketing.com/using-survey-io/

    2) Baseline your funnel - Part of figuring out the right test to run is determining the potential upside of the the test. For example, if a conversion point in your funnel has a 90% drop off rate the potential lift is probably much higher than if a conversion point has a 1% drop off rate. While it's rarely that extreme, you want to look at any point and say "if it were near optimal, how high do I think I could get it?" Focusing on something with a lot of potential for lift is a better use of time than focusing on something that is already pretty good. If you can get benchmarks from other companies, that can be useful in thinking about the upside. But otherwise, you'll have to rely on your gut. Once you start testing, you'll have a better idea which areas are easy to improve. If you run a bunch of tests and can't move the needle, that's a good indication that the step was probably pretty good to begin with.

    3) Identify points of friction - For identifying points of friction I like UserTesting.com and in flow surveying (Qualaroo in my case). For UserTesting.com I give people an onboarding task to complete on my website and watch a bunch of people try to complete it. I'm able to find lots of ideas for points of confusion or concerns we aren't addressing in messaging.

    The challenge with user testing is that it is hard to simulate an actual person with a need for your product trying to complete the task. That's where in flow surveying can help. You can target questions to specific points in your conversion funnel and ask people questions about their experience who are really interested in your solution.

    For example, at one company I had a huge drop off rate for a particular referring channel after users signed up for a free account but before they downloaded our free software. Random A/B tests of messaging, button styles/sizes etc didn't fix the drop off. But when we surveyed people asking if there was anything preventing them from downloading, a ton of people wrote that they were worried that the product wasn't really free. Our next test gave them a choice between downloading the free or paid versions, which improved our conversion rate at that step by about 300%.

    This webcast can give you some ideas for questions that help you prioritize testing: http://get.qualaroo.com/golden-questions/

    4) Identify aha moment - I also use UserTesting.com for identifying aha moments. I record the user session for some target prospects trying to complete the overall onboarding for a product (from home page visit through actually using the product). I run several people through the process and look for that moment when people say "wow this is great" or "Oh, I get it. Very cool." I'm looking for any comments where people express enthusiasm for what we are doing. These help my testing by figuring our tests that try to front load the experience that people were having in our onboarding process when they had their aha moment.

    Hope this helps.

    • AC

      Alex Cook

      about 6 years ago #

      Sean - this is what I'd call a golden comment. I haven't checked out UserTesting or Qualaroo yet and plan to now. Thanks!

      Totally agree with baseline funnel metrics. Brandon, I'm guessing your flow is more than just one page (what happens behind the landing page is also important). It's surprising how many startups don't know their baseline and don't look at it often (aka once a week). I'm a huge fan of Mixpanel (similar to KISSMetrics). Most of these metric packages require an engineer to implement the events (aka the user visited the login screen). So, I always question everything... especially if someone else created the metrics. There's always a chance an event isn't tracking properly. It's a bummer to make decisions based on a funnel metric only to find out it wasn't tracking correctly weeks later. I often run through my own funnel and make sure all of the events are firing as expected and just question everything. All this said, I'm not sure what stage you're at. Ff you don't have significant traffic at the top of your funnel... granular metrics probably won't be statistically significant. I'd rely more on qualitative feedback if you're early.

      For qualitative feedback:
      (1) One of my favorite tools is live chat. For that I've been using SnapEngage (http://snapengage.com/). In particular, their Proactive option helps drive up the number of customers willing to chat.
      (2) Another overlooked tool: the phone! Even though it's time consuming, there's something magical about connecting with a real customer who is on the page you just launched. Plus, some people just prefer to communicate by voice.
      (3) While not remote... I love just visiting customers in person for tests. I worked on a small business software solution a few years ago. One thing you can't really measure remotely is the distraction factor. It's amazing how many phone calls and other distractions the average small business owner receives in a 15 minute time frame... all the more reason to simplify (aka shorten) your flow as much as possible.
      (4) Although I haven't used it yet in a big way, I also love what Intercom is doing (https://www.intercom.io). It basically allows you to establish rules then send off manual or automated messages to users via email and within the app itself. Mixpanel people also offers a similar solution. Sounds like Qualaroo does, as well.
      (5) Sometimes I also just test flows on my friends. I'm not a user testing expert but I basically just: (A) figure out the steps I want to test (aka a sign up flow) (B) set some reasonable expectations, then (C) ask the tester to think out loud as they go through the flow (D) ask open ended questions and be a great listener.
      (6) Mouseflow (http://mouseflow.com). The tool allows you to basically playback a web session, tracking where the user's mouse went. Plus, they've got heatmaps (which I don't use very often but they're still fun to look at).

    • BP

      Brandon Pindulic

      about 6 years ago #

      Hi Sean,

      I really, really appreciate this. Many great insights, and it definitely helped.

      Time to put your advice to work :)

      • SE

        Sean Ellis

        about 6 years ago #

        You're welcome. Glad it's helpful. Feel free to post follow up questions in thread if you need clarification on anything.

  • PL

    Peep Laja

    about 6 years ago #

    Perfect answer requires at least a full day workshop, but here's a condensed version.

    (Background: I run a conversion optimization company and run one of the most popular CRO blogs ConversionXL: http://conversionxl.com/blog/)

    Optimizely is for running tests, but it's not used for determining WHAT to test (although each test result contributes to your customer theory - if you send all test data to Google Analytics (a must!) and segment the hell out of results).

    You can’t test everything at once, and traffic is precious. So you need to know how to prioritize your tests.

    Start with understanding the mindset and buying process of your target audience, and identifying "problem areas" of your website.

    Audience: survey your recent buyers to figure out who they are, what they want, what was their shopping process like, what were some of the causes of friction, why they almost didn't buy. You can use Google Docs surveys, Typeform or SurveyGizmo for this. You can copy my favorite questions here: http://conversionxl.com/how-to-identify-your-online-target-audience-and-sell-more/

    Use Qualaroo to figure out why people leave your site without making a purchase. My favorite question: Is there anything holding you back from making a purchase? Y/N and ask for explanation.

    Next: Heuristic analysis. Analyze each of your key pages (the ones with most traffic or part of the sales funnel), and assess them for the following criteria:

    * Value proposition
    * Friction
    * Clarity
    * Noise and distraction
    * Urgency

    Also, make sure the site follows usability standards.

    Longer explanations of each here: http://conversionxl.com/how-to-increase-sales-online-the-checklist/

    Write down all of the things you identify as you need to check all the "findings" against site usage data - Google Analytics, mouse tracking tools.

    Google Analytics - need 3 full days to explain what to look for. I spend many consecutive days in GA to look for stuff when working on clients' websites.

    Two top things to check to identify underperforming pages:

    1) Make sure your sales funnel is defined and tracked. Funnel flow is always insightful.
    2) Use The Ring Model by Craig Sullivan. Ring model is a way to look at the ‘layers’ or ‘levels’ reached. This works for many (but not all) websites. It focuses on depth of engagement, not pages viewed.

    It’s similar to a funnel as it helps you see the key loss steps. The main point is to help you see the ‘big picture’ involved.

    The Ring Model helps you see where flow is “stuck”—where the traffic is not flowing down to the next level. The main benefit here is that it helps you see which layer of your website needs the most help.

    What you do here is you measure unique pageviews per layer / step. Lets say you run an e-commerce site. Your layers would be then: home, category / search, product, cart, checkout steps.

    If you had 300k product page views, 5k adds and 1k checkouts where would your problem be? Cart adds! If you had 300k product page views, 100k adds and 1k checkouts your problem is in a different place!

    Mouse tracking: Use SessionCam or Clicktale to check how far down people scroll, and if any unimportant stuff gets too little attention. Form analytics is also really great. Pro tip: spend 4-5 hours watching user session replay videos - you'll learn a ton.

    Prioritizing tests:

    Use PIE framework by WiderFunnel that ranks tests by 3 criteria: potential uplift, importance of the page and/or traffic to the page, and ease of implementation. Get details here: http://www.widerfunnel.com/conversion-rate-optimization/how-to-prioritize-conversion-rate-optimization-tests-using-pie

    • SE

      Sean Ellis

      about 6 years ago #

      Thanks Peep. Awesome answer! Shows how advanced you can go with this stuff when it's your full time focus - or when you can combine the learnings of a group of people in an agency focusing on it full time.

      I used Clicktale a while ago for session recordings but found that it didn't work with our site at the time. Do you use heat mapping at all? If so, has it been helpful for testing ideas?

      It's interesting to watch CRO transition from something a marketer dabbles in to more of a specialized science.

    • BP

      Brandon Pindulic

      about 6 years ago #

      Great advice Peep!

      I subscribed to ConversionXL last week. Looking forward to more insightful posts :)

    • AC

      Alex Cook

      about 6 years ago #

      Thanks Peep.

  • ME

    Marc Eglon

    about 6 years ago #

    Deep replies from Sean and Peep so far.

    Kyle Rush's Mozcon presentation is also worth watching. His work on the Obama campaign is impressive because of (a)the volume of traffic and (b) the gravity of the outcome.


    • BP

      Brandon Pindulic

      about 6 years ago #

      This looks great. Always interesting to see what people who’ve worked on Obama's and Romney’s campaigns have discovered due to the sheer amount of traffic they incurred around election time.

Join over 70,000 growth pros from companies like Uber, Pinterest & Twitter

Get Weekly Top Posts
High five! You’re in.