Leave a comment

Starting at 11am EST our startup's homepage is going to be a testing ground for the Welcome Mat.

This approach riffs off a popular article my boy Mattan Griffel wrote, that Andrew Chen later reposted the article under the title: The Highest ROI Way to Increase Signups: Make a Minimal Homepage.

What's funny is, despite having aggressively tested and proven this approach for other consumer startups I've worked with, for our own startup which is a B2B/B2C hybrid, we've hesitated to implement it. The risk being we lose out on the SEO rich keywords on our site, which has helped drive in the thousands of page views per day for us.

This said, I recently started messing around with SumoMe's Welcome Mat on ProductHunt. The app launched to near 500 upvotes last month. Looks a lot like a version of the minimal homepage right?:

To try and prove this makes sense, over at least the next 24 hours, I'll be live blogging an A/B test we've hacked together on our homepage using Optimizely, SumoMe and MixPanel.

I'll be including as much data and learnings as they come in, BUT I'd love to here your thoughts here or below the post.


Early promising results. For just the Welcome Mat, seeing a 9-10% conversion off of 146 views:

This is in addition to any sign ups we get after the Welcome. 

****SECOND UPDATE (11am-6pm)*****

As expected, that initial high conversion rate is starting to stabilize, and has come down a bit to just under 7%. 

Also, I realized one slip up: I'd set our webinar pop up to display after 5 seconds, not on exit intent.

What this means is that for if you've seen our Welcome Mat, you missed our exit intent pop up. This is definitely a missed opportunity that'd improve our overall conversion rate. No biggie because:

For far we've brought in 3X more webinar sign ups with the Welcome Mat than without.

Here’s how the results are looking (without bringing in other sign up data which I'll do tomorrow):

You’ll notice, the conversion rate for a webinar sign up is only slightly higher with the Welcome Mat. This makes sense given the copy and the offer are almost identical.

What appears to be happening is although the conversion rate is similar, we have a lot more (217 versus 80) people seeing the Welcome Mat compared to the pop up over a similar period of time.

Although we’re still on our way for the conversion rate difference to be statistically significant:

Statistical significance only matters if we're comparing conversion rates. By starting off strong with a Welcome Mat, we're able to definitively reach and convert more people. In this case, 3X more.

Tomorrow, I’ll do a final check in with some analysis that looks into total sign ups for the day.

****THIRD UPDATE (11am-11pm)*****

After 24 hours the results of the experiment appear very promising. We’re seeing a 2.6X improvement in webinar sign ups, and 40% lift in total sign up rates the Welcome Mat.

Check out the below video that goes into a little more detail for us as well as next steps. I also dig a little into our quick and dirty A/B testing implementation with Optimizely, MixPanel and SumoMe which saved time in getting a valid test live:

Let me know your thoughts! Is there anything we should test that we can learn from together? Let me know what you think in the comments below.

  • NK

    noah kagan

    over 5 years ago #

    Hadn't really thought of anyone doing this but curious how the results turn out.

  • NE

    Nat Eliason

    over 5 years ago #

    Cool experiment, thanks for the plug Conrad!

  • SC

    Simon Cave

    over 5 years ago #

    Great to see you doing this experiment

  • II

    illoh ifeoma

    over 5 years ago #

    I've used the welcome mat, and I find that it killed engagement with the page.
    As it's designed to fall over the home page, people don't scroll deeper and get to the home page juicy stuff.
    Using the welcome mat truly improves signups at the risk of killing engagement with the page. It's all now based on your goals.
    You want more signups or you want deeper engagement with your content?

    • CW

      Conrad Wadowski

      over 5 years ago #

      How have you seen it hurt engagement on the page for you? Would be great to hear what you've learned. Engaging with as many folks over email has been quality engagement for us.

  • SP

    Steven Pesavento

    over 5 years ago #

    Great, experiment and thanks for keeping us up-to-date as the test was in session! Kudos