Leave a comment
Get the GH Bookmarklet

Ask GH

I'm currently looking to increase the number of users that find success with our product. I've been conducting qualitative research through customer interviews, during which I'm asking them what was the "a-ha" moment so that we can redesign the product and front load that moment but haven't been able to reach a conclusion yet. With that in mind and drawing inspiration from presentations about how Facebook and Twitter realised they'd be able to retain users if they hit a certain number of followers/interactions on their apps, I am analysing the first interactions new users have with the product manually (viewing their timeline of events on kissmetrics). When analysing user data to try and uncover the a-ha moment in your product, did you apply any statistical methods/software tools to do this, or did you just plow through the data manually?

  • SH

    Samuel Hulick

    over 4 years ago #

    This doesn't sound like an onboarding problem, it sounds like a product/market fit problem -- if you don't know what's amazing about your product, hoping your users will tell you is probably a poor strategy.

    My recommendation is for everyone at the company to be *very* clear on the specific way in which you help people succeed, and aligning your entire product experience around it.

    Really, the "aha" moment should come before people even sign up -- if it hasn't clicked before they've pulled the trigger, what's driving them to do so in the first place? Relying on curiosity as a motivator isn't playing to your strengths, and hoping people will figure out a product's value by learning its interface is a really risky proposition.

    Get clear on how you make people better, then clearly and credibly let them know *before* asking them to get started. Once that's in place, hypothesize what the best "quick win" you can provide in a single sitting is, make sure it's consistent with the value you provide, and test/iterate from there.

  • SE

    Sean Ellis

    over 4 years ago #

    One of the easiest ways to see it is through user testing for new qualified potential customers. Watch them through the onboarding process and literally wait for them to say "wow, this is (looks) great"...

    Another thing that I do is use the "how would you feel if you could no longer use this product" and filter by actions that people have taken. When I see spike in the percentage that would be "very disappointed" I get indications of the aha moment.

    Hope this helps.

    • AL

      Arsene Lavaux

      over 2 years ago #

      To build on the first point @Sean brought up, I'd like to share a real-life example of user testing that I conducted on a very early version of a mobile app I designed and coded. Important to note that I look to target my assumed "must-have" segment to get a feel for product-market fit propensity and iterate my product design accordingly: https://www.usertesting.com/v/79e7855f-1fc5-4deb-a3ed-484eba375ed9?back=%2523study_1672475%23vp-tab_answers#/answers

      A few seconds into the test, you see and hear the "aha".

      Note that I also conduct a "post first experience" survey, and although statistically it's not meaningful to do a product-market fit survey similar to what Sean mentioned in his second point above, instead, I am getting a directional feel for net promoter score "how likely are you to recommend to a friend".

      Based on some other qualitative information you can get with such online and what I call "bartender testing" (go to a bar, or whatever, identify your assumed must-have target segment and put your product without any explanations in their hands, just watch what they do, what they understand) real-life testing, in my experience, you can really zero in on the critical "aha moment". You design for it, you focus your iterations.

      This, in my opinion, is one of the major keys to sustain growth down the road.

      Hope this helps!

  • LT

    Luke Thomas

    over 4 years ago #

    One thing I've been doing recently and found a ton of success with is asking the standard NPS survey "On a scale of 1-10, how likely are you to recommend _____________ to a friend or colleague?" and then after asking "what caused you to answer this way?"

    It accomplishes two things:

    1. For fans of the product, you can find out exactly why they gave you a good rating (and they typically mention the "aha moment" proactively.)

    2. For people who don't like the product, you can find out what you need to improve.

  • AS

    Alicia Shiu

    about 4 years ago #

    I realize I'm a little late -- just saw this discussion. I agree with everyone who says you should have a clear view of what value you're providing your users, and have some hypotheses going in as to what your 'A-ha' moment is. While the previous comments mostly mention qualitative data (surveys), they don't talk much about the data you can get from user analytics, i.e. tracking actions that your users take (which I take you're using kissmetrics for). I recently wrote a post detailing a pretty simple statistical method to look for your "7 friends in 10 days" success metric. Short version: look at user retention, and see which actions differentiate retained/engaged users vs. those who stop using your app using a binary classification test. Details here: http://blog.amplitude.com/2014/07/29/find-the-key-to-your-apps-growth-without-an-army-of-data-scientists/

    Hope it's helpful!

  • CR

    Chirag Rajasab

    over 4 years ago #

    Hmm... there's no formula to find that. The other day I was talking to the Growth lead of Evernote, he's still looking out for '7 friends in 10 days’ equivalent for Evernote.

  • AT

    Alan Tsen

    over 4 years ago #

    My 2 cents, I think generally looking for the proverbial growth needle in the data haystack can really skew the direction you take. I think early on you need to have a clear hypothesis as to what you think the amazing thing about your product is and test that.

    The 'Facebook 7 friends in 10 days' thing is really a manifestation of the behaviour driven by the type of engagement the product is meant to be driving - social interactions. In this regard, I think the growth team would have focused on the core of the product's value and then looked at what was driving the behaviour - not vice versa. Good luck with the product!

  • JB

    Jon Bishop

    over 4 years ago #

    I've both used statistical methods and plowed through data manually. I only use statistical analysis if there's enough data overall and, even when I do, I always start manually to get a feel for the data (how clean is it, is there enough data with each action I'm looking at, etc.)

  • MG

    Masha Grin

    about 4 years ago #

    A bit late but still... For freemium startups the only validation of the product value is a customer’s willingness to pay. Assuming that value recognition is an a-ha moment all you need to do is to collect characteristics of your paying users (use your preferred tracking solution). Who are they, what were their actions, achievements, ‘movements’ just before they start to pay? Then you need to overlap sets to find intersections. This will give you understanding of common actions and characteristics that are probably make people pay. So, you will be able to rebuild the site and your app’s welcome scenarios to lead users to take particular actions.

  • AS

    Abrar Shahriar

    over 2 years ago #

    1. Social Media Track down: You can track down the user behavior from your social media account (Facebook, Twitter, Linkedin,etc.) The content you get most likes, comments or negative feedback.

    2. Find which offer gets the most reaction among the all products.

    3. You can also track down user reaction from your blog commenting section.

  • EF

    Ed Fry

    about 1 year ago #

    Just to add to @sean @samuelhulick @lukethomas thoughts on NPS, messaging alignment, and user testing -

    We researched and analysed how companies like Typeform, DigitalOcean, Appcues, AdEspresso and more find their "aha!" moments and identified three trends:

    1. Raw product usage (quick start, easy. Less than 15 minutes). Guess at a trigger.

    2. Quick regression analysis (identify more and better opportunities. An hour or less). Remember, specificity allows you to hone in your content on what matters and convert more. Find a handful of triggers.

    3. Data science and artificial intelligence. Find many triggers, and deliver precise content (ongoing, five-figure spend)

    Since these increase in complexity and potential return, start with the simplest. The basis of the more advanced techniques comes from the simpler techniques anyway. Start with a guess.

    As your product qualified leads project continues, you can move onto progressively more complex analysis (and also earn the buy-in for the resources you need to develop your campaign further).

    You can read more about how to do the analysis, get buy-in and what other teams found on finding their "aha!" moments here: https://get.hull.io/complete-guide-pqls/chapter5/

  • KI

    Kevin Indig

    about 1 year ago #

    There's a neat tool called inspectlet (I'm not affiliated or anything). They provide pretty good session tracking of your users. That allows you to understand how they use product features and which ones they keep coming back to. What's nice about that approach is that you see how long it takes people to get to the a-ha! moment, which you can then optimize and measure.

Join over 70,000 growth pros from companies like Uber, Pinterest & Twitter

Get Weekly Top Posts
High five! You’re in.