Leave a comment

Analytics tools are great for collecting data that’s easy to measure, and visualising it in beautiful charts. Sadly, this usually leaves you with more questions than answers.

Unlike other tools, analytics products are primarily read-only. You don’t go there to take action, you go there to get answers and insights so that you know to take action elsewhere.

These products should focus on the job of “answering questions and providing insights” for their customers, rather than falling into the category moat of offering the stock line/bar/pie chart with a time selector; that’s what everyone else is selling.

When traffic spikes, the question is Why? When conversions drop, the question is Why? When sign-ups flatline, the question is Why? Leaving your customers to do visual gymnastics as they try to piece together parts of your product is a bad experience.

Here are four improvements that would make your visualisations more about answers, and less about gradients and drop shadows

1. Annotate the data
2. Annotate the slopes
3. Exclude (or flag) incomplete periods
4. Enable projections

  • SC

    Shana Carp

    over 5 years ago #

    Prediction models require knowing a lot more about your business than what analytics software know.

    How much do you want your analytics software to know about you - should they be a warehouse for something else?

  • DA

    David Arnoux

    over 5 years ago #

    From my experience the biggest issues often come from teams missing the first step. I've seen this over and over: They've already got GA and Mixpanel installed but have no idea what they're looking for yet.
    Determining what metric(s) are important at your stage of development is key. Lean Analytic's OMTM (one metric that matters) or McClure's Pirate Metrics are often good starting points. Actually sitting down and determining these metrics/KPIs are is too often overlooked. If done correctly it makes data analysis so much easier. You know what you're looking for.

    As for "why" things are happening in your data, I would definitely recommend a healthy mix of exit surveys + asking users + inspectlet.com screen recordings to answer part of the question.

  • EW

    Eric Wu

    over 5 years ago #

    I would love to see the product improvements outlined in the post. I also like the challenge the author has about making the analytics platforms answer more of the WHY. But, I also agree with @shanac that real insights require much tighter integration with the business practices. As well as, a better more sources of data.

    However, most platforms like Google Analytics, Omniture, and WebTrends are really designed to be more industry agnostic (even though you can see the eCommerce roots). Plus they're only getting into pulling in external sources of data.

    If you look at a product like Chartbeat, it's designed with publishers in mind. So while they don't provide annotations in the way demonstrated in the post, there are some real insights it can provide (if set up properly). It can tell you what articles are trending (e.g. velocity) along with the sources. They also try to provide more natural language messaging by saying things like "Though popular, your audience spent less than an average of 30 seconds reading this article" or even flagging "Missed Opportunities". A competitor to Chartbeat in the publishing space is Parsely. It comes with a much heftier price tag, but some it does a few things better than Chartbeat.

    Diverging a bit from the specific nature of the post, I think it's an interesting question of how much of the WHY you'd want your analytics platform to provide.

    I think it's hard for a platform to tell you WHY something has happened if it doesn't collect data from other sources. It's also potentially a slippery slope when put into the hands of a novice or worse a novice business manager.

    For example, when I work with business analysts to figure out WHY something has happened, we definitely go to the analytics first to look at the general trends. But many times we'll have to ask questions our brand marketing team if there was a recent television, radio, or print campaign going on that might lead to unattributed flux in traffic. Or maybe there was a change in the Google algorithm that caused a drop, which could be signaled in analytics as an outlier, but what if you a seasonal business that sees a decline during that period. Or maybe there was a national / world event that caused some fluctuations across all your channels.

    I would love to see the day where we wouldn't need something that resembles the business intelligence architecture, where you warehouse and dump your various raw data sources together just so you can munge them back together. For me the killer analytics app would be something like Segment.IO + Machine Learned Insights + Tableau. In other words, a single drop in solution that would collect data from multiple sources, derive the insights, and nicely visualize them for your various business owners.

    • SC

      Shana Carp

      over 5 years ago #

      Tableau is not designed for ML insights. There are other BI tools that are better for this, which integrated with SAS/SPSS and R.

      There aren't a lot of modern BI tools that say, integrate with python, and treat project management, other kinds of data, and say a DMP with ease. (hell, there isn't a dmp for startups or startup publishers/content marketers yet)

      Though my question still stands - is your web analytics supposed to know that much about you, because that could really change how it tracks (I know at least one analytics platform that does thing that, and I do not like the fact that when you create new segments, they apply them backwards, because it is totally possible that this segment should only exist forward and not backwards. it makes me wonder if the raw integrity of the data they are tracking is preserved irrespective of the segments - and how do I look at that)

      Tracking, analysis, and action are actually three different issues that are closely related.

      Take an AB testing tool - it involves all three. You could assign more aspects of analysis to the computer - though many people are mentally uncomfortable with directly handing over segmentation to a computer sometimes, particularly if a test "fails". If so, should the computer act? Computers have faster reaction times than people for computation related activity.

      Maybe the AB test is closely strategic, and it involves opening up long term 6 month forward thinking answers. Then what?

      System design gets complicated for these three questions. Which is why people should stop complaining and start thinking about the fundamentals, and then design products around the fundamentals. Maybe in fact what we all really need is a small sized DMP.

  • AA

    Anuj Adhiya

    over 5 years ago #

    There's a great discussion in the original posts comments which are worth reading as well

  • DL

    Dylan La Com

    over 5 years ago #

    Yeah these are great suggestions from @destraynor

    GA's Intelligence Events + Alerting are actually pretty powerful, unfortunately they're separated from the other parts of the product and wrapped in an overly-complex interface. There's a lot of room for improvement there, though.

    • NK

      narek khach

      over 5 years ago #

      Yep, exactly! We've talked with a lot of product owners before refining our concept, and some of the biggest complaints with GA and Mixpanel is complexity, too all-over-the-place.

  • RR

    Richi Rich

    over 5 years ago #

    I think the bottom line of analytics is to find the source of conversion, and making them actionable. I agree that GA is all over the place but that is to accommodate all analytics gurus. I think you can run analytics based on 5 charts, conversion funnel,(2 of those), conversion sources, PPC performance, best performing pages and micro-goals(at least 2)

  • JS

    Jeff Sauer

    over 5 years ago #

    Couldn't agree more. Web Analytics tools could benefit immensely from the ability to provide context to reports/correct incorrect data.

  • DT

    Des Traynor

    over 5 years ago #

    Great discussion here, glad you all enjoyed the piece

  • NK

    narek khach

    over 5 years ago #

    Heatmapps will be tackling some important aspects of this.

    This article brings up some great issues around analytics tools, they are good at capturing data but then what? Startups, SMBs and enterprise all are looking for insightful intelligence, not just data.

    We are working on the product now (hope to have it ready by January) that aims to tackle that problem space. If anyone is interested you can sign up for early invites for Heatmapps (http://heatmapps.com/).

  • RS

    Robin Schwartz

    over 5 years ago #

    I do believe there's a lot more we could get out of traditional analytics and I love some of the suggestions in the article, but there are other options. I work for Appsee, (appsee.com) a mobile analytics company that provides VISUAL solutions. As mentioned in a previous comment, heatmaps can give insights, and that's something we do, but aside from that we actually have a feature that records actual user sessions and allows for video playback. So in the case of app makers, they are able to watch users interact with their app and determine where there are problems and what might not be working. Data is wonderful, but not if you can't do anything with it.

    Just thought introducing a different type of analytics tool would be helpful.