Modelling with Google Analytics

I try to teach my students more than ‘stuff to do’ — I want to change the way they think about products entirely. Part of this involves showing them how to approach every business and product as a unique piece of machinery that requires exploration in order to gain understanding.

In order to simplify complex models or products using analytics, metrics or other abstractions of what is ‘actually happening’ — you need to start with what is ‘actually happening’ rather than the abstractions presented to you.

My students aim to understand more than ill-directed tactical approaches to the work of optimising flow and user experience within products; The tiny atomic working parts of a product are very important, as is the entire model of the machine at scale. More importantly, there is the qualitative aspect of how hormonally and emotionally — the product makes you feel. In each moment to moment with the product, how does this experience change?

There are micro and macro patterns in the ways that products, experiences, businesses and outcomes are shaped — and by understanding how they all work holistically, a truth is shared.

I often show students this watch and ask “The face of this watch is showing the wrong time. You are given these two pieces to fix. How do you make it so that it shows the right time?”

I get many answers, some more complicated than others and some more hilarious than they should be.

However, what they grasp after thorough discussion is that you cannot make the watch face say ‘anything you like’ unless you know how the machinery works. If you want to make it say the correct time — you need a reference time source of course — but you also need to know how all these little cogs and wheels work together. This is what many people miss — the little details AND the big picture.

So if you want to change the company dashboard — you need to decompose those metrics, break it down, see how they interrelate, magnify or support each other. What their ratios are. How one may magnify or diminish the importance of another.

Every little cog and wheel in here will contribute different amounts to what is displayed on the face of the watch. Some will provide a tiny contribution and others will be massively geared to impact some other part of the watch. It is both the ‘whole’ of the watch and the consideration of the ‘parts’ that is critical to understanding the product, in order to model and measure it correctly.

The truth is that the watch is just an example of a company dashboard showing KPIs and your job is to change the metrics. How could you go about doing that, without understanding how the underlying machinery worked?

You’d just be sticking a screwdriver in there and randomly trying stuff — and that’s more common in companies than you’d realise. The scientific term is called ‘Guessing’.

The more advanced agencies that I’ve worked with, follow a distinct process that’s unique. Of course, every one of them would argue that ‘their way’ or ‘the methodology’ that they followed was responsible for their success with clients. However, despite the differences, there are common attributes to agencies and practitioners that get better outcomes and faster.

The answer is patterns — and prioritisation. I’m going to show you some models that came up on working for client sites for the last 5 years. Don’t worry — this isn’t real data. I’ve made something up that isn’t like reality but explains perfectly the thought process and what we subsequently found.

I will give you some completely different patterns that we spotted — and how we arrived there. Then I’ll explain why these are important — and show you how we broke one of these models down further.

In reality, I’m using automated tools, analytics, discovery and insight methods to rapidly narrow down the range of what I’m looking at. The crucial nature of finding what really matters (knowing your baseline experience) helps improve the product experience for more people, faster and with greater impact on the company metrics and goals.

The four examples will be:

A micro site for a Non Profit

A subscription based product

An expert consulting service

An ecommerce site

Before I write up each example, I wanted to explain that these models were not created randomly — there was a purpose to them. To find stuff.

Whether we were figuring out that a bunch of landing pages really sucked (without knowing why) or that a huge chunk of traffic left on one form field (without knowing why) or that 5 pages drove 89% of revenue for one funnel or that 9 landing page templates drove 77% of new visitor orders — we were guided by more than data.

By doing the groundwork to prepare for modelling — including walking the site, tagging and key journeys and auditing the google analytics configuration — we provided a foundation for understanding all the data. We also figured out what data was reliable or unreliable for making subsequent sketched models.

Rapid Modelling then helps sketch out ‘views’ of a site (or parts of the site or product) that might highlight differences, questions, outliers, things that are higher, things that are lower — things that just look weird, bad or good.

Without the final part — adding in the qualitative side of the modelling — we can’t really think about the experience properly. Let’s just examine why we do this — is it because I’m a UX guy? No. Because it works.

If I test a website and there’s a massive UX problem or device flaw, I ought to be able to see this impacting the data. Imagine I find that iPhones can’t use a part of the site navigation. I can then go and split the data to find out what impact this might be having. I can check more thoroughly to see if it’s just iPhones that are impacted or all mobile phones.

If I see an analytics setup and there’s a massive UX problem or device flaw in there but hidden from me, I ought to be able to see this impacting the data. I might find that a landing page template converts at 1/10th of another group of pages — for no obvious reason. After checking, I find that mobile devices are the only device class impacted. I then do user testing and handset testing, where I discover a heap of problems that are driving the numbers.

If you use qualitative research, it will focus and sharpen your analytics inquries. If you use analytics data, it will focus and sharpen where you spend time doing qualitative research. Voila! A perfect symbiotic relationship.

In this series, I’ll be explaining one of the best ways to make complex sites simple, to reduce the sheer noise and volume of data to something that will cause actions to be taken and questions to be asked.

Modelling is something I didn’t realise I used until I started recording how I ‘find stuff’ inside the analytics data and qualitative tools and research I have access to. It was an instinctive process but I’ve managed to figure out all the main approaches I pick from — to apply to products and figure out useful things.

The best agencies are using automated modelling, data pulls and analysts who have time to figure stuff out (because everything else is automated). They hire enough analysts so they can automate their own work for continual improvement. This seems like common sense but in many companies, the analysts are swamped with running reports for other people, not figuring stuff out.

Lastly, don’t worry if when you make models or look into things, that there’s nothing there in the end. You have to go down a helluva lot of empty rabbit holes to find the one that contains what you’re looking for. It’s the looking that counts and how you do it using smarter and faster automated tools.

I’ll help you to understand how to break down some common site models and design your own, using simple tools, reports and a pencil! This was the favourite module for my students last year, so I hope this will help you practice and acquire skills to help you do this in your head.

I’m off to Measurecamp tomorrow to run my first modelling workshop format — so I’ll be back soon with an update! I’ll leave you with this wonderful quote from Avinash Kaushik:

“The very best analysts distil, rather than dilute. The very best analysts focus, when most will tend to gather. The very best analysts display critical thinking, rather than giving into what’s asked. The very best analysts are comfortable operating with ambiguity and incompleteness, while all others chase perfection in implementation / processing / reports. The very best analysts know what matters the most are not the insights from big data but clear actions and compelling business impact from usually a smaller subset of key data”

C.