Do These 3 Things Before Watching Session Replays: Advice from CRO Experts
Jack Maden | February 28, 2017
Jack Maden | February 28, 2017
This is part two of a series in which we look at how to get the most out of session replays. By the end of this series, we hope you’ll have gained three things: a logical process for session replay analysis, steps to cut down on the number of sessions you have to watch, and techniques for watching a balance of sessions to ensure you investigate your project from all angles.
Last time, we asked CX and CRO experts why session replay is becoming so important to improving digital experiences. In this article, we focus on the key steps you and your team should take before watching session replays.
Watching back your first few session replays is a revelatory moment.
Goodness gracious, you think, people actually are using our website.
They’re not just numbers in a report. They’re not just metrics! They’re actually there, with their frantic mouse movements, their impatient scrolling, their darting, wildly unsuccessful attempts to navigate your lovingly crafted website.
You sit there with voyeuristic shame and excitement. You devour session after session. You can’t get enough of it.
Sooner or later, however, the novelty begins to wear off.
Oh look, here’s another struggler attempting to fill out our checkout form. Go on, nameless user! You can do it! Just a couple more field– oh. You’ve abandoned. Like all the others.
Except, no, not like all the others. It’s more complicated than that. Some people are filling out the form just fine. Others seem to lose interest halfway through.
So is this form a problem or not?
Gloomily, you load up another session, thinking all is futile.
It's great fun initially to watch back sessions, seeing how individuals are interacting with your site. But in the excitement it’s easy to get carried away and not actually set up a process that allows you to benefit from doing it.
It can seem overwhelming to glean insight from hundreds or even thousands of individual session replays, taking place on different devices, browsers, from different parts of the world, and with different intentions.
So how can you go about intelligently systemizing the sessions you watch to gain useful, practical, actionable insights?
Getting the most out of session replays starts with being prepared. Here’s the three things you should do before getting stuck in to watch your first batch of sessions.
1. Stop watching random session replays
That's right: the first step to get the most out of session replays is to stop watching session replays! This may seem counterintuitive, but watching back sessions without a clear reason for doing so is a needless time sink, and you're probably not going to find anything fixable or interesting by simply watching back sessions at random.
Theresa Baiocco Farr is founder of ConversionMax. She specializes in helping mid-sized businesses increase their online revenues with conversion rate optimization (CRO). She’s a repeat speaker at conferences such as Conversion Conference and Pubcon, and is regularly featured on Unbounce, the Wall Street Journal and Marketing Day:
"The worst thing you could do when getting started with session replay is to try to watch all the videos, or even choose sessions at random. You’ll never manually find any patterns in user behavior and you’ll waste a ton of time."
Johann Van Tonder is the COO at AWA digital, a leading international Conversion Rate Optimisation agency specialising in ecommerce, and co-author of E-commerce Website Optimization, the world's first how-to business book specifically for ecommerce websites:
"Avoid aimlessly replaying everything from beginning to end. In the discovery phase, you can look at a couple of clips in each area of the site just to get an overview. But the real power of the tool is unlocked once you've identified areas to investigate further."
2. Establish what you're investigating
Session replays should not be treated as a way to establish issues. Rather, they should be utilized as a way of scrutinizing known problem areas.
What’s the project you’re currently working on? Have you been tasked with investigating why website sign ups have fallen year-on-year? Why revenue is down for a certain demographic?
If not working on a project, then initiate one. What area of your website do you want to focus on? The checkout flow? The sign up process? Content engagement?
Come up with a set objective or an overarching goal for your analysis. Once you contextualize your investigation, everything you do becomes more purposeful.
"Watching session replays and analyzing what exactly is going on can be a very time-consuming task, especially when you have bulk replays and you don’t know where to begin. It’s very important that you have clear focus and use your session replays strategically.
"It's very important that you have clear focus and use your session replays strategically."
"The first step is to set up a focus strategy on why you want to look at session replays, when you should look, and what you want to find. Structuring the session replays in practical uses, such as finding bugs or finding device specific issues, will give you a focus and can help you go through the replays you need to watch and which ones won’t have any value to you."
"Session replay tools can be daunting in the amount of information they provide, a lot of it qualitative.
"It’s therefore important to ensure that all your efforts are geared directly towards the metric you are trying to improve – that’s the quantitative measure, and everything should be tied back to it. The objective should be to generate insight on to how to improve specific goal conversion, and that objective should be broken down to a clear and manageable flow."
3. Decide on a note-taking process that is logical, repeatable, and shareable
When you have a set objective or investigation, document it.
'Is it time to watch sessions yet?' you stamp your feet, impatient.
No, dear reader, it's not. Having a way to organize your observations from session replays is essential. Otherwise you will fall into the same old trap of blearily watching session after session, becoming numb to visitor behavior, and getting nothing from it.
If your session replay tool offers filtering, commenting and tagging – like Decibel Insight’s does – then organizing sessions can all be done in-app, minimizing the management issue.
If, however, your tool does not offer this, or you simply want to have an external record, then an old-school spreadsheet is a good way to go.
Either way, you’ll want to organize sessions in logical ways that everybody using the tool will be instantly familiar with. It might be a good idea to sit down as a team to decide on naming conventions that work for everyone. How will you label problems? As well as their severity?
One system that can work is ‘raw’ to ‘logical’. When tasked with watching a batch of sessions, make notes of any raw observations and insights you see. Then quantify them into the agreed system.
"The downside to session replays is that because it’s so easy to gather a large amount of recordings, you will have many that provide little to no value and take a tremendous amount of your team’s time. If you’re not recording sessions for a specific reason, e.g. how visitors behavior on your checkout page, you’re left with a lot of noise to start making sense of.
"Establish some baseline tags for the videos that you can apply while watching them, because it can take a while for patterns to emerge in the replays."
"In my experience the most essential way to control for this is tagging your videos. Establish some baseline tags for the videos that you can apply while watching them, because it can take a while for patterns to emerge in the replays. Simple examples are tagging videos where conversion goals are completed or abandoned. I like to tag videos where it looks like the visitor had a problem or if an error of some sort was encountered.
"As you notice different ones, you can begin tagging them ‘error-01’, ‘error-02’, etc so you understand how often this is occurring and evaluate more deeply later. There are several things you could tag, so the key is to know what tags you will use before starting to review them."
So there you have it, key steps to take before watching session replays: stop watching sessions at random, establish what you’re investigating, and clarify a note taking system with your team in which you can track, gather, and easily share insights.
Now we’ve clarified what to do before, next up in the series we’ll be looking at the best place to start watching sessions. Stay tuned!
If you can’t wait until the next post and want guidance now, download our free session replay ebook, packed full of industry insight and expert advice.