Mobile stress: Slower web pages lead to increased user frustration and lower engagement
I’ve been dying to write a post that takes a deeper dive into the research I presented at Velocity Europe last month, but wanted to wait until we had everything neatly packaged in a report for you to download. It’s here! So now I can write about it! WIN.
With just 40 minutes in my session, I wasn’t able to share all the findings from our three-part study, which blended electroencephalography (EEG) and eyetracking technologies with implicit response testing to track things like frustration, emotional engagement, and various aspects of brand perception on a moment-by-moment basis throughout a series of online transactions via smartphone.
Background: Why we engaged in this research
No one has ever engaged in this kind of testing. Why do it now? (I mean, other than the fact that it sounds cool and looks like a scene from The Matrix.)
When we were weighing our research options at the beginning of this year, we knew that we wanted to study people’s expectations and assumptions about using the mobile web. Our hypothesis was that shopping on smartphones comes with inherent stress, and that this stress worsens when pages hang. The question was how to explore this hypothesis in a way that could yield fresh new insights.
We considered commissioning a survey about mobile user expectations. We ended up scratching that idea for a number of reasons, the main one being that there are already numerous surveys in this area. I know this because I cite them all the time. Without discounting survey-based research, which I do believe has merit, there’s a problem with surveys: they can all too easily be dismissed simply by pointing out the (admittedly true) fact that what people say doesn’t always match how they feel or behave.
People also have a tendency to misremember past experiences, so after-the-fact reporting needs to be taken with a large grain of salt. I see this regularly with surveys about user expectations and performance. For every survey stating that users expect pages to load at least as fast on their phones as they do on their desktops, there are countless naysayers who claim that mobile users are realists who accept that mobile pages are, by their very nature, slower.
We decided that we wanted a study that cuts through the noise of opinion. EEG research is important and exciting because it imposes unprecedented rigor on testing our assumptions. It’s also valuable in that it allows us to measure people’s moment-by-moment reactions as they move through a site in real time.
Our approach
With this project, we knew we were pioneering new territory, and we knew we’d need outside expertise to help us do it. We assembled a team of global specialists from Seren (leaders in customer experience and service design), Neurosense (a global leader in implicit methodologies) and NeuroStrata (expert consultants in blending neuromarketing applications). Together, we developed an approach we dubbed the Emotional Engagement Measurement Process, which I described at the top of this post.
Participants were given standardized shopping tasks on four sites — John Lewis, Tesco, EasyJet, and Ryanair — using an iPhone. We studied participants on these tasks, both at the normal speed over Wifi and also at a consistently slowed-down speed (using software that allowed us to create a 500ms network delay). The participants did not know that speed was a factor in the tests; rather, they believed that they were participating in a generic usability/brand perception study.
From the EEG data, we were able to extract measures of frustration and emotional engagement for the browsing and checkout stages of both the normal and slowed-down versions of all four sites.
Key findings
Over the course of our study, we had five “aha” moments. We learned that:
- Mobile users are significantly affected by slow performance. A 500ms connection speed delay resulted in up to a 26% increase in peak frustration and up to an 8% decrease in engagement.
- Slow sites can seriously undermine overall brand health – across both desktop and mobile platforms.
- The nature and scale of the impact varies, depending on a number of factors (e.g., inherent strength/weakness of brand).
- Brands with already fragile consumer affinity are at higher risk.
- The greatest risk of page delays is to purchase intent — people’s desire to actually spend.
Today I want to focus on finding #1: Slow pages result in increased user frustration and decreased engagement.
At the outset of our testing, we established baseline readings of participants using the test sites at normal speed. We then introduced the 500ms network delay. Looking at our EEG data, we noticed two metrics that were affected the most by performance degradation: frustration and engagement.
First, let’s look at peak frustration levels:
Three out of the four sites saw increased frustration peaks when the site was slowed down. In general, John Lewis and EasyJet had higher frustration peaks during browsing when the site was slowed down, with John Lewis experiencing the highest peak, at 26%, during the browsing phase. Ryanair experienced its highest frustration peak, at 22.8%, during the checkout phase.
There are two primary tasks users perform on an ecommerce site:
- Decision-making (browsing)
- Form-filling (checkout)
Decision-making is the more cognitively demanding, and therefore leaves users more vulnerable to becoming frustrated if other elements (such as page rendering) slow them down. The browsing stage is where most of the decision-making occurs, therefore it is not surprising that this was where we were most likely to see increases in frustration. (The exception was Ryanair, which saw a higher increase in frustration at the checkout. But that’s a subject for another post, which I’m planning to publish next week.)
Next, let’s look at emotional engagement:
Emotional engagement is a positive metric. Three out of the four sites saw a decrease in average engagement when the site was slowed down. John Lewis customers were slightly less likely to engage with the browsing experience while the Tesco site was more affected during the checkout phase. Both browsing and checkout were affected on the EasyJet site.
Conclusion and takeaways
It would be great if there were a grand unified theory of performance and user perceptions. There isn’t. Every site is different, and performance issues affect every site differently. Depending on the page being looked it, the neurological impact of slower load times can be either heightened or offset by other aspects of the page, including layout, visual design, content, and the order in which page resources render.
What we can take away from this research:
- Performance has a clear and measurable impact on users at a pre-cognitive level.
- This early research validates the premise of neuroscientific testing.
- Given the accessibility and affordability of the technology used in this study, I predict that more and more companies will look to neuromarketing to help identify performance and usability issues in their own websites and applications.
DOWNLOAD: Mobile Web Stress: The impact of network speed on emotional engagement and brand perception
