One of my first experiences in witnessing the power of visualizing performance data was several years ago, when I started work at Strangeloop (which has since been acquired by Radware). We had been doing a proof of concept of our front-end optimization solution (then known as Site Optimizer, now called FastView) with a prospective customer. We had great results. Among other things, we cut start render and page load times by more than half on all the landing pages we tested on.
All the results — and bear in mind, these were fantastic numbers — were neatly presented in a set of waterfall charts with an accompanying Excel doc. But the customer was only somewhat impressed and still on the fence. Then someone on our team had the idea to generate timed side-by-side videos showing the “before and after” renders for each page. The videos were a hit. The customer immediately shared them with everyone on his management team. Much excitement ensued. We closed the sale. After that, videos became a cornerstone of our POC process.
In hindsight, this “aha” moment sounds like a no-brainer. We humans are visual creatures. Most of us respond better to data visualizations than we do to raw data. Yet it’s amazing how often we find ourselves defaulting to spreadsheets and waterfall charts.
While we can’t always replace our spreadsheets and waterfalls with awesome graphics, it’s exciting to see that more and more tool vendors are experimenting with ways to help us visualize our performance data and make it infinitely more persuasive. In this post, I’m going to highlight four tools I use all the time to demonstrate performance issues. (Not only are these great tools, they’re also my favourite price: free!)
What it does: Developed by Patrick Meenan, WebPagetest is a synthetic performance test that has become a staple for many folks in the performance community. It allows you to test any page on the web to measure a tonne of performance metrics, from first response to load time, all of which will be generated in a handy-dandy waterfall chart. But that’s just the beginning. You can customize your test for connection type, browser, location, latency, bandwidth, packet loss, and bunch of other variables.
But for me, the most exciting WebPagetest feature is the ‘Capture Video’ option, which generates excellent visualizations, chiefly videos and filmstrip views. WebPagetest lets you generate a timed video of your page render. Even more exciting, you can run tests of your pages alongside competitors’ pages (click on the ‘Visual Comparison’ tab), and generate side-by-side videos comparing your load times to your competitors’, like this:
You can also generate a frame-by-frame filmstrip view of the page render. This is super handy for getting quick insight into performance bottlenecks, as well as the user experience for people on slower connections. (e.g. Are people waiting ages for your feature content, which loads last?) Graphics like this are handy for embedding into reports when videos aren’t an option:
You can also run WebPagetest private instances to do all kinds of other cool stuff, such as test the impact of new tools and optimizations. We use WebPagetest heavily at Radware. If you’re not already using it, I strongly recommend that you check it out, play around with it, explore its many features (not all of which I’ve covered here), and make it part of your everyday toolkit.
When it’s useful: When you need to generate awareness (or outright fear) around the fact that your competitors are outperforming you. When you want a bird’s eye view of a page’s performance so you can figure out where the performance bottlenecks are. When you want to do A/B or multivariate testing of the impact of tools/content/other variables on load times.
What it does: Perfmap is one of those tools that seems so simple, so obvious, and so useful, you wonder why it never existed up till recently. Developed by Mark Zeman, it’s a free Chrome extension that generates a heatmap of page resources, along with the timing for when the image finished loading and the time (in parentheses) it took for the image to load. The legend attached to the bottom of the page shows timings for the full page load. (Side note: This entire post was inspired by Mark’s excellent Velocity talk, A Better Waterfall Chart.)
To demonstrate, here’s how Perfmap generates a heatmap for a page on this site:
Let’s take a closer look at the heatmap. Below you can see that the main image on the page took 640ms to load, and finished rendered at the 1183ms point. The coloured bar at the bottom of the page is a timeline indicator that also shows that the page’s total load time was 2189ms. On the live page, hovering over a coloured area on the heatmap moves the timeline indicator to show you when that image was fully loaded. Cool, right?
When it’s useful: When you need a quick, easy way to demonstrate to designers and content creators and other spreadsheet-averse individuals the impact that large, unoptimized images have on load times.
What it does: Born out of one of the many capabilities of WebPagetest, SPOF-O-Matic is another handy Chrome extension. It detects likely single points of failure (SPOFs) on any page on the web. It also allows you to simulate how a page would perform if specific third-party scripts were unavailable.
I’ll illustrate how it works, using TechCrunch as a guinea pig. First, clicking on the SPOF-O-Matic icon generates a list of potential front-end SPOFs:
Then clicking on the “Generate SPOF Comparison Video” link takes you to WebPagetest, which generates a side-by-side video showing what this page would look like if these third parties were non-responsive:
And you can also generate a filmstrip version of the video for embedding in documents:
When it’s useful: Third-party scripts are a huge potential performance pain. This doesn’t get talked about nearly as much as it should. SPOF-O-Matic is a useful tool for creating awareness about the threat that non-performant scripts present to your site.
What it does: Ghostery is an extension that’s available for pretty much every desktop and mobile browser. Interestingly, this tool was originally developed to give regular web users real-time visibility into the third-party pixels, beacons, etc. that track and gather their user data, and then allow users to block individual scripts. (It also makes it easy to re-enable them, if you change your mind later.)
I started using Ghostery as a companion to SPOF-O-Matic, to quickly identify all the third-party scripts on a given page. Here’s how it looks when I visit the same TechCrunch page I visited above:
But the folks behind Ghostery quickly realized that their app offers a lot of value to site owners as well, and they’ve created an enterprise version that does a lot of nifty, powerful stuff behind the scenes. With the largest tracker database on the web (1900+ trackers and 2300+ tracking patterns), the enterprise version gives site owners a real-time bird’s eye view, via cloud graphics — of all the externally scripts on any given page. At a glance, you can learn things like which external scripts are slowing down your pages, which are stale and linking to dead pages, and which are creating security holes.
For example, below is a graph that shows the entire third-party marketing cloud for Sears.com. You can clearly see the relationships between all the scripts, including which third-parties are making calls to other scripts, i.e. fourth-party and even fifth-party calls.
And below is the same marketing cloud, but this time depicting the real-time latency for each provider. At the time of this screen grab, you can see that MediaMath, Rubicon, and Integral Ad Science are all running slow. Clicking on the MediaMath bubble reveals that latency is 2060ms.
You can also click the ‘Timeline’ and ‘Tree’ tabs to get more conventional waterfall-style charts as well, but I love these cloud graphs for making a fast, dramatic impact.
When it’s useful: When you realize your site is overrun with third-party scripts and you’ve lost control (or are on the verge of losing control). When you’re serious about tackling third-party performance and security and need a tool that lets you track your entire third-party ecosystem. When you want/need insight into the potentially unauthorized fourth- and fifth-party calls your authorized third-party scripts are triggering.
As I said at the top of this post, data is much more persuasive when it’s presented visually. Wouldn’t it be great if more of our RUM tools could generate some or all of these kinds of visualizations? (Hint!) If you know any other tools that do a compelling job of visualizing performance issues, let me know.