Metrics, metrics everywhere (but where the heck do you start?)

Metrics, metrics everywhere
(but where the heck do you start?)

@tameverts @cliffcrocker
#velocityconf

Who cares about performance today?
How do I measure performance?
How fast am I?
How fast should I be?
How do I get there?

Start render DNS TCP TTFB
DOM loading DOM ready Page load Fully loaded
User timing Resource timing Requests Bytes in
Speed Index Pagespeed score 1s = $$ DOM elements
DOM size Visually complete Redirect SSL negotiation

“47% of consumers expect a web
page to load in 2 seconds or less.”
Akamai, 2009

1s = $27M DNS
144ms
Start render
1.59s
Hero image render
2.01s

Androiddevicefragmentation
OpenSignal,August2015

How do we measure UX for mobile?

https://gist.github.com/larahogan/b681da601e3c94fdd3a6

Technology for collecting performance metrics
directly from the end user’s browser
Involves instrumenting your site via JavaScript
Measurements are fired across the network to a
collection point through a small request object
(beacon)

What makes RUM great
 Always on
 Every user, every browser, everywhere
 Able to capture human behavior/events
 Only getting better

Questions your RUM data can answer

What are
my users’
environments?

How do visitors move
through my site?

How are my
third-party scripts
performing in
real time?

What impact does
performance have
on my business?

Uses automated agents (bots) to measure your
website from different physical locations
A set “path” or URL is defined
Tests are run either ad hoc or scheduled,
and data is collected

What makes synthetic monitoring great
 Rich data collected (waterfall, video,
filmstrip, HTTP headers)
 Consistent “clean room” baseline
 Nothing to install
 Doesn’t require users / ability to
measure pre-production and
competition

Questions your synthetic data can answer

How do I compare to the competition?

How does the
design of my
pages affect
performance?

How does the newest version of my site
compare to previous versions?

How well am I sticking to my performance budget?

What if my third parties fail?
Original: 3.5s
SPOF: 22.7s

38© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Why are these numbers so different?
I don’t trust your data. Your numbers are wrong.
How are you calculating page load time?
I can’t share two sets of numbers with the business.

“But it loads so much faster for me!?!”
2015 Macbook Pro
Warm browser cache
FIOS
X86 – Windows 7 VM
Completely cold cache/DNS
Throttled bandwidth

http://soasta.io/perftimings

Browser support for NavigationTiming
http://caniuse.com/#feat=nav-timing

48© 2014 SOASTA CONFIDENTIAL - All rights reserved.
Network operations
Front-end developer
Marketing and site operations
Designer
C-level

Use case: Measure
network performance

I need visibility into…
 issues with authoritative DNS servers
 impact of denial of service attacks
on end users
 efficiency of connection re-use
 tier 1 connectivity issues (load balancer,
web server)

Measuring DNS and TCP
function getPerf() {
var timing = window.performance.timing;
return {
dns: timing.domainLookupEnd -
timing.domainLookupStart};
connect: timing.connectEnd - timing.connectStart};
}

What’s with all those zeros!
 Connection reuse
 DNS caching
 Prefetching

Focus on higher percentiles
85th percentile
Median (50th)

Use case: Measure
front-end browser events

How do I…
 understand the impact of back-end
versus front-end on page speed?
 investigate how DOM complexity affects
performance?
 measure a “fully loaded” page?

Start render DNS TCP TTFB
DOM load event DOM ready Page load Fully loaded
User timing Resource timing Requests Bytes in
Speed Index Pagespeed score 1s = $$ DOM elements
DOM size Visually complete Redirect SSL negotiation

Isolate front-end vs. back-end

Isolate front-end vs. back-end
return {
ttfb: timing.responseStart - timing.connectEnd};
basePage: timing.responseEnd - timing.responseStart};
frontEnd: timing.loadEventStart -
timing.responseEnd};
}

Measuring the critical rendering path

Investigate DOM events
return {
DLoading: timing.domLoading –
timing.navigationStart};
DInt: timing.domInteractive –
DContLoaded: timing.domContentLoadedEventEnd –
timing.navigationStart;
DContLoadTime: timing.domContentLoadedEventEnd –
timing.domContentLoadedEventStart};
DComplete: timing.domComplete -
PLoad: timing.loadEventStart -

Understanding critical rendering path
 DOM Loading – browser begins parsing initial
bytes of the document
 DOM Interactive – doc parsed, DOM has been
constructed
 DOM Content Loaded – No blocking style sheets
 DOM Complete – All processing complete, all
assets downloaded
https://developers.google.com/web/fundamentals/performance/critical-rendering-path

2618 DOM nodes
86 DOM nodes
Visualizing DOM complexity

Use case: Measure
object-level performance

I need to understand…
 how third-party content affects my
performance
 how long a group of assets takes to
download
 how assets served by a CDN are
performing

Browser support for Resource Timing
http://caniuse.com/#feat=resource-timing

Cross-Origin Resource Sharing (CORS)
Start/End time only unless Timing-Allow-Origin
HTTP Response Header defined
Timing-Allow-Origin = "Timing-Allow-Origin" ":" origin-list-
or-null | "*"

ResourceTiming
var rUrl = ‘http://www.akamai.com/images/img/cliff-crocker-
dualtone-150x150.png’;
var me = performance.getEntriesByName(rUrl)[0];
var timings = {
loadTime: me.duration,
dns: me.domainLookupEnd - me.domainLookupStart,
tcp: me.connectEnd - me.connectStart,
waiting: me.responseStart - me.requestStart,
fetch: me.responseEnd - me.responseStart
}
Measuring a single resource

Other uses for ResourceTiming
 Slowest resources
 Time to first image (download)
 Response time by domain
 Time a group of assets
 Response time by initiator type (element type)
 Cache-hit ratio for resources
For examples see: https://github.com/lognormal/beyond-page-metrics

Using Resource Groups
PLT impact not due
resource groups
PLT impact
correlates with
improvement
from image domains

Use case: Measure
the user experience

I just need to understand…
 when users perceive the page to
be ready
 how long until my page begins
to render
 when content above the fold is visible

The fallacy of “First Paint” in the wild
 Support for First Paint is not standardized between browsers
 Metric can be misleading (i.e. painting a white screen)

First Paint is not equal to Start Render!
Chrome – “First Paint” True Start Render

UserTiming Interface
 Allows developers to measure performance of
their applications through high-precision
timestamps
 Consists of “marks” and “measures”
 PerformanceMark: Timestamp
 PerformanceMeasure: Duration between
two given marks

Browser support for UserTiming
http://caniuse.com/#feat=user-timing

Measure duration between two marks
performance.mark(“startTask”);
//Some stuff you want to measure happens here
performance.mark(“stopTask”);
//Measure the duration between the two marks
performance.measure(“taskDuration”,“startTask”,“stopTask”);

How long does it
take to display
the main product
image on my site?

Record when an img loads
<img src=“image1.gif” onload=“performance.mark(‘image1’)”>
For more interesting examples, see:
Measure hero image delay
http://www.stevesouders.com/blog/2015/05/12/hero-image-custom-metrics/
Measure aggregate times to get an “above fold time”
http://blog.patrickmeenan.com/2013/07/measuring-performance-of-user-
experience.html

Adoption of
UserTiming
for top 25
shopping sites
(Global)
Nope
User
Timing
Sales

How do I measure performance
when the onload event loses
relevance?
Use case: Measure
performance of SPAs

http://www.ryan-williams.net/hacker-news-hiring-trends/2015/october.html

What is a SPA?
• Single Page Application
• HTML loads once
• Fluid user experience
• Can be great for performance
• Hard to measure

onload event
visible resources

Measuring SPAs
• Accept the fact that onload no longer matters
• Tie into routing events of SPA framework
• Measure downloads during soft refreshes
• (support in boomerang.js for Angular and other
SPA frameworks)

Use case: Measure
business impact

I need to understand…
 how performance affects business KPIs
 how our site compares to our
competitors

http://www.soasta.com/blog/mobile-web-performance-monitoring-conversion-rate/

2% increase in conversions
for every 1 second of improvement

Cut load time by 80%
83% traffic increase
32% increase in time on
site
108% increase in
interaction rate with ads

So what?
You must look at your own data.

Not all pages are created equal
For a typical
ecommerce site,
conversion rate
drops by up to 50%
when “browse”
pages increase from
1 to 6 seconds

Not all pages are created equal
However, there is
much less impact
to conversion
when “checkout”
pages degrade

What is the Conversion Impact Score?
The Conversion Impact Score (CIS) is a relative score that ranks page groups
by their propensity to negatively impact conversions due to high load times.
For each page group, the Conversion Impact Score is calculated using the
proportion of overall requests that are associated with that group, along with
the Spearman Ranked Correlation between its load times and number of
conversions. The Conversion Impact Score will always be a number between
-1 and 1, though scores much greater than zero should be very rare. The more
negative the score, the more detrimental to conversions that high load times
for that page group are, relative to the other page groups.

TL;DR
The Conversion Impact Score answers this question:
How much impact does the performance
of this page have on conversions?

http://www.soasta.com/blog/website-monitoring-conversion-impact-score/

Create a performance budget
Setting a Performance Budget
http://timkadlec.com/2013/01/setting-a-performance-budget/
Performance Budget Metrics
http://timkadlec.com/2014/11/performance-budget-metrics/

Set meaningful, page-specific SLAs

Chapter 8:
Changing Culture
at Your Organization

Free download (until November 4)
http://soasta.io/time
ismoneybook

performancebacon.com
performancebeacon.com

Metrics, metrics everywhere (but where the heck do you start?)

More Related Content

What's hot (20)

Viewers also liked (19)

Similar to Metrics, metrics everywhere (but where the heck do you start?) (20)

More from SOASTA (20)

Recently uploaded (20)

Metrics, metrics everywhere (but where the heck do you start?)

Editor's Notes