Date post: | 18-May-2015 |
Category: |
Technology |
Upload: | philip-tellis |
View: | 1,575 times |
Download: | 3 times |
Why measure?boomerang
data data data
Measuring the web with boomerang
Philip Tellis / [email protected]
Boston Performance Meetup / 2010-09-16
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
$ finger philip
Philip [email protected]
@bluesmoonyahoogeekhttp://bluesmoon.info/
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
The slow webMeasurementsMeasuring with javascript
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
The slow webMeasurementsMeasuring with javascript
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
The slow webMeasurementsMeasuring with javascript
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
The slow webMeasurementsMeasuring with javascript
Less than 20% of page load time is something we can measureand fix during development
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
The slow webMeasurementsMeasuring with javascript
It’s what we can’t control that bites us
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
The slow webMeasurementsMeasuring with javascript
Too many variations
browsers
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
The slow webMeasurementsMeasuring with javascript
Too many variations
plugins
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
The slow webMeasurementsMeasuring with javascript
Too many variations
OSes
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
The slow webMeasurementsMeasuring with javascript
Too many variations
viruses
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
The slow webMeasurementsMeasuring with javascript
Too many variations
antiviruses
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
The slow webMeasurementsMeasuring with javascript
Too many variations
microwaves
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
The slow webMeasurementsMeasuring with javascript
Too many variations
baby monitors
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
The slow webMeasurementsMeasuring with javascript
Too many variations
naughty neighbours
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
The slow webMeasurementsMeasuring with javascript
Too many variations
file shares
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
The slow webMeasurementsMeasuring with javascript
Too many variations
governments
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
The slow webMeasurementsMeasuring with javascript
Too many variations
rodents
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
The slow webMeasurementsMeasuring with javascript
Too many variations
Try simulating all that in the lab!
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
The slow webMeasurementsMeasuring with javascript
We need to measure real end-user performance
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
The slow webMeasurementsMeasuring with javascript
We need to measure real end-user performance from the realend-user’s box
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
The slow webMeasurementsMeasuring with javascript
Ask the user?
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
The slow webMeasurementsMeasuring with javascript
While this might work, it isn’t necessarily representative
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
The slow webMeasurementsMeasuring with javascript
Why not just use javascript in each page that you want tomeasure?
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
The slow webMeasurementsMeasuring with javascript
But javascript can’t measure everything... we get as close aswe can
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
The slow webMeasurementsMeasuring with javascript
It also adds page weight
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
The slow webMeasurementsMeasuring with javascript
However, it is available everywhere and gives us flexibility
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
What?How does it work?Accuracy
boomerang is...
A piece of javascript that you add to your web page where itmeasures and beacons back to you, the end user’s perceivedperformance of your page
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
What?How does it work?Accuracy
How?
<script src="boomerang.js" type="text/javascript"></script><script type="text/javascript">BOOMR.init({
user_ip: "<user’s ip address>",beacon_url: "http://yoursite.com/beacon.php"
});</script>
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
What?How does it work?Accuracy
What does it do?
About once a week, measures user’s bandwidth andlatency to your serverOn (almost) every request, measures the time it took toload the current pageBeacons these results back to your server
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
What?How does it work?Accuracy
How does it do it?
Let’s take that one at a time
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
What?How does it work?Accuracy
Measuring latency
Download a 32 byte gif 10 times in sequenceMeasure the time to download eachDiscard the first measurement because it’s overpricedCalculate the arithmetic mean, standard deviation andmargin of error of the rest
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
What?How does it work?Accuracy
Wait, did you say overpriced?
The first image might require a DNS lookup and TCPhandshakeSlow start is not an issue since 32 bytes fits in 1 packet
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
What?How does it work?Accuracy
Measuring bandwidth
After the latency test is done, we download progressivelylarger imagesStop at the first image that times outRedownload that image a few more timesCalculate the median, standard deviation and margin oferror of the largest images
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
What?How does it work?Accuracy
Measuring latency before bandwidth helps here
Those 10 latency images do a lot to widen the TCPwindow sizeThe bandwidth images make much better use of bandwidthThe image we end with uses the most bandwidth
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
What?How does it work?Accuracy
How do we measure page load time?
In the onbeforeunload event, measure the time andstore it in a cookieIn the onload event, check the cookie, and measure thedifference with the current timeWe also make sure that the page that set the cookie is thereferrer of the current page
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
What?How does it work?Accuracy
What? Two pages?
Yes, this needs two pages and cookies. If those aren’tsupported, we try to use the WebTiming API.
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
What?How does it work?Accuracy
How accurate is it?
Latency measurements are very accurate (±1%)Bandwidth is to an order of magnitude. For badconnections can be ±30%
Page load time sometimes has outliers, you needpost-filteringThe margin of error tells you how good your data is
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
FilteringGroupingData
What do we do with the data?
Sanity checking to:Remove fake dataRemove abusive dataMaybe just rate limiting
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
FilteringGroupingData
What do we do with the data?
Statistical analysis to:Remove outliersAggregate based on bandwidth blocksMeasure trends over time and correlate them with codechanges
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
FilteringGroupingData
Bandwidth blocks
0-100 kbps100-300 kbps300-2000 kbps2-6 Mbps6+ Mbps
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
FilteringGroupingData
Bandwidth blocks
Group page load times based on bandwidth block
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
FilteringGroupingData
Bandwidth blocks
Ref: Analysing Bandwidth & Latency – YUI Blog
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
FilteringGroupingData
Bandwidth blocks
Data points from some countries may require narrower bands
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
FilteringGroupingData
Geographic data
Looking at latency from different geographic locations can tellyou where to put your next CDN
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
FilteringGroupingData
ISPs
Grouping data by ISP can tell you who’s behaving badly
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
FilteringGroupingData
Storing the data
We log all beacon requests to apache’s log access fileLow traffic sites could write directly to a DBOthers have suggested using CouchDB as the beaconserverDaily summaries can be sent across to ShowSlow
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
FilteringGroupingData
More data
Write plugins to get more performance dataWe already have a DNS pluginI’m thinking of an IPv6 v/s IPv4 pluginWhat about a full WebTiming plugin?Can we measure connection setup time?
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
FilteringGroupingData
You decide
Once you have the data, you can do anything with it
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
FilteringGroupingData
Thank youhttp://github.com/yahoo/boomerang
http://yahoo.github.com/boomerang/doc/
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
FilteringGroupingData
Photo credits
flickr.com/photos/21233184@N02/4389412851
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
FilteringGroupingData
Contact me
Philip Tellisyahoogeek@bluesmoonhttp://bluesmoon.info/slideshare.net/[email protected]
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang
Why measure?boomerang
data data data
FilteringGroupingData
References
github.com/yahoo/boomerangMore bandwidth doesn’t matter (much) – Mike BelsheAnalysing Bandwidth & Latency – YUI BlogIt’s the latency, stupid – Stuart CheshireThe statistics of web performance
Boston Performance Meetup / 2010-09-16 Measuring the web with boomerang