RUM Distillation 101 -- Part I

transcript

• Philip Tellis• @bluesmoon• philip@bluesmoon.info• SOASTA• boomerang

UX Fest 2013 / 2013-10-01 RUM Distillation 101 1

RUM Distillation 101

Philip Tellis / philip@bluesmoon.info

UX Fest 2013 / 2013-10-01

Did you know that most Rum comes from theCaribbean?

But the largest selling Dark Rum in the world isOld Monk

Jean-Antoine Nollet – 1746

Today Measuring the experience of Real Users is notquite as shocking.

Real Users

• Get delighted• Get frustrated• Have credit cards• ... that they are attached to• Form an impression from experience

Real Users

What can we measure?

Anything that the user does!

Anything that the user does!(in the browser)

But we really care about performance here. . .

So let’s talk about How

Modern Browsers – no problem

Real Users don’t always use the latest browsers.

So there’s a lot of magic to support the other31.38%

The other 31.38%...

1 Get a timestamp before they leave a page2 Save it in a cookie3 Compare with timestamp when the page loads

The other 31.38%...

1 Get a timestamp before they leave a page2 Save it in a cookie3 Compare with timestamp when the page loads

Then there’s some magic to handle edge-cases

...magic

• What if the user goes off-site between pages?• What if there’s SSL involved?• What about new tab?• What about those strange cases where they leave a page

after they load a new one?• What about those common cases where they open

multiple pages from a search engine but don’t actually lookat them?

But it doesn’t need to be. . .

We put all this magic into boomeranghttp://lognormal.github.com/boomerang/doc/

boomerang

• BSD licensed JavaScript library• Measure load time, bandwidth, latency• DNS lookup time, NavTiming, Connection Type• Page Parameters• And anything else you can write a plugin for

And it beacons all of this back to a predefined URL

So what then?

Are you Ops, F2E or Product?

Spot the datacenter

It’s actually in Nederland

How does bandwidth change around the world?

Did you know that IE7 seems to be faster than IE8?

But that’s mostly IE9 users browsing in IE7compatibility mode.

What does a website’s performance distribution looklike?

That’s my Blog (please visit)

Log-normal distribution

Bimodal distribution

Distributions

• Sparse distribution suggests that you don’t have enoughdata points

• Log-normal distribution is typical• Bi-modal distribution suggests two (or more) distributions

combined

Distributions are interesting, but how does it affectengagement?

Did you know that user patience wrt performance isa cultural thing?

It’s tied both to where a user is from, and also thetype of service.

different types of sites

We borrowed a biological term called LD50 andapplied it to Load Time v/s Bounce Rate

This helps you fix only the stuff that’s hurting you.

Did you know that users bounce if a page is toofast?

But that’s mostly because error pages are the fastest.

Measure→ Hypothesize→ Test → Fix← Repeat ←↩

What delighted users a few years ago is now anexpected baseline, the absence of which will frustrate

Changing Expectations

Thank You!Questions?

• Philip Tellis• @bluesmoon• philip@bluesmoon.info• www.SOASTA.com• boomerang• LogNormal Blog

Image Credits

• Jean-Antoine Nollethttp://en.wikipedia.org/wiki/Jean-Antoine_Nollet

• New York Times Olympic Medal Winnershttp://www.nytimes.com/interactive/2012/08/05/sports/olympics/the-100-meter-dash-one-race-every-medalist-ever.html

RUM Distillation 101 -- Part I

Technology