Martin Geddes - Lean Networking

Martin Geddes www.martingeddes.com [email protected]

This presentation was given as a keynote at the Illinois Institute of Technology Real Time Communications

conference in Chicago on 11th September 2012

© 2012 Martin Geddes Consulting Ltd

Cosmologies

This presentation is about belief systems – ‘cosmologies’ – and our most basic assumptions about the

nature of the (networking) universe.

Do our models explain observed reality? If not, the model has scope to be improved.

This is the motion of Mars across the sky, as seen on

successive nights.

Reality vs Model

The ancient Greeks struggled to explain this retrograde motion.

Indeed, the word ‘planet’ means ‘wanderer’.

Geocentric

Their basic assumption was that the Earth is the centre of the universe.

This is an anthropocentric viewpoint.

Everything revolved around the Earth, because that’s where we

are, and what could be more important than us?

Gods!

Geocentric

So the explanation for the motion of planets was to appeal to

unknown and unknowable forces.

Ptolemy

Then this guy comes along with a better idea in the 2nd century AD.

Epicycles!

Geocentric

The planets go in little loops as they go round the Earth.

All is explained!

Copernicus

Except that didn’t really account for what we observed. There

was a residual motion that wasn’t explained by epicycles. So the basic assumptions had to be

re-visited.

A mere one and a bit millennia later, Copernicus suggests that

maybe we (and the Earth) aren’t the centre of the universe after

all.

#fail

Heliocentric Geocentric

His heresy was to say that the Sun was the centre of the solar

system, and everything went around that.

Which is a shift in the origin of how we reasoned about

planetary motion, and required us to let go of the

assumption the whole universe revolves around

us.

Isaac

Newton

This process of refinement continued.

#fail

Heliocentric Geocentric

Gravity

Gravity explained the elliptical motion.

The Sun turned out not to be the centre either – as we understood galactic

structures.

You Know Who

And there were still unexplained residual differences between

observed reality and our models. So relativity fills

that gap.

Phantom Dark Energy Model

(2007)

This refinement of cosmological models continues to this day.

“Obese” “Lean”

Networking Cosmologies

I am proposing that we need to shift our ‘origin’ and adopt a better cosmology in order to make progress in networking.

The existing dominant

cosmology…

A suggested improved

cosmology…

Agenda

Lean

Manufacturing

Lean Conversations

“Lean Networking” Implications

WHAT &

HOW

WHY (“Obese Pipes”)

Internet, WebRTC,

policy

Future of Distributed

Network Computing

The rest of this presentation

involves comparing a lot of ‘obese’ and

‘lean’ cosmologies.

LEAN MANUFACTURING GETTING THE BASIC IDEA

Theory of

Constraints

This book introduced the basic principles that we are

interested in, and was a ‘cosmological shift’ in

process manufacturing.

Supply of raw

materials

Demand for finished

goods

Bottleneck

Every production system has stages of processing raw inputs into valued

outputs.

There is always a bottleneck somewhere

in the system.

Different Worldviews

Cost Accounting

Utilisation (Cost)

Batch

Bad

Cost per Unit

Throughput Accounting

Flow (Throughput)

Single-piece

Good

Lead Time

Focus on

Work unit

Variation

Key metric

Batches make sure there’s no idle time. Variation just drives

up unit cost.

Variation in quantity and quality of

demand is ordinary.

A resource standing idle is a major waste.

Aha! Fallacy

WRONG!

Goldratt’s great insight was to

identify this fallacy.

Journey to ‘Lean Anything’

Flow efficiency

Re

sou

rce

eff

icie

ncy

LOW HIGH

LOW

HIGH

Are the machines

busy?

Is stuff produced fast

enough?

The natural tendency of managers seeing idle men and machinery is to make

it get busy.

But you can’t get flow efficiency this way. Coping

with variation requires slack, and you don’t have

any.

“A plant in which everyone is working all the time

is very inefficient.”

Eliyahu M. Goldratt

Busy machinery isn’t the same as creating value, which is from products delivered to customers

on time.

Inventory

(WIP)

Keeping machinery busy upstream of a bottleneck just

creates work-in-progress (WIP), which ties up capital,

requires storage, and inhibits flexibility.

Journey to ‘Lean Anything’

Flow efficiency

Re

sou

rce

eff

icie

ncy

LOW HIGH

LOW

HIGH So to get to production nirvana, you need to master flow efficiency (i.e. low lead time) first,

then figure out resource efficiency.

1. Visualise your workflow

2. Limit work-in-progress (WIP)

3. ERR… THAT’S IT!

Key features of methodology

Goldratt gave us a methodology, which I

have grossly oversimplified to these

two steps.





Source: agilemanagement.net

Cumulative Flow Diagram

There are flow visualisation techniques

which emphasise the metrics we care about:

lead time and WIP.





Regulate

Inflow Limit WIP by preventing more inflow into the

system than the bottleneck can process.

LEAN CONVERSATIONS WARM-UP EXAMPLE

Before we start on networking, let’s

practise these techniques. How could we use

lean principles to escape from email

hell?

“A lesson

from lean

manufacturing”

We’re not alone in thinking about this

problem in this way.

A human standing idle is a major waste.

Aha! Fallacy

WRONG!

Bottleneck

24

You really don’t want to spend your life doing

email. And even if you do, there’s a finite amount of time resource to dedicate

to the cause.

Journey to ‘Lean Conversations’

Flow efficiency

Re

sou

rce

eff

icie

ncy

LOW HIGH

LOW

HIGH

So how can we lower lead time and WIP to

create flow efficiency in our email handling?





My inbox

Empty… (except new messages since I got off the plane)

I don’t keep my emails staring at me from my

inbox, as if it was a to-do list anyone can scribble

on.

Kanban visualisation system

I use a product to visualise and prioritise

my work – and at a glace see what needs to be

done next.

An example ticket, linked back to the original email that made the request.





Limited WIP

I also force myself to focus on a

maximum of two activities at once.

My attention is the bottleneck.

My personal workflow

I limit WIP to force me to recognise my finite work capacity.

Regulate

Inflow

24

How can we do this for email?

“Jane’s inbox has reached its limit of 30 unread items. To resubmit your message at a later time, click here.”

“You have sent 3 messages to John. He has a limit of 2 messages outstanding per person.

Which message do you want to delete?”

Holiday hell: “You have 319 new messages.” – Just send them all back and ask people to re-submit!

End ‘inboxbloat’ by limiting WIP

LEAN NETWORKING INTRODUCING

That was the salad. Now for the burger.

Three kinds of network

Network of Promises

Network of Possibilities

Network of

PROBABILITIES

Manage priority, control loops

Manage flow, contention

and trade loss/delay

Manage sessions, admission

and capacity

NON-DETERMINISTIC STOCHASTIC DETERMINISTIC

Internet world

TDM circuit world, IMS world

Bounded flow isolation

Assured loss and delay

Unconditionally stable

Scavenger traffic

UK demo ISP running since 2007

The benefits of connections and connectionless… without needing circuits to reserve capacity or congestion control loops!

My colleagues have built something that appears

impossible in the existing cosmology.

A network element standing idle is a major waste.

Aha! Fallacy

WRONG!

How? Same as Goldratt – they saw the fallacy

common to all networking.

Journey to ‘Lean Networking’

Flow efficiency

Re

sou

rce

eff

icie

ncy

LOW HIGH

LOW

HIGH

Flow efficiency in networks means

containing loss and delay to meet the

bounds required by the application.

Stop making networks

mindlessly busy creating WIP





Improper Probability

Cumulative Distribution Function

See how loss and delay accrue along the path,

using a probability mass.





Queues are

Bottlenecks

Delay = WIP

Waiting packets are the networking equivalent of

WIP.

WIP, WIP, WIP

Today’s networks maximise WIP, with

TCP/IP being the worst offender.

We use the word ‘idle’ to mean ‘lazy’ as well as to describe a network link that is not transmitting.

This gives us a clue we have made an

inappropriate moral value judgement

“Immense harm is caused by the belief

that work is virtuous.”

Bertrand Russell

What underpins this is a belief that it is morally

wrong for links to be idle.

Limit WIP!

Instead we should build networks so that they

limit packet ingress to the known downstream

bottleneck, and minimise WIP.

Work-

Conserving

Queues

That means the most basic assumption built into

every network router is wrong – or at least

unhelpful and costly.

Work-conserving queues are ones which always transmit if the link is idle and a

packet is waiting.

Networks don’t do work!

They do translocation.

Category Error

The clue seemed to be in the name –

‘net-work’ – but they aren’t machines that

do work.

This is a big deal.

Putting stuff in the wrong place stops you doing something useful, i.e.

moving packets can create negative value

Work metaphor ! !

A lot of the ‘work’ routers do in fact destroys value.

QUALITY ATTENUATION A BETTER COSMOLOGY

Bandwidth Quality

Attenuation

Let’s give ours a name too.

Let’s give the old cosmology a name – the ‘bandwidth’ model.

Bandwidth ΔQ

And abbreviate it.

An idea like entropy or noise…

…with many mathematical representations

What is ΔQ?

Shannon solved this ‘noise’ stuff for single links; we’re doing it

for complete multiplexed systems.

An ideal network copies data instantly and perfectly

Real networks instead create loss and delay

Zen of Netwrking Alan Turing

assumed this.

And that’s it. All they do is

degrade.

Networks are trading spaces

Zen of Netwrking

This is important.

Pay attention.

They aren’t pipes. They just allocate the

degradation compared to ‘instant and perfect’.

Networks are trading spaces that allocate

{loss-and-delay}

Zen of Netwrking

Which can be traded (e.g. drop

more packets means less delay

for the rest)


disappointment

Zen of Netwrking

They allocate this

disappointment (degradation) to different flows.


quality attenuation

Zen of Netwrking

Which we give a fancy name


ΔQ

Zen of Netwrking

And make it harder to type.

1. ΔQ exists

2. ΔQ is conserved

3. ΔQ has two degrees of freedom (for mutable contention)

The Three Fundamental

Laws of Networking

If you offer a load to a network, loss and delay will happen.

You can’t un-lose or un-delay stuff.

Rather like pressure-volume-temperature you learnt in school for a piston, networks have load-

loss-delay.

These are the ‘thermodynamic’ laws we have to work with and constrain the best

outcome we can deliver.

Bandwidth

Inter-process comms

“Beads on a string”

ΔQ

So we are working with a different mental model that shifts our origin.

Networks aren’t about pushing packets along one behind another. That’s an anthropomorphic model

that wrongly treats packets like physical packages.

Instead, networks just do inter-process communications with

added loss and delay.

Bandwidth

0 ∞ ΔQ

Our origin is ‘zero loss and delay’. That makes the mathematics tractable –

rather like putting the sun at the centre of elliptical orbits and forgetting

epicycles.

In the bandwidth model, a packet is assumed to stay at rest until we do

‘work’ to push it along. The default is it takes an infinite time to get there.

Bandwidth ΔQ

Quality is the presence of something

positive

Quality is the absence of something negative

People talk about ‘quality of service’, but what is this

‘quality’ they talk of?

Our quality is “the absence of excessive disappointment”.

Their quality is “priority over other packets to get work

done”.

Bandwidth

Instants Averages

ΔQ

“Megabytes per second” Loss/delay probability distributions.

Bandwidth

Non-work conserving queues

Work-conserving queues

ΔQ

Doing work that creates negative value is dumb.

Stop doing it!

OBESE PIPES THE PROBLEMS WITH

Two kinds of obesity

Network of Promises

Overprovision! NON-DETERMINISTIC

Overbuild! DETERMINISTIC


The Internet folk want you to wildly overprovision networks

to make quality-demanding applications work. This is

expensive.

The traditional telco folk want to

build lots and lots of parallel

networks to separate out

different quality requirements.

This is expensive. (This doesn’t actually work – but they haven’t

figured it out yet, so let’s keep it a secret between you and me.)



Overprovision! NON-DETERMINISTIC

Flow efficiency

Re

sou

rce

eff

icie

ncy

LOW HIGH

LOW

HIGH YOU

ARE

HERE

The Internet is stuck here. TCP is designed to saturate networks – i.e. create

resource efficiency. It can’t create flow efficiency.

Offered load

Tran

spo

rte

d lo

ad a

t w

ith

in-b

ou

nd

s q

ual

ity

LOW HIGH

LOW

HIGH

PREDICTABLE CHAOTIC Network

components often behave poorly under

high load

Necessary to manage the load on the network to

remain within the predictable

region of operation

We have lost control over the

customer experience

The Internet is also unstable at

all loads and suffers

congestion collapse effects.


Network of Promises

Overbuild! DETERMINISTIC

Flow efficiency

Re

sou

rce

eff

icie

ncy

LOW HIGH

LOW

HIGH

YOU

ARE

HERE

Traditional telco folk were

heading the right way with technologies

like ATM.

But they’ve gone totally the wrong way since then with IMS which

creates a horribly bloated and inflexible

network.

Network obesity means

LIFESTYLE CHANGES!

Aspiration Quantify desired outcomes

Expectation Define quality need

Execution Match delivery to requirement

Assurance Closing the loop

Nobody wants to hear the lean

networking doctor tell them that their

morbid obesity demands lifestyle

changes.

e.g. how often is it OK for YouTube to show the “circle of death”? Average time to first

frame? MOS score for VoIP apps?

How do aspirations map to statistical bounds on

loss and delay?

How to make applications fail in the

right order when instantaneous demand

exceeds supply? Did it do what you expected? How do you

know?

INTERNET, WEBRTC & POLICY

SO WHAT?

This is a new technology to put real-time two-way

voice and video into web browsers.

Bufferbloat

The phenomenon that Internet

engineers struggle to explain and resolve

in their current cosmology is called

‘bufferbloat’.

This problem won’t find a resolution

without a change in cosmology; attempts

to fix it will spawn new problems.

The network

Hierarchy of Need

3. Uncontended: Low loss and delay

2. Stable: High stationarity

1. Feasible: High capacity

Note: exact requirements are application-dependent

!

!

The bandwidth school of thought

focuses entirely on capacity. This is not

enough.

Applications need finite bounds on loss and delay to work.

Rapid variation in loss and delay cause

application failure.

We’re not measuring the things that actually matter,

Dropping flow isolation

+ Rising peaks of failure

+ Loss of stationarity

= More ‘extreme weather events’ (and poor for real-time comms)

The Internet’s

‘Global Warming’ problem

The Internet is intrinsically unsuitable for carrying all of society’s real-time

communications needs.

The Internet has an inbuilt ‘statistical noise’ problem…

We’re multiplexing more apps, devices and users

together; and doing more real-time applications as

well as bulk data.

So bufferbloat-like effects cause more problems.

…because TCP breaks basic control theory principles.

Not good.

End-to-end

principle

The guiding ‘end-to-end principle’ on which Internet architecture is based is

a post-rationalisation of bad design decisions from the 1970s.

This has a cost our kids will carry.

Has broken philosophical foundations: Value comes from flows with bounded ΔQ

– Not individual packets

Loss is OK and delay can be good! – It’s not a fault or moral failure to avoid work

System of two degrees of freedom (loss/delay) – Not just one (delay)

Network Neutrality is

‘beads on a string’ model

Network neutrality is intellectual nonsense. But

once most people thought it was obvious the Sun went

round the Earth too.

In man vs mathematics, the mathematics takes a

really hard negotiating stance

Network Neutrality ! !

Network neutrality as currently conceived puts regulators in conflict with the mathematics of statistical multiplexing. The maths will win.

There are rigorous and meaningful ways of defining

non-discrimination.

Just not this way.

CLOSING THOUGHTS WE’RE NEARLY DONE

Technological Revolutions &

Financial Capital

Carlota Perez

Steam, Coal, Iron, Railways

Electricity, Steel & Heavy

Engineering

Oil, Petrochemicals & Automobiles

IT & Telecoms ? Biotech, Nanotech

1770 2012

Techno-economic revolutions go through this predictable cycle.

The Turning Point

Amazing! It works!

Fit-for-purpose

You are here! c.70 years after transistor

invented.

Networking becomes embedded into

everyday life, invisible in the way electricity is.

We’re done inventing the basic component

technologies – datagrams,

smartphones, etc.

The Turning Point

Networking as alchemy

Networking as chemistry

Networking has been done on a “try it and see” basis, without

rigorous mathematical foundations.

ΔQ model fills the intellectual hole Turing

left behind when he assumed translocation

was instant and perfect.

“Translocatability” now has intellectual

foundations as solid as “computability”.

Obese monoservice networks

Which means we can begin to treat network obesity caused by

excessive consumption of monoservice (single class)

networks.

Build network

Tweak network to get apps working

Get complaints

Build it bigger

Tweak it again

Select new technology

End the Cycle of Failure

Because that really isn’t working any more.

What we need is…

Ubiquitous data transport for a digital society

And will be even less suitable for what we need

in future.

Lean polyservice networks

We need to go from ‘black and white’ to ‘colour’ networks. Each

flow has a different quality (‘colour’) need.

ΔQ RINA

Beyond cloud & internet:

Distributed Computing

ΔQ is only half of the puzzle. The Internet is mis-architected in

other ways. Check out Recursive Internet Architecture for how it

should be done.

New book by Martin Geddes:

“The Internet is Just a Prototype”

Coming in 2013

www.martingeddes.com

Free newsletter

Congratulations, you have survived this intellectual assault course. For further intellectual assaults, read my newsletter.

http://www.futureofcomms.com/

http://www.martingeddes.com/

www.futureofcomms.com

Future of Voice

& Telco-OTT workshops

London, England October 23rd & 24th

For a more gentle re-shaping of your views on communications,

come to a public workshop.



Twitter: @martingeddes Email: [email protected]

Martin Geddes

Founder, Martin Geddes Consulting Ltd

Get in touch

Additional credits and thanks to: Neil Davies, Peter Thompson, John Day, Fred Goldstein

Pleased to hear from you.

https://twitter.com/

https://twitter.com/

mailto:[email protected]

Thank You

Date post:	20-Aug-2015
Category:	Technology
Upload:	martin-geddes
View:	7,600 times
Download:	1 times