28. Januar, Zürich-Oerlikon

Our Journey to Cloud Cadencethoughts to share from Microsoft Developer Division’s Agile Transformation

Jeff BeehlerPrincipal Program Manager, Visual Studio Cloud ServicesMicrosoft Corporation

Agenda

Agenda

Experience of MS Developer Division

How we enable continuous feedback

A 21st century concensus

Global Distributed Software DevelopmentThe Visual Studio Team

Common challengesQuality assuranceGeographic distributionHolistic project viewCollaboration and efficiencyFlow of customer value

Developer Division size (according to TFS)

3,659 Active users last 7 days

681,004 Total checkins

426,443 Work items

16,855,771 Source code files

15

Stats from DevDiv TFS Server…one divisional server instance out of 42 throughout the company

Terabytes of data

DevDiv: Servant of Three MastersCreates conflicting priorities among stakeholders

Should force a discussion of single backlog

At divisional level, reconciled largely through resourcing

VS2005 – Starting Culture

Conway’s LawThe best and brightest

Autonomy, job rotation, promotion

The currency of loveHeadcount

Dysfunctional TribalismDon’t ask, don’t tell

Schedule chicken

Metrics are for others

Our tribe is better

Our customers are different

WasteEasy credit

No interest charge for debt

Debt Crisis VS 2005 Bug Step @Beta 1

Deferral a common technique (see top line)

Teams carry undone work (both functional and ‘ility)

Endgame hard to predict

3-4 Years to Build It Service It (10 years)

Product Cycle Pre 2005

M1…M3: Develop the code

err…M3.1…M3.3 Recode

Beta1: Integrate and pray

Beta2: Test like hell

RC (release candidate) 0..n: Final builds

RTM: Ship it!

QFEs and hot fixes

Service packs

M0: Plan and cost the release

Business Model: One-time License

Three Waves of Learning to VS2012First wave: address technical debt

Second wave: amplify flow of value

Third wave: reduce cycle time

Moving from 2005 to 2008

Get Clean, Stay Clean

SchedulingEngineering Principles

Product Backlog

Measurement & Hardening

Defining Done

Moving from 2005 to 2008

Get Clean, Stay Clean

SchedulingEngineering Principles

Product Backlog

Measurement & Hardening

Defining Done

Focus of MQ (Milestone for Quality)Get CleanUnderstand our Debt

Test AutomationProduct Code BugsTest Case BugsTest Infrastructure Bugs

Put our Debt on the TableCost our DebtEliminate the Debt

Stay CleanMeasure our Quality

Quickly & ConsistentlyAnytime, Anywhere

Quality earlier in the cycleMake life easier

Common practices

Get Clean, Stay Clean

SchedulingEngineering Principles

Product Backlog

Measurement & Hardening

Defining Done

Product Backlog End-to-End Scenarios, Value Props, Experiences, Features

Scenario Scenario

Value Proposition

Value Proposition

Value Proposition

Value Proposition

Experience Experience Experience Experience

Feature

Feature

Feature

Feature

Feature

Feature

Feature

In your business, do you need to…?

Epic: What if you could…? Would you buy or upgrade?

Theme: Let me show you how to…? Would you use it that way?

User story: As a…, I can…

Common practices

Get Clean, Stay Clean

SchedulingEngineering Principles

Product Backlog

Measurement & Hardening

Defining Done

Quality Gates

Applied to Features

Common practices

Get Clean, Stay Clean

SchedulingEngineering Principles

Product Backlog

Measurement & Hardening

Defining Done

Changes to Pre- and Post- Game

Build it (2 years) Service It (10 years)

Planning: Start and groom product backlog

MQ: Improve our practices, get ready for the next version

M0: What are our goals for this release? What customer value do we deliver?

M1…M3 5-week sprints: Develop and test the code

CTP (customer technical preview): Targeted customer release to collect feedback on mainline scenarios

Beta1: First broad customer visibility; collect feedback

Beta2: Validate recent changes with customers; collect feedback

Quarterly: Powertools and Feature Packs for current release

Business Transformation from Packaged Product to Subscription

RC (release candidate) 0..n: Final builds

RTM: Ship it!

Common practices

Get Clean, Stay Clean

SchedulingEngineering Principles

Product Backlog

Measurement & Hardening

Defining Done

Engineering Principles

Drive quality upstreamIncrease efficiency by doing things right the first time

Don’t defer workBug tail not filled with hidden work

Keep main branch customer readyEnables consistent delivery of high quality CTPs

Product not tattooed with incomplete or unstable code

Ready to ship when the time is right

Use Feature Branches for new feature developmentOne branch per feature

Only RI feature branch when quality gates are met

Feature Branch PU Staging branch Main

Automate testingAutomated testing at multiple levels

Tests run anywhere

Common practices

Get Clean, Stay Clean

SchedulingEngineering Principles

Product Backlog

Measurement & Hardening

Defining Done

Interim E2E Perf Scenario Drilldown

1-Apr 16-Apr 5-May 15-May 8-Jun 10-Jun 24-Jun 30-Jun 10-Jul 13-Jul

HighEnd 0.31023278985111

4

-0.10721271721218

2

-0.33104060180189

1

NaN -0.07063981264637

NaN NaN -0.52233208192394

3

-0.08516621208942

38

NaN

Typical -0.39822681161445

8

NaN NaN -0.49038009393799

5

NaN 0.21669451660943

3

0.11830451214511

2

NaN NaN -0.61039542988915

4

-70%

-50%

-30%

-10%

10%

30%

VB WebApp Large – Customer Submitted (weighted)

HighEnd

% f

rom

Orc

as

Trend:

VS 2008 SP1

VS 2010

0 200 400 600 800 1000 1200 1400

Main.20630.VS.PRO.Vista Start VS & Load SolutionOpen webform to Source View activeFirst Switch to Design ViewSecond Switch to Design ViewShow Toolbox

Seconds

Common practices

Get Clean, Stay Clean

SchedulingEngineering Principles

Product Backlog

Measurement & Hardening

Defining Done

Initial planning

spike

Beta 1 plannin

g review

Beta 2 review

Tightened process rules @ Beta 1

reduce WIP

Additional tightening

@ Beta 2

Actual Feature Flow in VS2008

2005 Debt vs. 2008 Debt

15x Debt Reduction!

Minimal Deferral

2x Schedule Improvement

High Predictability

Huge Customer Sat Rise

VS 2005 Beta 1Product Bugs only

VS 2008 Beta 1ALL bug debt

Three Waves of Learning: VS 2012First wave: address technical debt

Second wave: amplify flow of value

Third wave: reduce cycle time

Having removed debt, we could focus on flow of value

Focus on the flow

VS2010 Aspirational Lean ScenariosProduction mismatchesWaiting for build setupUI regressionsPerformance regressionsMissed requirements or changesNo repro

Planning black boxLate surprisesBuild breaksParallel development painBewildering adminButterfly effects or legacy fearCode & fix

What if you could experience No More…

Wave 2 Realizations Actions beget reactions, good and bad

Product ownership needs explicit agreement

Grooming product backlog cannot be skipped

Backlog need qualities of service

Celebrate successes, but don’t declare victory

VS 2010 Recognition

GARTNER: Magic Quadrant for Application Life Cycle Management 5 June 2012

VOKE: Test Market Mover ArrayJuly 2010 Evaluating VS2010

Community votes for best product

IDC: IT PPM Market LandscapeDecember 2010 Evaluating VS2010

Industry analysts recognize Microsoft leadership

The Third Wave of Learning: VS 2012First wave: address technical debt

Second wave: amplify flow of value

Third wave: reduce cycle time

Customers

Importance to your

business

Satisfaction with

Existing

Is this in the right directio

n?For each idea… Most

important Least

important Does the existing functionality meet

your needs?Once I work with something I can’t find it again          

 I can’t tell what changed from build to build, especially over time          

 I don’t know who is the expert of a given area          

 I need another pair of eyes on my code          

 

The three questions they vote on

And the votes are visible and discussed

Real UsersMatter

Improved cycle time

Sep 12P18

Sep 12P18

Jun 12P17

Jan 12P14

Jan 12P13

Apr 12P16

Jan 10RC

Jun 11P6

Mar 10ENU

Dec 11P12

Apr 12P10

May 12RC

Dec 10P1

Aug 12P10

Apr 04M3

May 09Beta 1

Mar 12P15

Oct 09Final Beta

Feb 12Beta

Sep 11P10

Feb 06RTM

May 11P5

Oct 11P11

Jan 09M3

May 12P10

Nov 07ENU

Jul 12P10

Jun 07Final Beta

Apr 12P10

Apr 11P4

Jul 04Beta 1

Jan 08RTM

Mar 12P10

Jan 11P2

Jul 11P8

Dec 11P10

May 10RTM

Jul 12P18

Sep 11P10

Apr 05Final Beta

Jan 07M3

Apr 07Beta1

Oct 07RC

Jul 11P7

Feb 11P3

Jul 05RC

Jun 12P10

Jul 11P9

Jun 12P10

Aug 11M3

Oct 05ENU

Oct 12P18

Nov 12P18

Dec 12P18

Dec 12P18

Jan 13P18

Nov 12VSU1

Aug 12P10Sep 12

P10Oct 12

P10Nov 12

P10Nov 12

P10Jan 13

P10Jan 13

P10Dec 12

P10

Aug 12RTM

Dec 04Beta2

continues

Post-VS2012 Value Delivery Highlights

Kanban phase 1Cross-browser testingSharePoint emulator for unit testingSharePoint load testingCode coverage for manual testingPause and resume long-running manual

testsCode mapIntelliTrace integration with System Center

Visual Studio 2012 Quarterly Update 1

• Kanban support• Drag & drop sprint planning• 400 character paths on server• Build drops• Service released and is no longer

“preview”• Send work items in email • Changeset summary view• Inline viewing and diffing for images• Account rename

Team Foundation Service

Agenda

Agenda

Experience of MS Developer Division

How we enable continuous feedback

A 21st century concensus

Continuous feedback

PRODUCT BACKLOG

Sprint

OPS BACKLOG

Monitor

Priorities

Testing

Code & Test

Ideas

Live site

Ideas

PRODUCT BACKLOG

Sprint

OPS BACKLOG

Monitor

Priorities

Testing

Code & Test

Ideas

Live site

Now everyone can storyboard ideas

Now make it continuous as a request from a PBI

And capture the results in the backlog

Priorities

PRODUCT BACKLOG

Sprint

OPS BACKLOG

Monitor

Priorities

Testing

Code & Test

Ideas

Live site

Work visible

IterationBacklog on a Taskboard

Code & Test

PRODUCT BACKLOG

Sprint

OPS BACKLOG

Monitor

Priorities

Testing

Code & Test

Ideas

Live site

Continuous Feedback while CodingTest a little

Code a little

Coding Feedback

Loop

Testing

PRODUCT BACKLOG

Sprint

OPS BACKLOG

Monitor

Priorities

Testing

Code & Test

Ideas

Live site

Explore and Map

http://procl.files.wordpress.com/2011/06/map-of-north-america-1806.jpg

Explore and Map

http://procl.files.wordpress.com/2011/06/map-of-north-america-1806.jpg

Realistic, Continuous,InstrumentedLab

http://procl.files.wordpress.com/2011/06/map-of-north-america-1806.jpg

Measure As You Go

Live site

PRODUCT BACKLOG

Sprint

OPS BACKLOG

Monitor

Priorities

Testing

Code & Test

Ideas

Live site

Escalatefrom Production

2 Metrics: Continuous Flow Of ValueDevelop

Idea to working softwareOperateIncident to resolution

PRODUCT BACKLOG

Sprint

CYCLE TIME

MEAN TIME TO REPAIR

OPS BACKLOG

Monitor

Our Journey:

The Virtuous Improvement Cycle

Reduction of Waste

Trustworthy Transparency

Increased Flow of Value

Shortened Cycle Time

Continuous

Feedback

Continuous Business

Improvement

There’s No Place Like Production

http://erikasaca.files.wordpress.com/2011/03/ruby-red-slippers2.jpg

Discussion

© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a

commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.