Agile Automotive (Final)

transcript

James JanisseVP Connected Navigation System

Driving Scaled Agile at TomTom

The Good, the Bad, and the Ugly

Agenda

• Context/Background

Agile in Automotive 2

Initial Engineering Context

Navigation

Portal

In-Dash

Companion

Mobile

Initial Engineering Context

• 5 locations

• 300+ engineers

• 40+ scrum teams

• 3 Agile Release Trains

• 6 products

– 3.2 million consumer units

– 2 million app downloads

– 1 billion map tiles served

– 3 million in-dash systems

Navigation

Portal

In-Dash

Companion

Mobile

January 2012 – The Organisation

• Situation

– Functional organizational structure

– Resource Managers “own” the engineers

– Bring developers to the requirement

– Execute via projects

• Complication

– Delays in projects impact staffing

January 2012 – The Organisation

• Situation

– Functional organizational structure

– Resource Managers “own” the engineers

– Bring developers to the requirement

– Execute via projects

• Complication

– Delays in projects impact staffing

• Impact

– Overhead to select, manage, staff teams

– Accountability?

– Every team has a learning curve

– No historical base for estimates

– Developers don’t have to maintain their own code

January 2012 – Performance Management

• Situation

– Stage-gate milestone governance

– Standard “project” oriented KPIs

• on-time

• on-budget

• sufficient quality of “feature complete”

• Complication

– How can you assess if Project X is more efficient than a previous

Project?

– Our business is continuous delivery not time-bounded projects

TT5: Approval to Start Feasibility study

TT4: Approval to Start Design

TT3: Approval to Start Development

TT2: Approval to Start mass production/ship

TT1: Approval to phase out

Agile in Automotive

January 2012 – Performance Management

• Situation– Stage-gate milestone governance

– Standard “project” oriented KPIs• on-time

• on-budget

• sufficient quality of “feature complete”

• Complication– How can you assess if Project X is more efficient than a previous Project?

– Our business is continuous delivery not time-bounded projects

• Impact– Difficult to enable continuous improvement

– Many releases are months-quarters late

– Did not actually manage or mitigate risk as projects can fail a milestone or “pass with concessions” and keep going

TT5: Approval to Start Feasibility study

TT4: Approval to Start Design

TT3: Approval to Start Development

TT2: Approval to Start mass production/ship

TT1: Approval to phase out

Agile in Automotive

January 2012 – The Product

• Situation

– Each customer is a distinct project

– Sometimes projects formed for a major new feature

– Each project is a unique branch of the mainline code, resulting in

multiple branches

• Complication

– Many projects working in all parts of the code with minimal module or

component ownership

9Agile in Automotive

January 2012 – The Product

• Situation

– Each customer is a distinct project

– Sometimes projects formed for a major new feature

– Each project is a unique branch of the mainline code, resulting in

multiple branches

• Complication

– Many projects working in all parts of the code with minimal module or

component ownership

• Impact

– Effort to merge code back together takes +20% effort

– Some defects are fixed multiples times

January 2012 – Integration and Deployment

• Situation

– Negligible automated testing

– No continuous integration

– Typically completed at the end or every few months

• Complication

– Scheduling integrations slots is like Air Traffic Control at Heathrow with

only one runway

– Integration often blocked by other project’s changes

– full regression requires 3-4 calendar weeks and a team of 4-5 people

– While the code of Project X was branched, Project Y checked their

changes back in after 4 months of work, so…

January 2012 – Integration and Deployment

• Situation– Negligible automated testing

– No continuous integration

– Typically completed at the end or every few months

• Complication– Scheduling integrations slots is like Air Traffic Control at Heathrow with only

one runway

– Integration often blocked by other project’s changes

– full regression requires 3-4 calendar weeks and a team of 4-5 people

– While the code of Project X was branched, Project Y checked their changes back in after 4 months of work, so…

• Impact– Effort to merge code back together takes +20% effort

– Some defects are fixed multiples times

January 2012: Customer Impact

• Situation

– 3-5 months acceptance period required for downstream integration/

commercialisation team

• Complication

– What happens if requirements changed?

– What happens if it is “compliant” but not exactly what was expected?

January 2012: Customer Impact

• Situation

– 3-5 months acceptance period required for downstream integration/

commercialisation team

• Complication

– What happens if requirements changed?

– What happens if it is “compliant” but not exactly what was expected?

• Impact

– What value is being added to the product during x months of

acceptance?

Conclusions from 2012

• Branching the code is evil

• User experience cannot be specified on paper, but

has to be felt via running code

• Riskiest stuff is always scheduled for the end

• Many products and releases are months late

• Quality is decreasing

• Costs are increasing

• Time-to-market is increasing

• Project orientation is wrong

• Poor Accountability

• Significant Waste

Agenda

• What we did

5 Key Changes 2012-2015

1. Adopted “Scaled Agile Framework”

2. Re-organised from Scrum Teams up

3. Adopted “High Assurance” S/W Requirements Model

4. Implemented Rally for Planning & Reporting

5. Automated testing with Continuous Integration

Why Switch?

Traditional “Waterfall” Agile

• Not developing new capabilities • Developing something new

• User experience is either not

applicable or completely understood

• User experience is important

• Requirements can be fully

documented, analysed, and are

universally understood

• Requirements are difficult to

articulate and share

• Time to initial value is not applicable • Time to first value is important

• Low technical risk • High risk technical challenges

Agile in Automotive

Why Switch?

Agile in Automotive

Why Switch?

Agile in Automotive

Why Switch?

Agile in Automotive

Why Switch?

Agile in Automotive

Audience Interaction

Agile in Automotive23

Agenda

• What we did

• The results

Summary of “The Good”

Observation Impact

Always release on fixed

schedule

• Reliable and predictable releases of production code

• Establishes fixed rhythm

Release quicker and

more often

• Fail fast (<2 weeks) is better than after 6 months

• Validate and adapt sooner

• Adapt to change/learnings

Run automated tests

suite per submission &

per day

• Detect/prevent issues with each new submission

• Mainline is always able to run

• No bottleneck at the end

• Reduces waste as others stay up to date

Single shared backlog

available to all to view

• Improved transparency and info sharing

• Done means working capability not task complete

Perpetual teams • Teams establish ways of working & esprit du corps

• Improves estimating by allowing historical comparisons

• Enables estimation accuracy analysis

• Team controls their own commitments

• Sustainable development

Agile in Automotive

Observation Impact

schedule

Release quicker and

more often

Run automated tests

per day

Agile in Automotive

Observation Impact

schedule

Release quicker and

more often

Run automated tests

per day

Agile in Automotive

Observation Impact

schedule

Release quicker and

more often

Run automated tests

per day

Agile in Automotive

Observation Impact

schedule

Release quicker and

more often

Run automated tests

per day

Agile in Automotive

Summary of the “Bad”

Observation Impact

Teaches the business that

Agile = 100% predictable

• What happened to iterative development

• What happened to incremental development

Enables the business to

change direction/ strategy/

priorities often

• Let’s face it, too much Agility is just an inability to make

choices and decisions i.e. chaos

Everyone can see everyone’s

backlog, priority, throughput…

• Everyone can second guess your prioritisation

• Everyone can second guess your estimates

Shows week teams • Teams that normally staid below the radar which no one

new what they did are suddenly very exposed to the

daylight

Goes on forever without a

break (HIP downtime)

• Projects used to have a nice slow start up and shut down

phase, so cyclical rhythm

• Now work is harder and does not let up

Observation Impact

priorities often

daylight

Observation Impact

priorities often

daylight

Observation Impact

priorities often

daylight

Observation Impact

priorities often

daylight

Summary of the “Ugly”

Observation Impact

Some teams will resist • Reject the need to be part of an ART

• Reject the need to have common ways of working…

Some people loose

• Project managers loose scope, resourcing, and budget

• Resource managers are no longer needed

Centralised decision

making shifts to

decentralised

• Evolving decisions to the lowest level threatens central

portfolio level experts like architects and makes guiding

independent teams hard

Fully defined

transforms to barely

sufficient

• Architects still love to make future proof architectural

designs and plans

• UX want to make pixel perfect designs for all use cases

Observation Impact

Some people loose

making shifts to

decentralised

Fully defined

sufficient

designs and plans

Observation Impact

Some people loose

making shifts to

decentralised

Fully defined

sufficient

designs and plans

Observation Impact

Some people loose

making shifts to

decentralised

Fully defined

sufficient

designs and plans

Summary

The Good Far out weighs

the Bad and the Ugly

You Decide

Decide to adopt SAFe

Stock = €3/share

Stock is now

€10.21 /share

Additional Info

Information

Observation Impact

Historical repository of

estimates and actuals

• Improves estimating by allowing historical

comparisons

Historical record of

velocity and delivery

reliability

• Visible to the whole business so that everyone

can see the health and output of all teams

Product

Observation Impact

Simpler & consistent

release schedule

• Makes customers lives easier to know that a new

PSI is available on fixed and shared dates

Reduced cycle time • Get innovation out the door faster

Allows testing to go

upstream

• Customer/downs stream acceptance testing can be

embedded with each story or feature to ensure it is

accepted at the end of the sprint vs waiting to accept

it post PSI delivery

People

Observation Impact

Bring requirements to

perpetual teams

• No start-up delay or effort

• Lifecyle management guaranteed

• Track record of working together

• Self improving teams

• Stop timesheet reporting

More sustainable

development

• Teams select how many stories to bring into their

release and sprints

Enables “bigger” picture

• Release planning days

• Release demo days

• Scrum of Scrums

Process

Observation Impact

Bring requirements to

perpetual teams

• Known way of working

• More accurate planning and estimating

• Tailored way of working per team

Eliminate project

milestones and stage gates

• Budgeting is an annual activity to balance

product/team spending

• Given budget, team size, and release dates are

fixed, team just has to perpetually prioritise scope for

maximum business value/PSI

Easy to add a new scrum

• Adopt de facto processes, schedules, tools,

methods…

Technology

Observation Impact

Automated testing • More thorough testing

• More timely test results

• Increases customer confidence

Continuous Integration Quicker feedback

Detect/prevent issues with each new submission

No bottleneck at the end

Reduces cycle time as others stay up to date

Facts based quality

analysis

Statistical analysis of code quality available to compare

across teams, products,

Self-organising quality improvements

Information

Observation Impact

Normalised story

points “suck”

• Seen as an invention by management with

no value for the team itself

Everyone can see

everyone’s backlog,

priority, throughput…

• Everyone can second guess your

prioritisation

• Everyone can second guess your

estimates

Product

Observation Impact

Limited by the critical chain

product/team

Each scrum team can aim for maximum efficiency,

velocity, quality but they are still part of an ART

• Don’t print features on the box at the start of the PSI

priorities often

• Let’s face it, too much Agility is just an inability to

make choices and decisions i.e. chaos

People

Observation Impact

Organisational design • Incompatible with a matrix organisation

Career management? • Do you want a scrum master giving career guidance

to everyone in their team?

Highlights inconsistencies

in roles & levels

• Historical debt of having uncontrolled job grades,

titles, levels, etc

• Issues with Jr. and Sr. Titles and levels

• Issues with relative size of ARTs

People

Observation Impact

Shows week teams • Teams that normally staid below the radar which no

one new what they did are suddenly very exposed to

the daylight

Less role types and levels

= less opportunities

• Less vertical career growth opportunities

• What happens if you are on a second tier ART?

Teams are not

interchangeable

• People are prone to comparing velocities, cycle

times without understanding non standard Story

Point scale, tech debt, technology choices, dev ops

issues…

Process

Observation Impact

Goes on forever

without a break (HIP

downtime)

• Projects used to have a nice slow start up

and shut down phase, so cyclical rhythm

Teams need to align

on cadence/dates

• Teams that never had to coordinate their

schedule suddenly have to adopt a

common schedule for their ART

Teams need to align

on basic Way of

Working

• Teams that has their own home grown way

of working suddenly need to standardise

how they work

Technology

Observation Impact

Not all tooling is fit for

purpose

• Effective portfolio and product

management across multiple teams is not

always supported

• Deep hierarchical backlog is hard to model

• Rolling up reports above scrum teams can

be difficult/impossible with light weight

scrum tools

Increase load on Dev

Ops (testing and

continuous

integration)

• Continuous integration with expanding

automated testing frameworks can get

larger and larger and slower and slowerAgile in Automotive 52

Information

Observation Impact

Shared information

and transparency can

lead to politics

• Teams can start mis-infiormation

campaigns to distract and deflect attention

for their own poor performance

Product

Observation Impact

PSI does not mean

release everything, all

once, every time

Potentially shippable does not mean that the

business is relieved of having to make

business decisions on risk/reward of each new

feature.

People

Observation Impact

• Reject the need to have common ways of

working…

Some people loose

• Project managers loose scope, resourcing,

and budget

Levels and role

consistency

• Can lead to a lot of unhappy employees,

HR issues, administration…

Process

Observation Impact

making shifts to

decentralised

• Evolving decisions to the lowest level

threatens central portfolio level experts like

architects and makes guiding independent

teams hard

Fully defined

sufficient

• Architects still love to make future proof

architectural designs and plans

• UX want to make pixel perfect designs for

all use cases

Sometimes one ART

runs over anothter

ARTAgile in Automotive 56

Technology

Observation Impact

Very large Monolithic

code base cannot be

continuously

integrated

• What happens when the number of

Integration slots available within a sprint

does not allow each scrum team to check

in daily, let alone weekly?

Very large Monolithic

code base cannot

complete daily

regression testing

• What happens when the automated

regression cycle takes more than 16 hours

(i.e. cannot be completed after hours)

Agile Automotive (Final)

Documents