Suite 202855 Oak Grove AvenueMenlo Park, CA 94025Phone (650) 470-0188www.smartorg.com

SMARTORG SM Value-Based Management Systems

Decision AnalysisThe Master Discipline

INFORMS Conference on O.R. PracticeMiami, May 1, 2006

Jim Mathesonjmatheson@smartorg.com

INFORMS 05-01-06 2 © 2000-2006

Decision Analysis is the Discipline concerned with decision-making — both solving problems and maintaining decision health.

• Philosophy– What are decisions and why should we make them?– How do we deal with an uncertain world?– What is a good decision?– What organizational culture promotes good decisions?

• Processes & Practices– What are the steps to reach a good decision?– What are the steps to capturing uncertainty well?– What are the design principles for organizations and their

processes?

• Tools– Decision Theory– Influence Diagrams– Computer Applications

INFORMS 05-01-06 3 © 2000-2006

Decision Analysis = Decision Engineering

• Would you design a bridge by guessing the size of various structural members?

• Would you build a chemical plant by asking an “expert” to guess the size of various process elements?

• Would you like the pilot of your aircraft to fly through fog without instruments?

• They why do people expect executives to use “gut feel” to make decisions?

• Decisions need to be engineered just like bridges and chemical plants

– There are sound engineering procedures for arriving at good decisions

– They need to involve the right people and to guide and motivate excellent implementation

– This discipline is called Decision Analysis

INFORMS 05-01-06 4 © 2000-2006

Decision Analysis is for developing clarity of actionwhen you don’t “know” what to do.

• If you have long experience that you are sure is applicable, make the decision and get on with it! – But remember people like Gladwell (2005) in “Blink” have

demonstrated many cases where intuition goes wrong.– Beware – Often people apply inappropriate operational habits to

strategic situations.

• If you have “inner knowing” of what to do – then do it!

• But when you don’t “know”, or when you want to reassure yourself that your experience is correct, or if you want to delegate to others or to explain to your superiors (who may not “know” or accept your “knowing), then decision analysis is appropriate.– I once had a very interesting lunch with a minister who was frustrated

that people wanted god to balance their checkbooks!

INFORMS 05-01-06 5 © 2000-2006

Many organizations and academics confuse operational strength with strategic intelligence.

In 1982 Peters and Waterman, writing In Search of Excellence, missed the mark on strategy by confusing operational and strategic excellence.

INFORMS 05-01-06 6 © 2000-2006

The skills required for effective operational management are counterproductive in strategic management.

• Focuses on the important issues

• Considers long time horizons

• Treats uncertainty strategically

• Chooses among significantly different alternatives

• Attends to detail and follow-through

• Concerns itself withnear-term performance

• Treats uncertainty statistically

• Avoids new alternatives— “Just do it!”

Strategic Management Operational Management

INFORMS 05-01-06 7 © 2000-2006

We boiled down key areas of the decision analysis into nine principles for designing a Smart Organization.

SmartOrganiza-

tion

Achieve Purpose

Mobili

ze R

esou

rces

Understand Environment

Continual Learning

ValueCreationCulture

CreatingAlternatives

SystemsThinking

Alignment &Empowerment

EmbracingUncertainty

DisciplinedDecision Making

Outside-InStrategic

Perspective

OpenInformation

Flow

David and Jim Matheson, The Smart Organization: Creating Value Through Strategic R&D, Harvard Business School Press, 1998.

INFORMS 05-01-06 8 © 2000-2006

The keystone is a Value Creation Culture – it is the driver of Value-Based Management.

INFORMS 05-01-06 9 © 2000-2006

We can test to see if the principles are operational.

It’s Smart. . . It’s Not Smart. . .

to aim always at value creation

to divert attention to other goals

to know what you do not know and to know what you cannot influence

to pretend you know or try to control outcomes you cannot influence

to take charge of your own destiny

to allow your competitors or fate to determine your future

to coordinate everyone effectively

to micromanage, or not manage at all

to know where you stand to believe provincial illusions

to aggressively inform and be informed

to build power by withholding information

to know the full implications of actions and events

to be myopic

to create valuable options to do the first thing you think of

Principle

1. Value Creation Culture

4. Embracing Uncertainty

7. Disciplined Decision-Making

9. Alignment and Empowerment

5. Outside-In Perspective

8. Open Information Flow

6. Systems Thinking

2. Creating Alternatives

3. Continual Learning

to anticipate and benefit from change

to get stuck where you are

Operational Testfor Principle

Value creation is a compelling argument for change

Uncertainty is understood, communicated, and managed

People have rapid, unrestrict-ed access to information

Systematic decision processes are used routinely

Meaningful information is available from the outside

A common understanding of strategies for value creation coordinates the organization

People understand complex cause-and-effect relationships

Multiple alternatives are created and evaluated

Improvements are continually identified and acted upon

INFORMS 05-01-06 10 © 2000-2006

Analysis of over 500 IQ tests revealed that smarter companies perform better – the discipline is worth it!

0%5%

10%15%20%25%30%35%40%45%50%

<90 90 - 110 >110

Freq

uenc

y of

the

top

quar

tile

perf

orm

ance

Company IQLow Medium High

INFORMS 05-01-06 11 © 2000-2006

Strategic Decisions need to be declared and bounded

• People and organizations can go through life on their momentum– Often their decisions are just “reactions” to the impacts of others– They often blame others for their misfortunes

• Today’s American auto companies are a great example– They blame high oil prices for lack of sales of profitable SUVs.– They blame overcapacity for many high costs, pensions, etc. – They blame Toyota for tying up suppliers for hybrid components– They should blame themselves for acting so dumb!

• We need to put a boundary around a piece of reality that we are focusing our decision attention upon.– Assessments of alternatives, uncertainties, and values penetrate this

boundary– The boundary needs to be tested and revised (reframed)– If a problem is technically difficult, modifying the boundary often

simplifies it.

INFORMS 05-01-06 12 © 2000-2006

A poster for

Teens

from the

Decision Education Foundation

INFORMS 05-01-06 13 © 2000-2006

A high-quality decision produces personal and organizational commitment to the best prospects for creating value.

AppropriateFrame

Creative,Doable

Alternatives

Meaningful,Reliable

InformationClear Values and Trade- offs

Logically CorrectReasoning

Commitmentto

Action

Elements ofDecisionQuality

These links also specify good design principles for each decision.

INFORMS 05-01-06 14 © 2000-2006

Measurement of decision quality can be used to guide a decision process.

DecisionQuality

1Appropriate

Frame

2Creative,Doable

Alternatives

3Meaningful, Reliable

Information

4Clear Values

andTradeoffs

5LogicallyCorrect

Reasoning

6Commitment

to Action

0% 100%

One hundred percent is the point at which additional improvement efforts would not be worth their cost.

Initial and Final Steering

CommitteeAssessments

INFORMS 05-01-06 15 © 2000-2006

The decision process needs to be designed to the needs of the organization and the decision being make.

INFORMS 05-01-06 16 © 2000-2006

Process design must address both organizational and analytical complexities. Narrow analysts often overlook organizational side.

High

Low

Low HighAnalytical Complexity

Organizational Complexity

• Uncertainty• Dynamics• Many interrelated factors• Many alternatives• Multiple interrelated decision criteria

DecideNow

TechnicalDecisionAnalysis

• Many parties in conflict

• Individual and organizational differences:

- Values, desires, and motivation

- Initial convictions- Frames of reference- Personalities and

competencies- Power and resources

• Group dynamics

FacilitativeLeadership

DialogDecisionProcess

INFORMS 05-01-06 17 © 2000-2006

For large organizational decision problems the Dialogue Decision Process (DDP) has been effective. It is carried out in weeks or months.

Dec

isio

n Te

amSt

rate

gy T

eam

Wor

king

Gro

up

• Alternatives• Improved

Information• Values

PlanEvaluatedAlternatives

• Frame• Challenges• Understanding

1. Assess Business Situation

2. DevelopAlternatives, Information, and Values

3. Evaluate Risk and Return of Alternatives

5. Plan for Action

6. ImplementDecisionand Manage Transition

0. DesignProcess

4. DecideAmong Alternatives

In other environments, simplified and software enabled processes provide better solutions.A good process is engineered to achieve decision quality in its organizational setting

INFORMS 05-01-06 18 © 2000-2006

For repetitive decision classes, such as product development, value-based management systems provide process guidance through web-based delivery. It is carried out in hours or days.

The definition part of the process sets up the project and its evaluation.

Next, the team needs to provide the information on the project.

This information drives a cash flow and uncertainty analysis of the opportunity.

The results are sent to the portfolio and incorporated into a PowerPoint deck for presentation.

Project Guidelines—a message from the portfolio team to all the projects—orients team members to your objectives and supports consistency across projects.

Web-based productivity tools: coach on call is an email help function, files keeps a library of documents for the project.

INFORMS 05-01-06 19 © 2000-2006

Strategy Table

The analytical side of decision analysis furnishes a powerful common set of tools.

DecisionStructure Deterministic

AnalysisProbabilistic

Analysis AppraisalInitialSituation

Iteration

InfluenceDiagram

12345

DeterministicModelA B C

DeterministicSensitivity

DecisionTree

ProbabilityDistributions

Value ofInformation

Decision Spiders

INFORMS 05-01-06 20 © 2000-2006

A Decision Hierarchy clearly shows what is being decided now.

Example: Manufacturing Plant Modernization

• Continue manufacturing

Policy Decisions

Take asgiven

• Plant configuration and location

• Technological stretch• Product range• Quality and cost position• Marketing strategy

Strategic Decisions Focus on in

this analysis

• Product design• Manufacturing operations• Marketing plans

Tactical Decisions

Decide Later

Example: Manufacturing Plant Modernization

• Continue manufacturing

Policy Decisions

Take asgiven• Continue

manufacturing

Policy Decisions

Take asgiven

• Plant configuration and location

• Technological stretch• Product range• Quality and cost position• Marketing strategy

Strategic Decisions Focus on in

this analysis

• Product design• Manufacturing operations• Marketing plans

Tactical Decisions

Decide Later

• Product design• Manufacturing operations• Marketing plans

Tactical Decisions

Decide Later

INFORMS 05-01-06 21 © 2000-2006

The Decision Hierarchy Determines Decision Columns in a Strategy Table

Decision Hierarchy

Tactics

Policies

Strategic Decisions

• Plant• Technology• Products• Quality• Marketing

Strategic Decisions (one column for each)

Strategy Table

Plant

Option 1

Option 2

Option 3

• • •

Technology

Option 1

Option 2

Option 3

• • •

Products

• • •

Quality

• • •

Marketing

• • •

Listing the options helps illustrate the scope chosen for decision-making; options will be combined later into strategic alternatives.

Decision Hierarchy

Tactics

Policies

Strategic Decisions

• Plant• Technology• Products• Quality• Marketing

Strategic Decisions (one column for each)

Strategy Table

Plant

Option 1

Option 2

Option 3

• • •

Technology

Option 1

Option 2

Option 3

• • •

Products

• • •

Quality

• • •

Marketing

• • •

Listing the options helps illustrate the scope chosen for decision-making; options will be combined later into strategic alternatives.

INFORMS 05-01-06 22 © 2000-2006

A Strategy Table reduces and many strategy possibilities into a few for evaluation.

Current

Close #1;build domesticgreenfield

Close #1;build foreigngreenfield

State of art

Proven

Current

Full line

One basicline andspecialties

Value-addedspecialtiesonly

Quality and cost leadership

Improved quality;deferredcostreduction

Minimalqualityimprovements

Sellquality and influence market growth

Sellquality

Current

PlantConfigurationand Location

TechnologicalStretch

ProductRange

Quality and CostPosition

MarketingStrategy

Aggressive Modernization

ModerateModernization

Consolidation

Run Out

StrategyAlternatives

Close #1

Current

Close #1;build domesticgreenfield

Close #1;build foreigngreenfield

State of art

Proven

Current

Full line

One basicline andspecialties

Value-addedspecialtiesonly

Quality and cost leadership

Improved quality;deferredcostreduction

Minimalqualityimprovements

Sellquality and influence market growth

Sellquality

Current

PlantConfigurationand Location

TechnologicalStretch

ProductRange

Quality and CostPosition

MarketingStrategy

Aggressive Modernization

ModerateModernization

Consolidation

Run Out

StrategyAlternatives

Close #1

INFORMS 05-01-06 23 © 2000-2006

Influence diagrams identify important variables and relationships in a decision problem.

VariableCosts

FixedCosts

Profit

Revenue Costs

MarketShare

Market Size

MarketPrice

ResultDecisionUncertaintyRelationship

Key

Product Features

INFORMS 05-01-06 24 © 2000-2006

Influence diagrams are useful at several points in the decision process.

Market Share

Market Size

Sales

• Structuring how information will be assessed

A B C12345

• Specifying relationships for the spreadsheet evaluation model

Project Team

– Uncertainties

– Information needs

• Identifying key factors that need to be considered

– Sources of value

– Decisions

An effective influence diagram saves time by ensuring that the evaluation “begins with the end in mind.”

INFORMS 05-01-06 25 © 2000-2006

Influence diagrams also identify information sources and show how this knowledge will be integrated.

Profits

Revenue Costs

Competitors

Price Sales Volume

CapitalInvestment

Labor Cost

DistributionCost

ProductionCost

Raw Materials

Costs

ManufacturingAutomation

ProcurementHuman Resources

SalesManufacturingMarketing

Project PlanningFinance

INFORMS 05-01-06 26 © 2000-2006

Case Study — HDTV

Determining the Best Strategy in an Uncertain World

INFORMS 05-01-06 27 © 2000-2006

TV product strategy was complicated by many uncertainties and involved internal and outside actors.

• A high-definition picture needs a big screen—but large CRTs are inherently too expensive for most consumers.– What display technologies should we pursue to enable mass

penetration?

• We are promising a revolutionary jump in picture quality—but we don’t know what consumers perceive as picture quality, or what they are willing to pay for it.

• What will happen to U.S. standards, and how will that affect Europe?

• The Japanese are already announcing HDTV broadcasts. How long will the EC support us in the face of strong pressure from Japan Inc.

To make any HDTV initiative work, broadcasters, equipment makers, and software creators must all make enormous coordinated investments—and the consumers must buy. Will they?

INFORMS 05-01-06 28 © 2000-2006

An influence diagram depicted a complex situation in a manageable from.

INFORMS 05-01-06 29 © 2000-2006

A Strategy Table assembled the major decisions components into a set of internally consistent strategies.

StrategyTheme Platform

IntroductionsFeature

Introductions

Feature Development

Capability

Display Development

Alliances and Cooperation

Key Decisions

Aggressive Analog

Evolutionary Improvement

Info Age

VCR—The Shortest Road

Conventional Plus

•••

Improved conventional (1995); HD

analog (2001)

HD analog (1995)

Improved conventional

(1993); HD digital (2003)

HD digital (1998)

Reserve all for HDTV introduction

Sequential annual introduction in conventional standard 16:9

aspect ratio (1992); CD audio (1993);

Comb filter (1994); 100Hz progressive

scan (1995)

Sequential annual introduction

conventional TV; CD audio (1992);

Comb filter (1993); 100Hz progressive scan (1994); 16:9

aspect ratio (1995)

Lead (develop IC

design capability)

Follow (purchase ICs)

Negotiate for exclusive IC rights with supplier

Co-develop key ICs with

competitor

Focus on large CRT

Emphasize projection LCD

Switch resources to digital mirror

device projection

Concentrate on large flat panel

advances

Take leadership of EC/industry HD

project

Strong role in conventional

standard council

Join USHDTV consortium

Passive participation

Manufacturing

Enhance large CRT capability

and quality control

Source initially from alliance

partner

Build pilot LCD projection plant

•••

These are not just platitudes in boxes. Each executive should know what it takes to execute it.

INFORMS 05-01-06 30 © 2000-2006

A Strategic Business Model showed the value results of scenarios depicted by the Influence Diagram.

INFORMS 05-01-06 31 © 2000-2006

Careful debiasing procedures were used to assess ranges of uncertainties and full probability distributions.

Value for Sensitivity Analysis Definition

Low

.1 .2 .3 .4 .6 .7 .8 .9

.8 Probability of Being “In the Range”

Low Base High

0 .5 1.0

• There is only a 10 percent probability that the variable will be less than or equal to this value– What would have caused a value this low?

• There is only a 10 percent probability that the variable will be greater than this value.– What would have caused a value this high?

• There is a 50 percent probability that the variable will be less than or equal to this value.– Would you bet above or below this value?

High

Base Case

INFORMS 05-01-06 32 © 2000-2006

Sensitivity “tornado diagram” identified the uncertainties key to value – there can be only a few.

–900 –600 –300 0 300 600 900

Change in NPV—Evolutionary High-End TV Strategy ($ millions)

Videophile Segment Size in 2000 Low High

Japanese 2nd-GenerationCost Advantage Yes None

36" 16:9 TV Cost High Low

New Broadcast Standard(when and what) None Terrestrial 16:9 (1995)

and analog HD (2001)

Proportion of TV > 30 inches Low High

Development Costs With government funding

Impact of Earlier TV Replacement Low High

Distribution Markup Low High

Programming Availability Response High Low

INFORMS 05-01-06 33 © 2000-2006

Aggressive Analog was dominated. Info Age a had higher upside but more risk than Evolutionary Improvement.

Rather than trying to reach a decision, the search began for a better alternative.

Cum

ulat

ive

Prob

abili

ty

1.0

.9

.8

.7

.6

.5

.4

.3

.2

.1

0

–2Net Present Value ($ billions)

AggressiveAnalog

EvolutionaryImprovement

InfoAge

–1 0 1 2 3 4 5

INFORMS 05-01-06 34 © 2000-2006

Based on an understanding of the original three alternatives, the team constructed a new hybrid strategy “Sooner and Later.”

Focusing on feature leadership in conventional TV, dropping analog HDTV, and devoting some resources to a cooperative, slower, digital development was the best approach.

Cum

ulat

ive

Prob

abili

ty

1.0

.9

.8

.7

.6

.5

.4

.3

.2

.1

0

–2Net Present Value ($ billions)

AggressiveAnalog

EvolutionaryImprovement

InfoAge

Soonerand Later

–1 0 1 2 3 4 5

INFORMS 05-01-06 35 © 2000-2006

The company committed to action as a result of a shared understanding of the business.

• Customer needs• Our capabilities• Competition

SharedUnderstanding of

Our Business

Information and entertainment will merge—we should slow down and develop a digitalHDTV.

If we don’t offer an alternative ASAP, theJapanese will destroy us.

These HDTV sets will be so expensive no one will ever buy one.

Before After

! !!

INFORMS 05-01-06 36 © 2000-2006

In this problem, like many corporate problems, risk attitude was not an issue.

• People continually confuse uncertainty and risk attitude

• The first order of business is to focus on value creation and the drivers of uncertainty in achieving high value. – This often leads to better strategies, and near stochastic dominance– In a learning situation, it is often better to proceed for a while to get

more information (and creativity) rather then tweaking a strategy

• If risk attitude seems important, it is usually tested by sensitivity to risk tolerance (exponential utility - which has constant risk tolerance)– In 50 years of professional practice, I have only seen one case in the

corporate domain that needed to go beyond exponential utility.

INFORMS 05-01-06 37 © 2000-2006

Decision Analysis in the Twenty-First CenturyA paradigm shift!

According to Thomas Friedman, the World is now “Flat.”

INFORMS 05-01-06 38 © 2000-2006

Today’s trends are forcing the evolution of decision analysis methods and delivery systems.

• Organizations are once again into decentralization and empowerment– Agile organizations push responsibility as low as possible

• Executives and managers are much better trained and familiar with decision analysis and similar approaches.– In the 60’s and 70’s most had no clue about decision analysis

• Everyone is looking for instant gratification, and they expect high-speed decision analysis– They are used to having information at their fingertips, and

participating in global teams– They want their strategic decisions made quickly, so they can move

on to operational effectiveness.

• By making the process of strategy creation and decision operational, we play into their new mindset– Strategy and operational thinking are very different.– But the process of creating strategy is an operational process.

INFORMS 05-01-06 39 © 2000-2006

Where does the decision analysis profession stop?

1. Decision Analyst - responsible for processing numbers

2. Decision Facilitator - responsible for meetings

3. Decision Consultant - responsible for attaining commitment

4. Decision Engineer - responsible for process, systems and organizational design

5. Decision Change Agent - responsible for personal, organizational, and cultural change necessary for routine, high quality decision making

INFORMS 05-01-06 40 © 2000-2006

The Ten Commandments of Decision Analysis

1. Decision Analysis is the One Master Discipline – do not follow false fads.

2. Work for the Decision-Maker and serve the organization he/she represents.

3. Construct a monetary value measure – nobody puts “scores” in the bank.

4. Beware of rates of return – nobody puts ratios in the bank either.

5. Have no regret – do not covet the pie you did not get or the other guy’s pie.

6. Beware of difference lotteries – they never happen.

7. Beware of triage – often “no brainers” have the biggest improvement potential.

8. If the problem is technically hard, change your frame – avoid “constraints”

9. Start simple and iterate – use the simplest model that gives decision clarity.

10.Change with the times and join in the new paradigm – the world is now flat!

INFORMS 05-01-06 41 © 2000-2006

Coming Attractions!

• SmartOrg Software Demonstration– Tuesday 3:00 PM– Decision Advisor® — Create and Analyze Value-Based Models– Portfolio Navigator™ — Deploy over the web

• Free Software Trials– Come to SmartOrg Exhibit Booth

INFORMS 05-01-06 42 © 2000-2006

Selected References• Friedman, Thomas L. 2005. The World Is Flat: A Brief History of the

Twenty-first Century, Farrar, Straus and Giroux

• Howard, Ronald A. and James E. Matheson eds. 1983 Readings on the Principles and Applications of Decision Analysis, Strategic Decisions Group.

• Howard, Ronald A. and James E. Matheson 1981. Influence Diagrams, Reprinted in Decision Analysis, Vol 2, No 3, Sept 2005, pp 127-143.

• Matheson, David and Jim Matheson, 1998. The Smart Organization: Creating Value Through Strategic R&D, Harvard Business School Press.

• Matheson, David and James E. Matheson, 2001. Smart Organizations Perform Better in Research • Technology Management, July-August 2001, pp 49-54.

• McNamee, Peter and John Celona, 2001. Decision Analysis for the Professional, Third Edition, SmartOrg, Inc.

• Spetzler, Carl S. and Carl-Axel S. Staël van Holstein. 1975. Probability Encoding in Decision Analysis, Management Science 22, pp 340-358.