The Art of the Trade Study Half Day Tutorial 30April2012

©2012 Strategy Bridge International Inc.

The Art of the Trade Study

Presented by

Mark A. Wilson [email protected]


Introductions

Please tell us your: § Name § Organization § Expectations

Fig. 2


Classroom Logistics

Some logistics for this tutorial: l Start and End Times l Breaks l Restrooms l Phone Calls, Cell Phones, etc. l Active Participation!

Fig. 3


Recommended Reading

Robert Clemen Hammond, Keeney, & Raiffa

Murnighan & Mowen

Fig. 4


Tutorial Objectives

n  Participants will be able to: l  Explain the rationale for formal trade study decisions

l  Apply a decision analysis process to analyze alternatives and select a preferred alternative

l  Analyze decision frame for appropriateness to decision context

l  Create useful decision criteria

l  Apply two different methods for modeling decisions

l  Explain common cognitive biases in technical decision-making

Fig. 5


Obstacles to Effective Decision-Making

What obstacles to effective decision-making do you encounter on your project(s)?

Fig. 6


What is a decision?

n A decision is a response to a situation in which: l There is more than one possible course of action; l The decision-maker can form expectations about the

outcomes following each possible course of action; and,

l Each outcome has an associated consequence that can be evaluated

Kenneth Hammond, University of Colorado

Fig. 7


Why bother to analyze decisions?

n  Adds structure to decision process l Structured thinking can help overcome biases l Formal structure permits decision traceability

n Permits others to understand decision n Allows decision process improvement

n  Provides a better outcome than random choice l We can’t control outcomes; the best we can do is

influence the probability of certain outcomes

As I think back over the years, I have been guided by four principles in decision making. First, the only certainty is that there is no certainty. Second, every decision, as a consequence, is a matter of weighing probabilities. Third, despite uncertainty we must decide and we must act. And lastly, we need to judge decisions not only the results but on how they were made.

Robert Rubin, 1999

Fig. 8


Why is Decision Analysis Important to Systems Engineering?

n  SE’s must constantly make decisions to define, prioritize, and balance technical, cost, and schedule concerns l Proper technical decision making must balance:

n  Cost effectiveness n  Performance n  Schedule n  Supportability n  Robustness

n  The Trade Study is a core SE skill l  Facilitates visible, traceable, justifiable, efficient decisions

Fig. 9

©2012 Strategy Bridge International Inc. Fig. 10

Decision Making and SE: ISO/IEC 15288 Perspective

©2012 Strategy Bridge International Inc. Fig. 11 Source: DAU ACQuipedia and Defense Acquisition Guide, Chapter 4

Technical Management Processes Technical Planning

Technical Data Management

Requirements Management

Configuration Management

Decision Analysis

Technical Assessment

Interface Management

Risk Management

Requirements Definition

Requirements Analysis

Architecture Design

Implementation Integration

Verification

Validation

Transition

Technical Processes

Decision Making and SE: U.S. DoD Perspective


Decision Making and SE: NASA Perspective

NPR 7123.1A, Chapter 3


NDIA U.S. DoD SE Effectiveness Survey (2005 – 2007)

Strong Relationship Moderately Strong to Strong Relationship

Moderately Strong Relationship Weak Relationship

Relationship of SE Processes to Program Performance

-13%

13%

13%

21%

25%

28%

28%

33%

34%

36%

37%

40%

-20% -10% 0% 10% 20% 30% 40% 50%

Project Monitor/ControlProject Planning

Config MgmtProduct Integration

VerificationRisk MgmtValidation

Reqts Devel & MgmtIPT Capability

Technical SolutionTrade Studies

Architecture

SE C

apab

ility

Gamma (strength of relationship)


Human Factors

Psychological Dimensions of Decision Making


What is Intuition?

n Pattern recognition without evident conscious thought

n Humans tend to alternate between automatic thinking (intuition) and controlled thinking (analysis) when making decisions

n Our subconscious can find patterns in situations and behavior based on very narrow slices of experience (John Gottman)

Where have you relied on intuition to make a decision?

Fig. 15


How do Engineers really make decisions?

Ana

lytic

Pro

cess

ing

(Rat

iona

l Ana

lysi

s)

Autom

atic Processing

(Intuition)

How often do you use your intuition to make engineering decisions?

Fig. 16


Decision Making

“The intuitive mind is a sacred gift and the rational mind is a faithful servant. We have created a society that honors the servant and has forgotten the gift....” Albert Einstein, Scientist, (1879-1955)

Fig. 17


Analysis or Intuition?

n  Rational analysis and rapid cognition (intuition) are not mutually exclusive l  Analysis is putting the facts in order and deciding based on the

importance of each fact l  Intuition is looking at the same facts and discerning a pattern

n  Dominance of technique is dictated by the situation l  Intuition - high speed; experience based pattern matching l  Analysis - not time-critical; lots of data; optimal solution needed;

team participation; need to explain / defend the decision to others

Which decision-making technique would you use if you are trying to avoid an object rapidly approaching your head?

Fig. 18


The Trouble with Intuition

l  Group Think l  Status Quo Bias l  Overconfidence Bias l  Availability Bias

Common Effects:

•  Failure to critically examine all alternatives

•  Tendency to continue to “do things the way we we’ve done them”

•  Illusion of control over stochastic events

•  Overestimation of probability of desired outcomes

Fig. 19

l  Hindsight Bias l  Confirming Evidence Bias l  Anchoring Bias l  Sunk Cost Bias l  Input Bias

n  Common Cognitive Biases in Engineering Decisions


Judgments Under Stress

How does “stress” (time, cognitive load, fatigue) affect human decision-making?

Principle of “Constancy”

- All organisms attempt to maintain stable relations with their environment

-  Disruption of stabilized relations presents a threat that induces a cognitive response to reestablish stability

-  Loss of constancy = stress, which triggers SNS “fight or flight”

- Autonomic nervous system = physiological activities over which you have no conscious control (heartbeat, digestion, perspiration…)

- sympathetic nervous system (SNS) – stress response

- parasympathetic nervous system (PNS) - relaxation

Fig. 20


To Counter Biases n  Call in external resources or outside experts to review your work whenever

possible. n  Have someone you trust review your work. n  Keep an open mind. Question everything. n  Think objectively by placing yourself in the position of an outsider. n  Get in the habit of questioning all assumptions. n  Engage a devil’s advocate when necessary, and encourage healthy debate. n  When possible, have two teams work on the same request to see what

differences emerge in their conclusions. n  Do not hold a personal attachment or assign over-importance to your own

recommendations. n  Rotate roles and responsibilities within your team so that a new set of fresh

eyes can help to uncover unknown bias. n  As a manager, try and hold your opinions to yourself. Allow your team to come

up with their own recommendations and solutions first, so that you do not bias their thinking.

Fig. 21


Evaluating Alternatives

A Recommended Decision Process for Systems Engineers


How are your technical decisions typically made?

BOGGSAT?

What decision aids, methods or tools do you use?

Fig. 23


Decision-Making Process Agree on

Decision Objectives “Framing”

Model the Decision

Identify Alternatives

Choose an Alternative

Conduct Sensitivity Analysis

Analyze Further?

Implement the Selected Alternative

No

Yes

Fig. 24


When asked what single event was most helpful in developing the theory of relativity, Albert Einstein reportedly answered:

“Figuring out how to think about the problem”

Fig. 25


The Master Decision-Making Skill: Framing

Context is the surroundings, circumstances, environment, background,

or settings which determine, specify, or clarify the meaning of an event.

How you define a problem will largely determine how you will go about solving it

Frame = Decision Context

Fig. 26


n  Abstract ideas that influence thinking and action in the organization

n  Behaviors that are celebrated or rewarded reflect true values

Culture and Risk Philosophy are crucial to the decision frame

Context of implementation will drive: n  Design n  Design Margin n  Reliability n  Quality

Fig. 27


Structuring Decision Objectives

n Technique: l Create an initial list of objectives l Create an objectives network l Separate “means” from “fundamental”

objectives n Means objectives: help achieve other objectives n Fundamental objectives: reflect what we really want n Key Question: “Why is that important?”

l Organize the fundamental objectives into a hierarchy

Fig. 28


Means Objective Network Example: What are all the things that are

important to you in selecting a car? Gets me to my

destination safely

Good reliability

Gets good gas mileage

Protects me if I’m in an accident

Know where I’m going Merges well

into traffic

Comfortable ride

Has airbags Has a GPS

Accelerates well

Looks attractive

Fig. 29

Reasonable cost to maintain

Price under $35K

Affordable LCC


Asking “Why?”

1.  Frame the problem initially. 2.  Ask “why is this important? ” 3.  Answer the question 4.  Why is that important? 5.  Repeat as necessary.

Fig. 30


Fixation on What’s Wrong People often get stuck by focusing all their

efforts on trying to fix what’s wrong instead of working toward what they truly want

“How we can fix it”

“What We Want”

X

Goal

(Specific Solutions)

Fig. 31


Understanding Fundamental Objectives – Example

l  Boeing wanted to use an aluminum-lithium alloy in the Boeing 777 wing structures to provide the opportunities of lightness, more payload, and increased fuel economy

l  However, aluminum-lithium experienced cosmetic surface cracking that could give a marketing advantage to Airbus

l  Boeing’s management decided the performance advantage was not worth the potential negative publicity

Net Expected Outcome

Expected Outcome of Risks Expected Outcome of Opportunities

| | | | | | | | | | | | | | | | | | | | |

-10 -5 0 +5 +10

Net Expected Value

Expected Value of Risks Expected Value of Opportunities

| | | | | | | | | | | | | | | | | | | | |

-10 -5 0 +5 +10| | | | | | | | | | | | | | | | | | | | |

-10 -5 0 +5 +10

Fig. 32


Helpful Questions

n  What is your ultimate objective?

n  What is the crux of the issue or problem?

n  How does the decision affect other decisions?

n  What information do we have about similar problems or decisions made previously?

n  How will we implement the solution?

“There are no decision aids that can structure a problem automatically. Rather, this crucial phase must be largely achieved through unaided human judgment.”

- Judgment and Choice, Robert Hogarth

Fig. 33


Case Study Discovery Activity

n Read the Factory Acceptance Test Failure Case Study and “frame” the decision problem from the perspective of the Contractor.

Fig. 34


Decision-Making Tip

n The proper decision frame is a necessary precondition to a satisfactory decision outcome.

n Spend sufficient effort on establishing the proper frame.

Fig. 35




Model the Decision




Analyze Further?


No

Yes

Fig. 36


Generating Alternatives

“Alternatives are the raw material of decision making” -Smart Choices by Hammond, Keeney & Raiffa

After the problem has been framed, ask: “How can we obtain the desired outcome?”

n  Challenge constraints – look at the problem from new angles

n  Be creative, let process diverge n  Gather information, if necessary n  Withhold judgment until the

evaluation phase

Class Discussion: How do you ensure that you are not considering the same

old alternatives while falling into a “status quo” trap? Fig. 37



n Assume you are the Contractor PM in the case, what alternatives can you generate in the FAT Failure scenario? l Hint: look beyond the obvious alternatives

Fig. 38


Parting Thoughts on Alternatives

n Separate alternative generation from alternative evaluation l Don’t fall into the status quo trap

n Alternative generation may cause you to redefine your decision frame l Some decision objectives may be modified or deleted l Decision objectives may be added as knowledge grows l Priorities of decision objectives may change

Fig. 39




Model the Decision




Analyze Further?


No

Yes

Fig. 40


Most Decision Problems are Multicriteria

n  Satisfy science requirements

n  Maximize design life

n  Minimize lifecycle cost

n  Maximize reliability

n  Minimize costs of production

n  Satisfy political stakeholders

Decision Criteria: The means by which a decision-maker measures the attributes of

alternatives in order to identify and assess discriminators

Fig. 41


Selecting Evaluation Criteria

n High quality decision criteria: l Are limited to those which will yield meaningful

discrimination between alternatives l Are reasonably independent of one other l Can be consistently defined and applied

n Standards or factors can be defined

Fig. 42


Common Mistakes

n Using fuzzy definitions for objectives l e.g., required capabilities

n Breaking objectives down too far l e.g., breaking cost into cost elements when from a

value perspective a $ is a $

n Using proxies in the objectives hierarchy rather than as measures l e.g., gas mileage is proxy, travel cost is objective

n Using a generic measure for many objectives l Particularize a measure for each objective

Fig. 43



n Define evaluation criteria suitable for choosing a preferred alternative if you are the Contractor PM in the FAT Failure case study. l Are some criteria more important than others?

Fig. 44


Common Decision Modeling Methods

n  Pugh Matrix n  Decision Trees n  Multi-attribute Utility n  Utility Curves n  Kepner-Tregoe n  Analytic Hierarchy Process (AHP)

Methods can be mixed to address different aspects of a particular decision. Choose the method best for the decision situation or the aspect of the decision.

Fig. 45


Trade Study Example: Choose a new laptop computer for travel

Criteria" Threshold" MacBook"Air"

Lenovo"ThinkPad"

T420"

Sony"Vaio Z"

Dell"XPS 13"

Weight" < 5" 3" 5" 2.5" 5"

Screen"Size (inches)"

> 13" 13" 14" 13" 13"

Memory (GB)" > 4" 4" 4" 4" 4"

Hard"Drive "(GB)"

256" 320" 128"

Price ($)" < 1000" 1600" 800" 2000" 1000"

> 128" 128"

Fig. 46


Pugh Matrix Technique

n  Create a matrix of decision criteria mapped to alternatives

n  Clarify true constraints (“must” conditions) l  Eliminate any alternative that does not meet a constraint

n  Rate each alternative on each criterion using: - (does not meet threshold requirement)

S (meets threshold requirement)

+ (exceeds threshold requirement)

n  Sum the values and the preferred alternative may be obvious

Fig. 47


Pugh Matrix Example

Criteria" Threshold" MacBook"Air"

Lenovo"ThinkPad"

T420"

Sony"Vaio Z"

Dell"XPS 13"

Weight" < 5" 3" 5" 2.5" 5"

Screen"Size (inches)"

> 13" 13" 14" 13" 13"

Memory (GB)" > 4" 4" 4" 4" 4"

Hard"Drive "(GB)"

256" 320" 128"

Price ($)" < 1000" 1600" 800" 2000" 1000"

> 128" 128"

Fig. 48

S (0) for “meets;” +1 for “exceeds;” –1 for “doesn’t meet”

+1 +3 0 0


Utility Theory


Utility Theory Foundations

n Daniel Bernoulli theorized that rational decision-makers will try to maximize expected utility (subjective desirability) rather than expected value. l Papers of the Imperial Academy of

Sciences in St. Petersburg (1738)

…utility resulting from any small increase in wealth will be inversely proportionate to the quantity of goods previously possessed. Daniel Bernoulli

Fig. 50


Utility Theory

n  The fundamental assumption: A Decision-Maker rates their preference for the commodity or criteria to maximize their expected value of that commodity or criteria. l  If true, then utility theory can be used to determine the score of

that criteria for the various alternatives. l  A utility score has to be assigned to each alternative. l  A utility function depends on the preferences of the individual

decision maker. l  The utility measures of the criteria are assumed to reflect a

decision maker's preferences in the following sense: n  (i) the numerical order of utilities for criteria preserves the decision

maker's preference order among the criteria; n  (ii) the numerical order of expected utilities of alternatives

preserves the decision maker's preference order among these alternatives.

Fig. 51


Utility Functions

n  Binary answer l  Yes/no l  Single value (energy to orbit) l  Minimum or maximum

n  Variance over range, for example l  Near spec in critical l  Above has some value

n  Optimum value in a range l  Upper and lower limits l  Best value in between l  E.g., passengers in commuter plane

Value"

Scal

e"1"

0"

Value"

Scal

e"

1"

0"

Value"

Scal

e"

1"

0"

Fig. 52


Applying Utility Theory

n  A rational person must be able to measure utility under all circumstances and to make choices and decisions accordingly

n  The facts are the same for everyone l  Consider: Are facts subject to perception?

n What is the utility of a car that can accelerate to 250 mph versus one that only goes to 200 mph?

n  What if you are buying the car for your teenager? n  What if you’re driving in the Indianapolis 500?

n  Utility curves represent value for a particular stakeholder or group of stakeholders with common values

Fig. 53


Prospect Theory

n Developed by Daniel Kahneman and Amos Tversky in the 1970’s

n Basis: l Emotion often destroys self-control that is essential to

“rational” decision-making l Humans are subject to (unrecognized) cognitive

biases

Fig. 54


Gains and Losses (Part 1)

n Decision 1 l Option A:

n 80% chance of winning $4K n 20% chance of winning zero

l Option B: n 100% chance of winning $3K

Fig. 55


Gains and Losses (Part 2)

n Decision 2 l Option C:

n 80% chance of losing $4K n 20% chance of losing nothing

l Option D: n 100% chance of losing $3K

Fig. 56


Analytic Hierarchy Process (AHP)


Can you compare apples to oranges?

How can you compare dissimilar things?

Fig. 58


Introduction

n AHP, developed by Tom Saaty, is a decision-making method for prioritizing alternatives when multiple criteria must be considered. l An approach for structuring a problem as a hierarchy,

or set of integrated levels.

n Designed to handle objective and subjective measures l Relative measurement (pair-wise comparisons) allows

decision-makers to be scale-independent

Fig. 59


AHP Approach

n How does AHP capture human judgments? l AHP does not require you to make an absolute

judgment or assessment l Process uses relative assessment between two items

at a time

n In relative measurement a preference or judgment is expressed on each pair of elements with respect to the common “parent” element

Fig. 60


1- Equal importance

3- Moderate importance of one over another

5 -Strong or essential importance

7- Very strong or demonstrated importance

9- Extreme importance

2,4,6,8 Intermediate values

Saaty Scale for Verbal Comparisons

Fig. 61


Applying AHP

n The decision-maker: l measures the extent to which each alternative

achieves each criterion, and l determines the relative importance of the criteria in

meeting the goal, and l synthesizes the results to determine the relative

importance of the alternatives in meeting the goal. n Synthesis is a key strength of the technique n Engineers are trained to analyze

Fig. 62


The Analytic Hierarchy Process (Saaty - 1971)

Decision Criteria

Goal

Alternatives Fig. 63


Example: Choose a New Telemetry Solution for Our Satellite Constellation

n  Operability l  Measure of the ease with which an operator can comprehend the

operating concept and operate the equipment n  Maintainability

l  Mean Time to Repair (MTTR) n  Reliability

l  Mean Time Between Failures (MTBF) n  Option Confidence (i.e., will work as advertised)

l  Degree to which equipment will operate properly with spacecraft l  Product maturity

n  Schedule Risk l  Risk that option can be delivered in time to meet next launch

requirement n  Growth Potential

l  To be able to have a device which is modular and readily expandable and upgrade-able

l  Capability of the design to readily accommodate technology insertion

Fig. 64


Fundamental Objectives Hierarchy

Operability

Decision Frame

Maintainability Reliability Schedule Risk

Growth Potential

Select a New Telemetry Solution

Option Confidence

Fig. 65


Example Top Level Criteria Weights

Row Total Weight

Operability Maintainability Reliability Option Confidence Schedule Risk Growth

Potential

Operability 1 3 3 0.2 2 0.33 9.53 13.2%Maintainability 0.33 1 0.25 0.2 5 0.33 7.11 9.9%Reliability 0.33 4 1 0.2 6 3 14.53 20.1%Option Confidence 5 5 5 1 9 3 28 38.8%Schedule Risk 0.5 0.2 0.167 0.11 1 0.33 2.307 3.2%Growth Potential 3 3 0.33 0.33 3 1 10.66 14.8%

72.137Grand Total

1 Equal importance 3 Moderate importance of one over another 5 Strong or essential importance 7 Very strong or demonstrated importance

9 Extreme importance 2,4,6,8 Intermediate values

Fig. 66


GFE Option Supplier A Supplier B

Operability 13%

GOAL

Maintainability 10%

Reliability 20%

Schedule Risk 3%

Growth Potential

15%

Select Telemetry Solution

Option Confidence

39%

Fig. 67


Evaluating Alternatives - Measures

n Decision criteria relate to alternatives by measuring the attributes of each alternative for each criterion

n Attributes are measures of achievement l Quantitative if available

Fig. 68


Build Rating Scales

n Choose ratings scales that make sense n Avoid constructed scales that are

vulnerable to individual interpretation l For example, Good-Better-Best, A-B-C, etc.

n Tailored ratings scales for each criterion l Use natural (objective) measures, if possible

Fig. 69

Build utility curves for each criterion.


n In general use and have a common interpretation to everyone l Speed in miles per hour

l Cost in U.S. Dollars

l Weight in tons

l Size in cubic feet

Natural Measures

Fig. 70


n Integrates multiple numerical and/ or verbal descriptions into a single description of the state of a fundamental objective. Applies to a specific decision context. l Usually a collection of factors

n Poor, Acceptable, Excellent n Low Risk, High Risk

l Context Dependent n Subjective scales are subject to varying interpretations

among decision-makers

Constructed Measures

Fig. 71


n  Normally means objectives that have a derived, or

implied relationship to a fundamental objective

l  Used because it relates to a fundamental objective.

l  Could be a natural measure.

l  Example: Vehicle weight in pounds as a proxy for

comfort........heavier cars tend to ride better.

Proxy Measures

Fig. 72


A Simple Model for Rating Alternatives

Sample Ratings Model

Operability Maintainability ReliabilityOption

Confidence Schedule RiskGrowth

Potential Totalsweight 13% 10% 20% 39% 3% 15%

Supplier A 100 60 60 60 30 100 70.3

GFE Option 100 60 100 100 60 30 84.3

Supplier B 30 100 100 30 100 60 57.6

100 < 40 hrs Training MTTR < 8 p-hrs MTBF >10K hrs Very High Low Modular Design60 40-80 hrs Training MTTR 8 - 24 p-hrs MTBF 5K - 10K hrs High Moderate Upgradeable30 80-100 hrs Training MTTR 24 - 40 p-hrs MTBF 1K - 5K hrs Moderate High Poor Growth Potential0 >100 hrs Training MTTR > 40 p-hrs MTBF < 1K hrs Low Very High Closed Architecture

Fig. 73


Example Criteria for Allocating Facilities Resources to Maximize Enterprise Benefit

n  Health and Safety l  Directly impacts the physical well-being of personnel

n  Mission Impact l  Directly enables operations, collection, or analysis

n  Regulatory Compliance l  Addresses a requirement imposed by Executive Order or legal

mandate

n  Security/CI Risk l  (Probability) X (Consequence) of unauthorized access, use,

disclosure, disruption, modification, perusal, inspection, recording or destruction

n  FCI Mitigation l  Addresses a Facilities Condition Index rating (measure of

deferred maintenance) Fig. 74


Decision Frame Allocate Facilities

Resources to Maximize Enterprise

Benefit

20% 20% 15% 35% 10%

Establish Relative Weightings of Criteria

Health & Safety

Regulatory Compliance

Mission Impact

Security/CI Risk

FCI Mitigation

Decision Criteria

Fig. 75


B A …N

Decision Frame Allocate Facilities

Resources to Maximize Enterprise

Benefit

C D E

20% 20% 15% 35% 10%

Rate Each Alternative on All Criteria

Health & Safety

Regulatory Compliance

Mission Impact

Security/CI Risk

FCI Mitigation

Decision Criteria

Alternatives Fig. 76


Example Tailored Scales

Fig. 77


Summary of AHP Advantages

n  Easy to use in trade studies n  Organizes, prioritizes and synthesizes complexity

within a rational framework n  Breaks down tangible and intangible criteria into

manageable components n  Fosters critical discussion and examination of

implicit assumptions when used with diverse groups l  Makes it possible to deal with conflicts in perception and in

judgment

Fig. 78




Model the Decision




Analyze Further?


No

Yes

Fig. 79


Making Choices

Fig. 80


Trade Study Flow Chart Identify the decision situation-

understand the objectives “Framing”


Model-Evaluate the Decision



Analyze Further?


No

Yes

Fig. 81


n Sensitivity analysis answers the question: “What makes a difference in this decision?”

n Sensitivity analysis determines whether small changes in judgments affect the final weights and rankings of the alternatives

l May lead the decision-maker to reconsider the decision frame or the sufficiency of alternatives

l Examine the possible effect of cognitive biases

n If so, the decision-maker may want to review the sensitive judgments

Sensitivity Analysis

Fig. 82


How do you set your decision trigger?

Missed opportunity results in worse consequences

Inappropriate choice is more damaging

Neutral Bias

You must assess the costs of potential negative outcomes for each type of error

Bias to gathering more information Bias toward

action

Fig. 83




Model the Decision




Analyze Further?


No

Yes

Fig. 84


Tutorial Summary

Approach When to Use Tool

Intuitive Decisions

n Expert decision maker n No stakeholder review required n Dynamic, fast-moving situations n Absence of supporting

information

Decision Tree

n Multiple options involving probabilities

n Alternatives involve a sequence of events

n Stakeholder review

Multi-attribute Value Models

n Stakeholder review n Team participation n Subjective evaluation involved

GOAL

Fig. 85


Parting Thoughts…

n There is no tool, process or system that can “make a decision” for you

n All decisions are subjective

n You should think about how a particular decision should be approached, but ultimately it is your job to…decide.

Fig. 86


Systems Engineer

“Toolkit”

Questions?

Configuration Management

Test

Engineering

Risk

Management

Cost Analysis

Team Leadership

Technical Decision-Making

Integration

Modeling & Simulation

Technical Planning

Requirements Management

Architecting

Technology

Assessment CONOPS

Formulation

Fig. 87


Date post:	19-Feb-2022
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

The Art of the Trade Study Half Day Tutorial 30April2012

Documents