INCOSE CAB Briefing November 2002 COSYSMO COnstructive SYStems Engineering Cost MOdel November 1,...

INCOSE CAB BriefingNovember 2002

USC

C S E University of Southern CaliforniaCenter for Software Engineering

COSYSMOCOnstructive SYStems Engineering Cost MOdel

November 1, 2002

Dr. Barry BoehmRicardo Valerdi

University of Southern CaliforniaCenter for Software Engineering (CSE)

INCOSE CAB Briefing


USC


Outline• Background on CSE, COSYSMO, and COCOMO II• COSYSMO Overview

– Operational concept and scope– Prototype demo

• Model Progress to Date– Front end sizing and drivers – Full life cycle sizing and drivers

• Calendar of activities/milestones• Action items• How can the CAB help?


USC


USC Center for Software Engineering (CSE)• Researches, teaches, and practices CMMI-based

Software engineering – Systems and software engineering fully integrated

• Focuses on better models to guide integrated systems and software engineering – Success models: stakeholder win-win, business

cases – Product models: requirements, architectures, COTS– Process models: spiral extensions, value-based RUP

extensions – Property models: cost, schedule, quality

• Applies and extends research on major programs (DARPA/Army, FCS, FAA ERAM, NASA Missions)


USC


• Commercial Industry (15)

– Daimler Chrysler, Freshwater Partners, Galorath, Group Systems.Com, Hughes, IBM, Cost Xpert Group, Microsoft, Motorola, Price Systems, Rational, Reuters Consulting, Sun, Telcordia, Xerox

• Aerospace Industry (6)

– Boeing, Lockheed Martin, Northrop Grumman, Raytheon, SAIC, TRW

• Government (8)

– DARPA, DISA, FAA, NASA-Ames, NSF, OSD/ARA/SIS, US Army Research Labs, US Army TACOM

• FFRDC’s and Consortia (4)

– Aerospace, JPL, SEI, SPC

• International (1)

– Chung-Ang U. (Korea)

USC-CSE Affiliates (34)


USC


USC-CSE Cost, Schedule, and Quality Models

• Build on experience with COCOMO 1981, COCOMO II – Most widely used software cost models worldwide– Developed with Affiliate funding, expertise, data support

• Collaborative efforts between Computer Science (CS) and Industrial Systems Engineering (ISE) Depts.– 3 CS PhD’s, 2 ISE PhD’s to date– Valerdi an ISE PhD student– Boehm joint appointment in CS, ISE

• COCOMO Suite of models– Cost, schedule: COCOMO II, CORADMO, COCOTS– Quality: COQUALMO– Systems Engineering: COSYSMO

• Uses mature 7-step model development methodology


USC


7-step Modeling MethodologyAnalyze Existingliterature

1

2

3

4

5

6

7

PerformBehavioral Analysis

Identify RelativeSignificance

Perform Expert-Judgement, DelphiAssessment

Gather Project Data

Determine BayesianA-Posteriori Update

Gather more data;refine model

A-PRIORI MODEL +

SAMPLING DATA =

A-POSTERIORI MODEL


USC


COSYSMO: Overview• Parametric model to estimate system

engineering costs• Covers full system engineering lifecycle• Focused on use for Investment Analysis,

Concept Definition phases estimation and tradeoff analyses– Input parameters can be determined in

early phases


USC


Key Members of the COSYSMO Working Group

Karen Owens, Marilee WheatonEvin StumpGarry Roedler, Gary HafenGary Thomas, John RieffTony Jordano, Don GreenleeChris MillerMarilee WheatonCheryl JonesBarry Boehm, Elliot Axelband,

Don Reifer, Ricardo Valerdi

Aerospace Corp.Galorath

LMCORaytheon

SAICSPCTRW

US Army/PSSMUSC


USC


COSYSMO

SizeDrivers

EffortMultipliers

Effort

Duration

Calibration

# Requirements# Interfaces# Scenarios# Algorithms

+Volatility Factor

- Application factors-5 factors

- Team factors-7 factors

- Schedule driverWBS guided by EIA/ANSI 632 & ISO/IEC 15288

COSYSMO Operational Concept


USC


Previous COSYSMO Evolution Path

Inception Elaboration Construction TransitionOper Test & Eval

1. COSYSMO-IP

2. COSYSMO-C4ISR

3. COSYSMO-Machine

4. COSYSMO-SoS

IP (Sub)system

C4ISR System

Physical MachineSystem

System of Systems (SoS)


USC


Revised View of COSYSMO Evolution Path(Results from last week’s meeting)

Oper Test & Eval

1. COSYSMO-IP

2. COSYSMO-C4ISR

3. COSYSMO-Machine

4. COSYSMO-SoS

Global Command and Control System

Satellite Ground Station

Joint Strike Fighter

Future Combat Systems

Initiate data collection for all and let the amount of data received determine what is included.

Include ISO/IEC 15288 Stages

DevelopConceptualizeTransition to Operation

Operate, Maintain, or Enhance

Replace or Dismantle


USC


EIA/ANSI 632

EIA/ANSI 632 - Provide an integrated set of fundamental processes to aid a developer in the engineering or re-engineering of a system

Breadth and Depth of Key SE StandardsSystem life

ISO/IEC 15288

Level of

deta

il

Conceptualize DevelopTransition to

Operation

Operate,Maintain,

or EnhanceReplace

or Dismantle

Processdescription

High levelpractices

Detailedpractices

ISO/IEC 15288 - Establish a common framework for describing the life cycle of systems

Purpose of the Standards:Purpose of the Standards:

IEEE 1

220

IEEE 1220 - Provide a standard for managing systems engineering

Input to 632/1220

Source : Draft Report ISO Study Group May 2, 2000


USC


ISO/IEC 15288 Key Terms• System

– a combination of interacting elements organized to achieve one or more stated purposes

• System-of-Interest– the system whose life cycle is under consideration in the context of this

International Standard

• System Element– a member of a set of elements that constitutes a system

– NOTE: A system element is a discrete part of a system that can be implemented to fulfill specified requirements

• Enabling System– a system that complements a system-of-interest during its life cycle

stages but does not necessarily contribute directly to its function during operation

– NOTE: For example, when a system-of-interest enters the production stage, an enabling production system is required

Source: ISO/IEC 15288.Source: ISO/IEC 15288.


USC


Systemelement

System-of-interest

Systemelement

Systemelement

SystemelementSystem

Systemelement

Systemelement

Systemelement

System

Systemelement

Systemelement

Systemelement

System

Systemelement

Systemelement

SystemelementSystem

Systemelement

Systemelement

System

Systemelement

Systemelement

System

Systemelement

Systemelement System

Systemelement

Systemelement

Systemelement

ISO/IEC 15288 System of Interest Structure

Make orbuy

Source: ISO/IEC 15288.Source: ISO/IEC 15288.


USC


Raytheon myCOSYSMO* Demo

*Developed by Gary Thomas

at Raytheon Garland


USC


Outline• Background on CSE, COSYSMO, and

COCOMO II• COSYSMO Overview





USC


4 Size Drivers1. Number of System Requirements

2. Number of Major Interfaces

3. Number of Operational Scenarios

4. Number of Unique Algorithms

• Each weighted by complexity, volatility, and degree of reuse


USC


Size Driver Definitions (1 of 4)

Number of System RequirementsThe number of requirements taken from the system

specification. A requirement is a statement of capability or

attribute containing a normative verb such as shall or will. It

may be functional or system service-oriented in nature

depending on the methodology used for specification. System

requirements can typically be quantified by counting the

number of applicable shall’s or will’s in the system or

marketing specification.

Note 1: Use this driver as the basis of comparison for the rest of the drivers.

Note 2: Use equivalent size weighted by complexity, volatility, and degree of reuse.


USC


Size Driver Definitions (2 of 4)Number of Major InterfacesThe number of shared major physical and logical

boundaries between system components or functions

(internal interfaces) and those external to the system

(external interfaces). These interfaces typically can be

quantified by counting the number of interfaces

identified in either the system’s context diagram and/or by counting

the significant interfaces in applicable Interface Control

Documents.


USC


Size Driver Definitions (3 of 4)Number of Operational Scenarios*The number of operational scenarios** that a system is specified tosatisfy. Such threads typically result in end-to-end test scenariosthat are developed to validate the system satisfies its requirements.The number of scenarios can typically be quantified by countingthe number of end-to-end tests used to validate the systemfunctionality and performance. They can also be calculated bycounting the number of high-level use cases developed as part ofthe operational architecture.Number of Modes of Operation (to be merged with Op

Scen)The number of defined modes of operation for a system. For example, in a radar system, the operational modes could be air-to-air, air-to-ground, weather, targeting, etc. The number of modes is quantified by counting the number of operational modes specified in the Operational Requirements Document.

*counting rules need to be refined **Op Scen can be derived from system modes


USC


Size Driver Definitions (4 of 4)Number of Unique AlgorithmsThe number of newly defined or significantly altered functions thatrequire unique mathematical algorithms to be derived in order toachieve the system performance requirements.

Note: Examples could include a complex aircrafttracking algorithm like a Kalman Filter being derived using existingexperience as the basis for the all aspect search function. AnotherExample could be a brand new discrimination algorithm beingderived to identify friend or foe function in space-basedapplications. The number can be quantified by counting the

numberof unique algorithms needed to support each of the mathematicalfunctions specified in the system specification or mode descriptiondocument (for sensor-based systems).


USC


12 Cost Drivers

1. Requirements understanding

2. Architecture complexity

3. Level of service requirements

4. Migration complexity

5. Technology Maturity

Application Factors (5)


USC


Cost Driver Definitions (1,2 of 5)Requirements understanding The level of understanding of the system requirements

by all stakeholders including the systems, software, hardware,

customers, team members, users, etc…

Architecture complexity The relative difficulty of determining and managing the

system architecture in terms of IP platforms, standards,

components (COTS/GOTS/NDI/new), connectors

(protocols), and constraints. This includes systems analysis,

tradeoff analysis, modeling, simulation, case studies, etc…


USC


Cost Driver Definitions (3,4,5 of 5)

Migration complexity (formerly Legacy transition complexity)

The complexity of migrating the system from previous system

components, databases, workflows, etc, due to new technology

introductions, planned upgrades, increased performance, business

process reengineering etc…

Level of service requirementsThe difficulty and criticality of satisfying the Key Performance Parameters (KPP). For example: security, safety, response time, the “illities”, etc…

Technology MaturityThe relative readiness for operational use of the key

technologies.


USC


12 Cost Drivers (cont.)

1. Stakeholder team cohesion

2. Personnel capability

3. Personal experience/continuity

4. Process maturity

5. Multisite coordination

6. Formality of deliverables

7. Tool support

Team Factors (7)


USC


Cost Driver Definitions (1,2,3 of 7)Stakeholder team cohesion Leadership, frequency of meetings, shared vision, approval cycles,

group dynamics (self-directed teams, project engineers/managers),

IPT framework, and effective team dynamics.

Personnel capability Systems Engineering’s ability to perform in their duties and thequality of human capital.

Personnel experience/continuity The applicability and consistency of the staff over the life of the

project with respect to the customer, user, technology, domain,

etc…


USC


Cost Driver Definitions (4,5,6,7 of 7)Process maturity Maturity per EIA/IS 731, SE CMM or CMMI.

Multisite coordination Location of stakeholders, team members, resources (travel).

Formality of deliverables The breadth and depth of documentation required to be formally

delivered.

Tool support Use of tools in the System Engineering environment.


USC


Mapping of Old to New COSYSMO-IP Drivers

Number of System Requirements Number of Major InterfacesNumber of Technical Performance

MeasuresNumber of Operational ScenariosNumber of Modes of OperationNumber of Different PlatformsNumber of Unique Algorithms

Old (7) New (4)

Number of System Requirements Number of Major Interfaces

Number of Operational Scenarios

Number of Unique Algorithms

Siz

e F

acto

rs

Level of Service Requirements

Architecture complexity


USC


Delphi Round 1 HighlightsRange of sensitivity for Size Drivers

# A

lgo

rith

ms

# R

eq

uir

emen

ts

# In

terf

aces

# T

PM

’s

# S

cen

ario

s

# M

od

es

# P

latf

orm

s

5.57

Relative

Effort

1

2.232.54

2.212.10

6.48

6

4

2


USC


Requirements understanding Architecture complexity Level of service requirementsLegacy Transition complexity COTS assessment complexity Platform difficulty Required business process

reengineering Technology Maturity Physical system/information subsystem tradeoff analysis complexity

Requirements understanding Architecture complexity Level of service requirementsMigration complexity

Technology Maturity

Ap

pli

cati

on

Co

st F

acto

rs


# of TPMs # of Platforms

Old (9) New (5)


USC


Delphi Round 1 Highlights (cont.)

Range of sensitivity for Cost Drivers (Application Factors)

EMR

1.93

2.812.13

2.432.24

4

2

Req

uir

em

ents

un

d.

Arc

hit

ectu

re u

nd

.

Lev

el o

f se

rvic

e re

qs.

Leg

acy

tra

nsi

tio

n

CO

TS

Pla

tfo

rm d

iffi

cult

y

Bu

s. p

roce

ss r

een

g.

1.741.13


USC


Number and diversity of stakeholder communities Stakeholder team cohesion Personnel capability Personnel experience/continuity Process maturity Multisite coordination Formality of deliverables Tool support

Old (8) New (7)

Stakeholder team cohesion Personnel capability Personal experience/continuity Process maturity Multisite coordination Formality of deliverables Tool support

Reqs Und


Tea

m C

ost

Fac

tors


USC


Delphi Round 1 Highlights (cont.)Range of sensitivity for Cost Drivers (Team Factors)

1.28

2.461.91 2.161.94

1.25

To

ol

sup

po

rt

Sta

keh

old

er c

om

m.

Sta

keh

old

er c

oh

esio

n

Per

son

nel

cap

ab

ilit

y

Per

son

al

exp

erie

nce

Pro

cess

mat

uri

ty

Mu

ltis

ite

coo

rd.

Fo

rmal

ity

of

del

iv.

1.841.78

EMR4

2


USC








USC


INCOSE Issues and AnswersIssue

Application scope

Life Cycle scope

Too many size drivers

Conflicting cost drivers

Overlap between COSYSMO and CII

AnswerFramework covers all

systems; initial model scope TBD by data

Full 15288 lifecycle

Reduced from 7 to 4

Reduced from 17 to 12

Candidate starting point identified


USC


COSYSMO/COCOMO II MappingPrevious candidate starting point

Development EIA Stage Inception

Elaboration

Construction

Transition Management

14

12

10

14 Environment/CM

10

8

5

5 Requirements

38

18

8

4 Design

19

36

16

4 Implementation

8

13

34

19 Assessment

8

10

24

24 Deployment

3

3

3

30 TBD TBD TBD

= COCOMOII = COSYSMO-IP

When doingCOSYSMO-IP andCOCOMOII, Subtract grey areasprevent doublecounting.

TBD


USC


632 - 20(Implement-

ation)632 - 21

(Transition)632 - 31

(Verification)632 - 32

(Readiness)632 - 33b

(Validation)15288

(Operations)

15288 (Maintenance or Support)

15288 (Retirement)

# System Requirements x x x x x# Major Interfaces x x x x x# Unique Algorithms x x x x# Operational Scenarios x x x x x x# Recursive Levels in the Design x x x x # Systems being Phased Out (Covered by Migration Complexity?) x x x# Operators / Maintainers x x# Training Courses x# Installations

# System Elements

# Length of Lifecycle

Size Drivers vs. EIA/ANSI 632 & ISO/IEC 15288 Stages Late in the Life Cycle

LegendBold = existing driverItalics = proposed addition


USC


Application factors

632 - 20(Implement-

ation)632 - 21




(Validation)15288

(Operations)


15288 (Retirement)

–Requirements understanding x x x x x–Architecture complexity x x x x x x x–Level of service requirements, criticality, difficulty x x x x x–Migration complexity x x–Technology risk (maturity & obsolescence) x x- Operational Complexity x

Cost Drivers vs. EIA/ANSI 632 & ISO/IEC 15288 Stages Late in the Life Cycle



USC


Team factors

632 - 20(Implement-

ation)632 - 21




(Validation)15288

(Operations)


15288 (Retirement)

–Stakeholder team cohesion x x–Personnel capability x x x x x x x x

–Personnel experience/continuity x x x x x x x x–Process maturity x x x x x–Multi-site coordination x x x x x x x x–Formality of deliverables x x x x x x–Tool support x x

Cost Drivers vs. EIA/ANSI 632 & ISO/IEC 15288 Stages Late in the Life Cycle



USC








USC


Parametric Cost Model Critical PathUsual # Months*

6 Converge on cost drivers, WBS

6 Converge on detailed definitions and rating scales

12 Obtain initial exploratory dataset (5-10 projects)

6 Refine model based on data collection & analysis experience

12+ Obtain IOC calibration dataset (30 projects)

9 Refine IOC model and tool

Critical Path Task

*Can be shortened and selectively overlapped


USC


Calendar of Activities: 2003(opportunities to accelerate tasks)

Telecon

2003 2004

INCOSE 2003

USC CSE Annual Research Review

INCOSEFall Workshop COCOMO Forum

Practical Software & Systems Measurement Workshop

Conference on Systems Integration

D J F M A M J J A S O N D

Paper & tutorial

submitted

Practical Software & Systems Measurement Workshop


USC


Action Items from Last Week1. Develop a project Plan2. Address technology maturity/obsolescence3. Refine driver definitions to incorporate

ISO/IEC 15288 definitions4. Incorporate System and People idea5. Refine drivers applicability matrix6. Develop data collection strategy7. Generate Data Collection Form8. Update Stakeholder Cohesion to include

diversity, identification and trust


USC


Outcomes From Last Week’s Workshop

• Reach consensus on resolving the issues

• Converge on scope of COSYSMO-IP model

• Address INCOSE issues• Address definitions of model

parameters• Discuss data collection process• Promote involvement by Affiliates• Define next steps for CSI and INCOSE

conferences


USC


How can the CAB help?• Model Calibration Data

– COSYSMO IOC will be delivered within 9 months of having 30 “clean” data points

• Commitment of resources to assist with model and data definition and collection

• Your support for our proposal to INCOSE SECOE

• Help in obtaining lead participants from other INCOSE Corporate Members

• Establish COSYSMO “owner” within INCOSE– Measurement Working Group willing

• Data, Data, Data


USC


Key Members of the COSYSMO Working Group

Karen Owens, Marilee WheatonEvin StumpGarry Roedler, Gary HafenGary Thomas, John RieffTony Jordano, Don GreenleeChris MillerMarilee WheatonCheryl JonesBarry Boehm, Elliot Axelband,

Don Reifer, Ricardo Valerdi

Aerospace Corp.Galorath

LMCORaytheon

SAICSPCTRW

US Army/PSSMUSC


USC


Points of ContactDr. Barry Boehm[[email protected]]

Dr. Elliot Axelband[[email protected]]

Don Reifer[[email protected]]

Ricardo Valerdi[[email protected]]

Websitehttp://sunset.usc.edu


USC


Backup Charts


USC


USC-CSE Research ($10M backlog)

• DARPA/Army: Model applications and extensions for Future Combat Systems

• DARPA: Architectures for mobile distributed systems (DASADA)

• FAA: Acquisition processes; COCOMO security extensions

• NASA: Empirical methods for High Dependability Computing

• NSF: Center for Empirically-Based Software Engineering (with U. of Maryland)

• NSF: Strategic Design (with CMU, Virginia, Washington)

• Industry Affiliates’ program


USC


General Affiliate Benefits• Affiliates-only Web portal

– Early access to tools, methods, papers, talks, student resumes

• Tools: COCOMO Suite, Architecture tools, WinWin• Technical Report series• Workshops on Affiliate-prioritized topics• Annual Research Review and Steering Group

meeting• Annual one-day professor-visit• Bilateral visit arrangements; internships• Conferences and special workshops• Monthly LA SPIN meetings• Tutorials and eWorkshops


USC


Collaboration Modes and Special Benefits• Software architecting assistance

-Aerospace, Hughes, JPL, Northrop Grumman, TACOM, TRW, Xerox

• Software process/cost/quality/cycle time assistance-Aerospace, Litton, Microsoft, Northrop Grumman, Raytheon, SAIC, Sun, TACOM, TRW, Xerox

• Management reviews of critical projects-Litton, Motorola, SAIC, SEI, TRW

• Reviews of corporate research programs-Daimler Chrysler, Draper Labs, Lockheed Martin, SAIC, SEI, SPC, Telcordia, TRW

• Joint research contracts-Aerospace, Lockheed Martin, Northrop Grumman, SEI, SPC, TRW

• Aid in commercializing USC-CSE research -C-Bridge, Galorath, Group Systems.com, Marotz, Price Systems, Rational


USC


Collaboration Modes and Special Benefits - II

• Special Projects-Aerospace, Auto Club, FAA, Fidelity, IBM, JPL, Litton, Northrop Grumman, Telcordia

• Joint workshops on key topics-Aerospace, Motorola, Rational, DOD/SIS, SEI, SPC

• Focused working groups (COSYSMO) -Aerospace, Galorath, Lockheed Martin, Raytheon, SAIC, SPC, TRW

• Visiting collaborators-Aerospace, Chung-Ang, C-Bridge, IBM, JPL, Litton, Northrop Grumman, SEI, TRW

• Corporate State-of-the-art tutorials-Boeing, Chung-Ang, Daimler Chrysler, Draper, EDS, FAA, Fidelity, IBM, JPL, Litton, Lockheed Martin, Lucent, Motorola, Microsoft, Raytheon, SAIC, SEI, SPC, Sun, TRW, Xerox

Date post:	28-Dec-2015
Category:	Documents
Upload:	christian-pearson
View:	215 times
Download:	2 times