Post on 16-Dec-2015
transcript
© Copyright 2011. Richard W. Selby. All rights reserved.1
Rick Selby
Director of Software ProductsNorthrop Grumman Aerospace Systems310-813-5570, Rick.Selby@NGC.com
Adjunct Professor of Computer ScienceUniversity of Southern California
Lessons Learned for Development and Management of Large-Scale
Software Systems
© Copyright 2011. Richard W. Selby. All rights reserved.2
Embedded software for Advanced robotic
spacecraft platforms High-bandwidth satellite
payloads High-power laser
systems Emphasis on both system
management and payload software
Reusable, reconfigurable software architectures and components
Languages: O-O to C to assembly
CMMI Level 5 for Software in February 2004; ISO/AS9100; Six Sigma
High-reliability, long-life, real-time embedded software systems
Organizational Charter Focuses on Embedded Software Products
Prometheus / JIMO
Software Peer ReviewSoftware Development Lab
Restricted
JWST NPOESS EOS Aqua/Aura Chandra
Airborne Laser
Software Analysis
Software Process Flow for Each Build, with 3-15 Builds per Program
GeoLITE AEHF MTHEL
© Copyright 2011. Richard W. Selby. All rights reserved.3
Lessons Learned for Development and Management of Large-Scale Software Systems Early planning
People are the largest lever Engage your stakeholders and set expectations Embrace change because change is value
Lifecycle and architecture strategy Prioritize features and align resources for high-payoff Develop products incrementally Iteration facilitates efficient learning Reuse drives favorable effort and quality economics
Execution Organize to enable parallel activities Invest resources in high return-on-investment activities Automate testing for early and frequent defect detection Create schedule margin by delivering early
Decision making Measurement enables visibility Modeling and estimation improve decision making Apply risk management to mitigate risks early
© Copyright 2011. Richard W. Selby. All rights reserved.4
Lessons Learned for Development and Management of Large-Scale Software Systems
Early planning
Lifecycle and architecture strategy
Execution
Decision making
© Copyright 2011. Richard W. Selby. All rights reserved.5
People are the Largest Lever
Barry Boehm’s comparisons of actual productivity rates across projects substantiated the COCOMO productivity multiplier factors of 4.18 due to personnel/team capability, etc.
Source: B. Boehm, “Improving Software Productivity,” IEEE Computer, Vol. 20, Issue 9, September 1987, pp. 43-57.
© Copyright 2011. Richard W. Selby. All rights reserved.6
Engage Your Stakeholders and Set Expectations
Source: B. Boehm, “Critical Success Factors for Schedule Estimation and Improvement,” 26 th International Forum on Systems, Software, and COCOMO Cost Modeling, November 2, 2011.
© Copyright 2011. Richard W. Selby. All rights reserved.7
Embrace Change because Change is Value
Criteria for high adoption rate for innovations: Relative advantage – The innovation is technically superior
(in terms of cost, functionality, “image”, etc.) than the technology it supersedes.
Compatibility – The innovation is compatible with existing values, skills, and work practices of potential adopters.
Lack of complexity – The innovation is not relatively difficult to understand and use.
Trialability – The innovation can be experimented with on a trial basis without undue effort and expense; it can be implemented incrementally and still provide a net positive benefit.
Observability – The results and benefits of the innovation’s use can be easily observed and communicated to others.
Source: E. M. Rogers, Diffusion of Innovations, Free Press, New York, 1983.
© Copyright 2011. Richard W. Selby. All rights reserved.8
Lessons Learned for Development and Management of Large-Scale Software Systems
Early planning
Lifecycle and architecture strategy
Execution
Decision making
© Copyright 2011. Richard W. Selby. All rights reserved.9
Prioritize Features and Align Resources for High-Payoff Synchronize-and-stabilize lifecycle has planning,
development, and stabilization phases Planning phase
Vision statement – Product and program management use extensive customer input to identify and prioritize product features
Specification document – Based on vision statement, program management and development group define feature functionality, architectural issues, and component interdependencies
Schedule and feature team formation – Based on specification document, program management coordinates schedule and arranges feature teams that each contain approximately 1 program manager, 3-8 developers, and 3-8 testers (who work in parallel 1:1 with developers)
Development phase Program managers coordinate evolution of specification. Developers design, code, and debug. Testers
pair up with developers for continuous testing. Subproject I – First 1/3 of features: Most critical features and shared components. Subproject II – Second 1/3 of features. Subproject III – Final 1/3 of features: Least critical features.
Stabilization phase Program managers coordinate OEMs and ISVs and monitor customer feedback. Developers perform
final debugging and code stabilization. Testers recreate and isolate errors. Internal testing – Thorough testing of complete product within the company. External testing – Thorough testing of complete product outside the company by “beta” sites such as
OEMs, ISVs, and end-users. Release preparation – Prepare final release of “golden master” version and documentation for
manufacturing.
© Copyright 2011. Richard W. Selby. All rights reserved.10
Develop Products Incrementally
Documents and Intermediate Activities
Milestone I release
Milestone II release
Project plan approval
Milestone 0
Schedule complete
Milestone III release
Code complete
Zero bug release
Release to manufacturing
Visual freeze
Pla
nn
ing
3-1
2 m
on
ths
Sta
bil
iza
tion
3-8
month
s
Feature complete
Vision statement
Specification document
Prototypes
Testing strategySchedule
Project review
Implementation plan
Specification review
Postmortem document
Internal testing
Beta testingBuffer time
Buffer time
Major Reviews
MilestonesTimeline
Su
bp
roje
ct
IIISu
bp
roje
ct
II Su
bp
roje
ct
I
Design feasibility studies
OptimizationsTesting and debugging
OptimizationsTesting and debugging
(Ship date)
6-1
2 m
on
ths
Develo
pm
en
t• Integration • Testing and debugging
2-5 weeks
• Buffer time2-5 weeks
6-10 weeks • Code and optimizations • Testing and debugging • Feature stabilization
Development Subproject 2-4 months
(1/3 of all features)
Phases
Dev
elop
men
t
6-16
mon
ths
Synchronize-and-stabilize lifecycle timeline and milestones enable frequent incremental deliveries
© Copyright 2011. Richard W. Selby. All rights reserved.11
Iteration Facilitates Efficient Learning
Figure 4.3-4. JIMO Incremental Software BuildsWe provide incremental software deliveries support integration and test activities and synchronize with JPL, Hamilton Sundstrand, and Naval Reactors to facilitate teaming, reduce risk, and enhance mission assurance.
Delivered to, Usage
201320122011201020092008200720062005CY 2004
PMSR1/05
Data Server Unit (DSU) Builds
Science Computer Unit (SCU) Builds
Flight Computer Unit (FCU) Builds
Note: Science Computer builds for common software only (no instrument software included)
FCU1
ATP11/04
SM PDR6/08
SM CDR8/10
BUS I&T8/12
SM AI&T8/13
Prelim Exec and C&DH Software
Prelim Exec and C&DH Software
FCU2
FCU3
FCU4
FCU5
FCU6
FCU7
Final Exec and C&DH Software
Science Computer Interface
Reactor Controller Interface
AACS (includes autonomous navigation)
Thermal and Power Control
Configuration and Fault Protection
SCU1
Final Exec and C&DH SoftwareSCU2
DSU1
DSU3
Prelim Exec and C&DH Software
DSU2 Final Exec and C&DH Software
Data Server Unique Software
Ground Analysis Software (GAS) Computer Builds
GAS1Preliminary Ground Analysis Software
GAS2Final Ground Analysis Software
A B C D
P
P
P
P
P
P
P
P
P
04S01176-4-108f_154
JPL/NGC, Prelim. Hardware/Software IntegrationJPL/NGC, Final Hardware/Software IntegrationJPL, Mission Module Integration
JPL, Prelim. Hardware/Software IntegrationJPL, Final Hardware/Software Integration
NR, Reactor Controller IntegrationNGC, AACS Validation on SMTBNGC, TCS/EPS Validation on SSTB
NGC, Fault Protection S/WValidation on SSTB
NGC, Prelim. Hardware/Software Integration
NGC, Final Hardware/Software IntegrationNGC, HCR Integration on SMTBJPL, Prelim. Integration into Ground System
JPL, Final Integration into Ground System
1 Requirements2 Preliminary Design3 Detailed Design4 Code and Unit Test/Software
Integration5 Verification and Validation
Legend: N is defined as follows:541 32=
2 53 41=
542 31=
PrototypeActivity
NGC
N Performer of Activity NJPL
Role/activity shared by JPL and NGC
Design Agent
P
Figure 4.3-4. JIMO Incremental Software BuildsWe provide incremental software deliveries support integration and test activities and synchronize with JPL, Hamilton Sundstrand, and Naval Reactors to facilitate teaming, reduce risk, and enhance mission assurance.
Delivered to, Usage
201320122011201020092008200720062005CY 2004
PMSR1/05
Data Server Unit (DSU) Builds
Science Computer Unit (SCU) Builds
Flight Computer Unit (FCU) Builds
Note: Science Computer builds for common software only (no instrument software included)
FCU1
ATP11/04
SM PDR6/08
SM CDR8/10
BUS I&T8/12
SM AI&T8/13
Prelim Exec and C&DH Software
Prelim Exec and C&DH Software
FCU2
FCU3
FCU4
FCU5
FCU6
FCU7
Final Exec and C&DH Software
Science Computer Interface
Reactor Controller Interface
AACS (includes autonomous navigation)
Thermal and Power Control
Configuration and Fault Protection
SCU1
Final Exec and C&DH SoftwareSCU2
DSU1
DSU3
Prelim Exec and C&DH Software
DSU2 Final Exec and C&DH Software
Data Server Unique Software
Ground Analysis Software (GAS) Computer Builds
GAS1Preliminary Ground Analysis Software
GAS2Final Ground Analysis Software
A B C D
P
P
P
P
P
P
P
P
P
04S01176-4-108f_154
JPL/NGC, Prelim. Hardware/Software IntegrationJPL/NGC, Final Hardware/Software IntegrationJPL, Mission Module Integration
JPL, Prelim. Hardware/Software IntegrationJPL, Final Hardware/Software Integration
NR, Reactor Controller IntegrationNGC, AACS Validation on SMTBNGC, TCS/EPS Validation on SSTB
NGC, Fault Protection S/WValidation on SSTB
NGC, Prelim. Hardware/Software Integration
NGC, Final Hardware/Software IntegrationNGC, HCR Integration on SMTBJPL, Prelim. Integration into Ground System
JPL, Final Integration into Ground System
1 Requirements2 Preliminary Design3 Detailed Design4 Code and Unit Test/Software
Integration5 Verification and Validation
Legend: N is defined as follows:541 32=
2 53 41=
542 31=
PrototypeActivity
NGC
N Performer of Activity NJPL
Role/activity shared by JPL and NGC
Design Agent
P
We provide incremental software deliveries support integration and test activities and synchronize with JPL, Hamilton Sundstrand, and Naval Reactors to facilitate teaming, reduce risk, and enhance mission assurance.
Delivered to, Usage
201320122011201020092008200720062005CY 2004
PMSR1/05
Data Server Unit (DSU) Builds
Science Computer Unit (SCU) Builds
Flight Computer Unit (FCU) Builds
Note: Science Computer builds for common software only (no instrument software included)
FCU1
ATP11/04
SM PDR6/08
SM CDR8/10
BUS I&T8/12
SM AI&T8/13
Prelim Exec and C&DH Software
Prelim Exec and C&DH Software
FCU2
FCU3
FCU4
FCU5
FCU6
FCU7
Final Exec and C&DH Software
Science Computer Interface
Reactor Controller Interface
AACS (includes autonomous navigation)
Thermal and Power Control
Configuration and Fault Protection
SCU1
Final Exec and C&DH SoftwareSCU2
DSU1
DSU3
Prelim Exec and C&DH Software
DSU2 Final Exec and C&DH Software
Data Server Unique Software
Ground Analysis Software (GAS) Computer Builds
GAS1Preliminary Ground Analysis Software
GAS2Final Ground Analysis Software
A B C D
P
P
P
P
P
P
P
P
P
04S01176-4-108f_154
JPL/NGC, Prelim. Hardware/Software IntegrationJPL/NGC, Final Hardware/Software IntegrationJPL, Mission Module Integration
JPL, Prelim. Hardware/Software IntegrationJPL, Final Hardware/Software Integration
NR, Reactor Controller IntegrationNGC, AACS Validation on SMTBNGC, TCS/EPS Validation on SSTB
NGC, Fault Protection S/WValidation on SSTB
NGC, Prelim. Hardware/Software Integration
NGC, Final Hardware/Software IntegrationNGC, HCR Integration on SMTBJPL, Prelim. Integration into Ground System
JPL, Final Integration into Ground System
1 Requirements2 Preliminary Design3 Detailed Design4 Code and Unit Test/Software
Integration5 Verification and Validation
Legend: N is defined as follows:541 32=
2 53 41=
542 31=
PrototypeActivity
NGC
N Performer of Activity NJPL
Role/activity shared by JPL and NGC
Design Agent
P
Delivered to, Usage
201320122011201020092008200720062005CY 2004
PMSR1/05
Data Server Unit (DSU) Builds
Science Computer Unit (SCU) Builds
Flight Computer Unit (FCU) Builds
Note: Science Computer builds for common software only (no instrument software included)
FCU1
ATP11/04
SM PDR6/08
SM CDR8/10
BUS I&T8/12
SM AI&T8/13
Prelim Exec and C&DH Software
Prelim Exec and C&DH Software
FCU2
FCU3
FCU4
FCU5
FCU6
FCU7
Final Exec and C&DH Software
Science Computer Interface
Reactor Controller Interface
AACS (includes autonomous navigation)
Thermal and Power Control
Configuration and Fault Protection
SCU1
Final Exec and C&DH SoftwareSCU2 Final Exec and C&DH SoftwareSCU2
DSU1
DSU3
Prelim Exec and C&DH Software
DSU2 Final Exec and C&DH SoftwareDSU2 Final Exec and C&DH Software
Data Server Unique Software
Ground Analysis Software (GAS) Computer Builds
GAS1Preliminary Ground Analysis Software
GAS2Final Ground Analysis Software
A B C D
P
P
P
P
P
P
P
P
P
04S01176-4-108f_154
JPL/NGC, Prelim. Hardware/Software IntegrationJPL/NGC, Final Hardware/Software IntegrationJPL, Mission Module Integration
JPL, Prelim. Hardware/Software IntegrationJPL, Final Hardware/Software Integration
NR, Reactor Controller IntegrationNGC, AACS Validation on SMTBNGC, TCS/EPS Validation on SSTB
NGC, Fault Protection S/WValidation on SSTB
NGC, Prelim. Hardware/Software Integration
NGC, Final Hardware/Software IntegrationNGC, HCR Integration on SMTBJPL, Prelim. Integration into Ground System
JPL, Final Integration into Ground System
1 Requirements2 Preliminary Design3 Detailed Design4 Code and Unit Test/Software
Integration5 Verification and Validation
Legend: N is defined as follows:541 32=
2 53 41=
542 31=
PrototypeActivity
NGC
N Performer of Activity NJPL
Role/activity shared by JPL and NGC
Design Agent
P
Power Power
Incremental software builds deliver early capabilities and accelerate integration and test
Iteration helps refine problem statements, create potential solutions, and elicit feedback
© Copyright 2011. Richard W. Selby. All rights reserved.12
Reuse Drives Favorable Effort and Quality Economics
0.00
0.25
0.50
0.75
1.00
1.25
1.50
Module origin
Fau
lts
per
mo
du
le (
ave.
)
Mean 1.28 1.18 0.58 0.02 0.85
Std. dev. 2.88 1.81 1.20 0.17 2.29
New development
Major revision Slight revision Complete reuse All
Analyses of component-based software reuse shows favorable trends for decreasing faults
Data from 25 NASA systems Overall difference is statistically significant ( < .0001). Number of
components (or modules) in each category is: 1629, 205, 300, 820, and 2954, respectively.
© Copyright 2011. Richard W. Selby. All rights reserved.13
Lessons Learned for Development and Management of Large-Scale Software Systems
Early planning
Lifecycle and architecture strategy
Execution
Decision making
© Copyright 2011. Richard W. Selby. All rights reserved.
Organize to Enable Parallel Activities
14 16
Legend
Auto-matable activity
Source: “Flowchart,” www.wikipedia.org, May 2010.
=
Track cycletime between activities
© Copyright 2011. Richard W. Selby. All rights reserved.15
Total Ave. Ave. / EKSLOC
Reviews 257 29 N/A
Prevention cycles
15 1.7 N/A
Defects 2621 291 7.3
Defects per review
N/A 15 N/A
Defects out-of-phase
N/A 8.1% 1.3
Invest Resources in High Return-on-Investment Activities
Return-on-investment (ROI) for software peer reviews ranges from 9:1 to 3800:1 per project
Return-on-investment (ROI) = Net cost avoidance divided by non-recurring cost 2621 defects, 257 reviews, 9 systems, 1.5 years High ROI drivers
Mature and effective processes already in place Significant new scope under development Early lifecycle peer reviews (e.g., requirements phase) Four of the five programs with >80% requirements and design defects had
relatively higher ROI
9
3326
37823867
292143
16
3638
1877
0
100
200
300
400
500
600
700
800
# D
efec
ts
0
500
1000
1500
2000
2500
3000
3500
4000
4500
RO
I
# Defects ROI
Projects
© Copyright 2011. Richard W. Selby. All rights reserved.16
Software Defect Injection Phase
0.0%2.3%
49.1%
9.0%12.1% 11.7%
2.5% 1.9%
10.6%
0.7% 0.1% 0.0%0.0%
5.0%
10.0%
15.0%
20.0%
25.0%
30.0%
35.0%
40.0%
45.0%
50.0%
System Development Phase
De
fec
ts (
%)
Automate Testing for Early and Frequent Defect Detection
Distribution of software defect injection phases based on using peer reviews across 12 system development phases
3418 defects, 731 peer reviews, 14 systems, 2.67 years 49% of defects injected during requirements phase
© Copyright 2011. Richard W. Selby. All rights reserved.17
Create Schedule Margin by Delivering Early
Source: “The Network Diagram and Critical Path,” www.slideshare.net, May 2010.
LegendCritical path
Critical path defines the path through the network containing activities with zero slack
© Copyright 2011. Richard W. Selby. All rights reserved.18
Lessons Learned for Development and Management of Large-Scale Software Systems
Early planning
Lifecycle and architecture strategy
Execution
Decision making
© Copyright 2011. Richard W. Selby. All rights reserved.19
Measurement Enables Visibility
Requirements
0
5
10
15
20
25
30
Jun-04 Jul-04 Aug-04 Sep-04
Plan Actual
LCL UCL
Reuse
0%10%20%30%40%50%60%70%
Proposal SSR PDR CDR
Plan Actual
LCL UCL
Technology Infusion
01234567
Jun-04 Jul-04 Aug-04 Sep-04
Plan Actual
LCL UCL
Progress
0
50
100
150
200
250
Jun-04 Jul-04 Aug-04 Sep-04
Plan Actual
LCL UCL
Cycletime
0
5
10
15
20
25
Jun-04 Jul-04 Aug-04 Sep-04
Plan Actual
LCL UCL
Deliveries
0
5
10
15
20
25
Jun-04 Jul-04 Aug-04 Sep-04
Plan Actual
LCL UCL
Pre-Delivery Defects
0
50
100
150
200
250
Jun-04 Jul-04 Aug-04 Sep-04
Plan Actual
LCL UCL
Post-Delivery Defects
02468
101214
Jun-04 Jul-04 Aug-04 Sep-04
Plan Actual
LCL UCL
Contact Contact
ContactContactContact
ContactData Data
DataDataData
DataOutliers Outliers
OutliersOutliersOutliers
Outliers
Help Help
HelpHelpHelpContactDataOutliers
Help
HelpContactDataOutliers
Help
Up Down Level 1 AllUp Down Level 1 All
Up Down Level 1 All
Up Down Level 1 All
Up Down Level 1 All Up Down Level 1 All Up Down Level 1 All
Up Down Level 1 All
DASHBOARD Metrics: Organization: Project: Manager: Contact: Status: Development ABC Products Division XYZ System FirstName LastName Name@ABC.com x12345 10/1/2004
Interactive metric dashboards provide framework for visibility, flexibility, integration, and automation
Interactive metric dashboards incorporate a variety of information and features to help developers and managers characterize progress, identify outliers, compare alternatives, evaluate risks, and predict outcomes
© Copyright 2011. Richard W. Selby. All rights reserved.20
Modeling and Estimation Improve Decision Making
0
10
20
30
40
50
60
70
80
90
100
0 10 20 30 40 50 60 70 80 90 100
Completeness ( = 100% less % of False Negatives)
Co
nsi
sten
cy (
= 10
0% le
ss %
of F
alse
P
osi
tives
)
Overall average without optimizations Predicting all to be "+"
Optimizations for Consistency Predicting none to be "+"
Optimizations for Completeness
Target: Identify error-prone (top 25%) and effort-prone (top 25%) components
16 large NASA systems
960 configurations Models use metric-
driven decision trees and networks
Analyses tradeoff prediction consistency versus completeness
© Copyright 2011. Richard W. Selby. All rights reserved.21
WBS: 4.0 Spacecraft IPTRisk Owner: McWhorter, Larry Printed: 14 Dec 2005Risk: CEV-252 - Flight Software Requirements Management
1 : Conduct BM1 2 : Estimate softw are requirements scope (preliminary)3 : Establish softw are control board (preliminary) 4 : Release SDP (w ith spec tree, change process)5 : Complete RTOS lab evaluation; Validate capabilities using sim 6 : Estimate softw are requirements scope (f inal)7 : Implement system development process f low models 8 : Spacecraft/etc users define/val/sim use-cases (I/F, funct, off-nom, ...)9 : Finalize/val/sim IFC1 requirements: Infrastructure SW 10 : Baseline allocation of SW requirements to IFCs w ith grow th/correction11 : Establish softw are control board (f inal) 12 : Conduct Sw RR13 : Finalize/val/sim IFC2 requirements: Inter-module & inter-subsystem I/F 14 : Initial end-to-end architecture model15 : Finalize/val/sim IFC3 requirements: Subsystems major functions 16 : Finalize/val/sim IFC4 requirements: Nominal operations17 : Deliver IFC3: Subsystems major functions 18 : Finalize/val/sim IFC5 requirements: Subsystems off-nominal operations19 : Finalize/val/sim IFC6 requirements: System off-nominal operations 20 : Deliver IFC7: No new capabilities; Only corrections; 1st mission ready
Ris
k S
core
26
25
2423
22
21
20
1918
17
16
15
1413
12
11
10
9
87
6
5
4
32
1
0
1
2
3
4 5 6
7
8
9 10
11
12
13
14
15 16
17
18 19 20
2005 2006 2007 2008 2009 2010
SDR
Moderate
High
Low
Apply Risk Management to Mitigate Risks Early
Last Updated: 03-October-05
CSRR PDR
Last Updated: 14-Dec-05
Events
CDR
TRL4
TRL5 TRL
6
TRL7
WBS: 4.0 Spacecraft IPTRisk Owner: Wood, Doug Printed: 15 Nov 2005Risk: CEV-252 - Flight Software Requirements Management
Planned Risk Level Planned (Solid=Linked, Hollow =Unlinked) Control PointsActual Risk Level Completed Completed
Ris
k S
core
26252423222120191817161514131211109876543210
Spacecraft/etc users define/val/sim use-cases (I/F, funct, off-nom, ...)
Conduct Sw RR
Finalize/val/sim IFC4 requirements: Nominal operations
Deliver IFC3: Subsystems major functions
2005 2006 2007 2008 2009 2010
Exit/Success Criteria:1. BM1 complete; customer concurs with approach2. Software requirements scope estimated (preliminary).3. Software control board established (preliminary); change control process established.4. SDP released. Spec tree defined.5. RTOS lab evaluation completed. Capabilities validated using sim.6. Software requirements scope estimated (final)7. System development process flow models implemented.8. Spacecraft/subsystems/etc. users define use cases (for I/Fs, functions, nominal ops, off-nominal ops, etc.)
completed. Validated using models/sim.
9. Finalize IFC1 requirements: Infrastructure SW completed. Validated requirements using models/sim.10. Baseline allocation of SW requirements to IFCs with growth/correction/deficiency completed.11. Software control board (final) established12. SwRR conducted. NASA customer agrees with software requirements.13. Finalize IFC2 requirements: Inter-module & inter-subsystem I/Fs completed. Validated requirements using
models/sim.14. Initial end-to-end architecture model completed.15. Finalize IFC3 requirements: Subsystems major functions completed. Validated requirements using models/sim.16. Finalize IFC4 requirements: Nominal operations completed. Validated requirements using models/sim.17. Deliver IFC3: Subsystems major functions completed. Validated capabilities using sim.18. Finalize IFC5 requirements: Subsystems off-nominal operations completed. Validated requirements using
models/sim.19. Finalize IFC5 requirements: Subsystems off-nominal operations completed. Validated requirements using
models/sim.20. Deliver IFC7: No new capabilities; Only system I&T corrections completed. SW complete for 1st mission.
Projects define risk mitigation “burn down” charts with specific tasks and exit criteria
© Copyright 2011. Richard W. Selby. All rights reserved.22
Lessons Learned for Development and Management of Large-Scale Software Systems Early planning
People are the largest lever Engage your stakeholders and set expectations Embrace change because change is value
Lifecycle and architecture strategy Prioritize features and align resources for high-payoff Develop products incrementally Iteration facilitates efficient learning Reuse drives favorable effort and quality economics
Execution Organize to enable parallel activities Invest resources in high return-on-investment activities Automate testing for early and frequent defect detection Create schedule margin by delivering early
Decision making Measurement enables visibility Modeling and estimation improve decision making Apply risk management to mitigate risks early