Date post: | 27-Mar-2015 |
Category: |
Documents |
Upload: | luis-lewis |
View: | 215 times |
Download: | 2 times |
Systems Engineering = Best Practices
Nicholas M. TorelliDeputy Director, Human Capital and Specialty Engineering
Systems and Software Engineering (SSE)Office of the Deputy Under Secretary of Defense
for Acquisition and Technology
Presented to2009 Systems Engineering Summit:
Harnessing Best Practices
Sponsored by the Huntsville Regional Chapter of INCOSE,
US Army, RDECOM, AMRDEC, DAU, and NASA
March 3, 2009
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 2
DoD Vision for Systems Engineering
Systems engineering principles and disciplines are fully accepted and assimilated into the DoD acquisition workforce positioning the DoD for acquisition excellence leading to a stronger national defense.
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 3
SSE’s 2009 Priorities
• Acquisition work force development• Renewed focus on early application of systems
engineering to affect affordability and total ownership cost• Increased level of visibility for our role in developmental
test and evaluation• Systems of Systems engineering tools and techniques• Integration of program protection activities into acquisition
oversight to address cyber threat• Measuring results of our efforts
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 4
DoD SE Best Practice Continuum
Policy &Guidance
Teamwork &Collaboration
Developingthe Workforce& AdvancingSE Practice
Program Support& Assessment
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 5
CBAJoint Concepts
MS CMS B
Strategic Guidance
MS A
ICD TechnologyDevelopment
Engineering and Manufacturing
DevelopmentO&S
MDD
Materiel Solution Analysis
CDRPDR
Full Rate ProductionDecision Review
* DoDI 5000.02, 8 December 2008
JCIDS Process
Production and Deployment
CPDCDD
or
Overview of Acquisition Policy Changes*
• Mandatory Materiel Development Decision (MDD)• Mandatory competing prototypes before MS B• Mandatory PDR and a report to the MDA (“the sliding PDR”)
[PDR Report to MDA if before MS B; formal PDR Assessment by MDA if after MS B]
• Configuration Steering Boards at Component level to review all requirements changes
• Renewed emphasis on manufacturing during system development: – Re-titles SDD phase to EMD with two sub phases: Integrated System Design and
System Capability and Manufacturing Process Demonstration– Establishes consideration of manufacturing maturity at key decision points
• Mandatory system-level CDR with an initial product baseline and followed by a Post-CDR Report to the MDA
• Post-CDR Assessment by the MDA between EMD sub phases
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 6
Acquisition Policy Opportunities for SE
• Early SE engagement with programs• Program Support Reviews (PSRs) Pre-MS A/B/C• Risk Reduction activities (e.g., Technical Risk
assessment in AoAs, Competitive Prototyping)• SE Technical Reviews - Informed Trades for Feasible
Solutions • Developmental Test and Evaluation
– Integrated DT/OT– Updated T&E Strategy at MS A
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 7
New Systems Engineering Enclosure
• Codifies several previous SE policy memoranda• Codifies a number of SE-related policies and statutes
since 2003:– Environmental Safety and Occupational Health– Modular Open Systems Approach– Data Management and Technical Data Rights– Item Unique Identification– Spectrum Supportability– Corrosion Prevention and Control
• Introduces new policy on Configuration Management
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 8
DoD Guidance
• Defense Acquisition• Systems Engineering• Developmental Test and Evaluation (DTE) • Modeling and Simulation (M&S) • Safety• System Assurance
http://www.acq.osd.mil/sse/pg/guidance.html
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 9
DoD SE Best Practice Continuum
Policy &Guidance
Teamwork &Collaboration
Developingthe Workforce& AdvancingSE Practice
Program Support& Assessment
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 10
OSD Systems Engineering Strategy
• Achievable Acquisition Strategy• Enhanced Gate Review Process• Enhanced Staff Capabilities
• Milestones A/B Critical• Correct SE Staffing• Component Development Planning• Pre-MS A analysis
NR
C S
tud
y
Re
co
mm
en
da
tion
sS
ys
tem
ic A
na
lys
is
Re
co
mm
en
da
tion
s*
*Based on 3,700 Program Assessment findings from 40 Programs Support Reviews
ReliabilityWorkforce
Development
DT&E Revitalization
• Integrated DT/OT• Reliability Improvement• Early SE• Access to Relevant Data for
Evaluation & Decision Making
DS
B D
T&
E R
ep
ort
& 2
31
Re
po
rt R
ec
om
me
nd
atio
ns
OS
D P
olic
y &
Gu
ida
nc
eEnhanced SE
Pre-MS B
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 11
The Defense Acquisition Community126,033 Government and Military Certified Professionals
500,000+ Defense Industry PersonnelSSE Functional Leader for SE, T&E, and PQM workforce
Workforce Development
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 12
SE Human Capital Strategy
• FY08 NDAA Section 852: DoD Acquisition Workforce Development Fund: >$300M per year across DoD– SE, PQM and T&E initiatives to Recruit, Train, and Retain the
workforce– DoD Human Capital Initiative - Published Annex for SPRDE,
PQM, and T&E Career Fields
• SE “core competency” assessment effort; completion - Summer 2009
• “Program Systems Engineer” career path• Partnership with INCOSE SE Certification Program
– “CSEP-Acq” aligned with Defense Acquisition Guidebook• Equivalency granted for DAU Courses SYS101 and SYS202
– Expanding potential for Industry “certifications” • what the future could hold
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 13
Human Capital Initiatives (Defense Acquisition Workforce Development
Fund 1)
1 Based on NDAA Section 852, Defense Acquisition Workforce Development Act
Defense Acquisition Workforce
Development Fund
Recruiting & Hiring
Training & Development
Retention & Recognition
Intern Programs
Recruiting Incentives
Outreach Programs
Journeyman Hiring
Highly Qualified Experts
Training Enhancement & Capacity Expansion
Acquisition WorkforceManagement & Student
Information System
Competency Management &
Assessments
Workforce Planning
Retention & Recognition
Career Broadening & Academic Programs
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 14
Notional DoD Systems Engineering Population
Wo
rkfo
rce
Siz
e
25-35 35-45 45-55 >55Workforce Age
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 15
Notional DoD Systems Engineering Workforce Strategy
Wo
rkfo
rce
Siz
e
25-35 35-45 45-55 >55Workforce Age
Train:Mentor
Retain
Recruit:Journeymen
Recruit: Highly
Qualified Experts
Rec
ruit:
In
tern
sRetain:
Rotational Training AssignmentsEducation with Industry
Qualification CardsIncentives
Hazardous Duty Pay for PSEs??
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 16
NDIA Systems Engineering Division E&T Committee
Mission: Strengthen Systems
Engineering capabilities through education, training and experience across government,
industry, and academia.
2009 Task:To address the issue of
“self-proclaimed” Systems Engineers, the
E&T Committee is attempting to determine the core essential set of
competencies that distinguish systems
engineers from domain engineers.
Contact Govt Committee Chair:Dr. Don Gelosh, CSEP-Acq ODUSD (A&T) SSE / HC [email protected]
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 17
Co
mp
. L
evel
SE Competency Model
Domain
Knowledge
Systems ThinkingUnit of
CompetenceCompetence Element(s)
Essence of
Systems
Problem Solving
Unit of
CompetenceCompetence Element(s)
Stakeholder
Requirements Analysis
Requirements Analysis
Unit of
CompetenceCompetence Element(s)
Decision Management
Risk Management
SE Technical
SE Technical Mgmt
Systems Engineer
Composite Competency Map
Architecture Design
Implementation
Lvl
Lvl
Lvl
Configuration
Management
Analytical
Analytical
Analytical
Analytical
Technical
Management
Technical
Management
Technical
Management
Technical
Management
Technical Planning
Behavior
Structure
Goal
Problem
Solving
Method
Analytical
Analytical
Time
Minimum
Essential
SE Criteria
Growth Professional / InterpersonalUnit of
CompetenceCompetence Element(s) Lvl
Life Cycle ViewUnit of
CompetenceCompetence Element(s) Lvl
Material
Solution Analysis
Technology
Development
EMD
Production &
Deployment
Operations &
Support
Factors that Affect
the SE Competency Profile
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 18
Research to Practice
SERC Governance & Collaborators
• Co-Sponsors• Government PMO• Tasking Activities• Industry• Associations• Academia
SERC Governance & Collaborators
• Co-Sponsors• Government PMO• Tasking Activities• Industry• Associations• Academia
Ongoing Developmen
t Acquisition Community (DoD and Industry)
SERC
Provide Lessons Learned and Challenges
Deliver New/Improved SE Methods,
Processes & Tools (MPTs)
PolicyGuidanceEducation & Training
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 19
Need for SE Research
• State-of-the-practice must keep up with application needs of DoD acquisition– Methods, processes, and tools to enable effective acquisition and
sustainment of systems– Leveraging Modeling & Simulation for SE
• Size and complexity of modern systems drive need for commensurate extensions of SE, – To System of Systems (SoS), to complex systems, to
“architecting,” to net-centric sets of services– Validation & Verification challenges
• SE “theory and practice” should be inclusive of, and establish linkages to, challenged sub-specialty areas– Software engineering, reliability engineering, system safety,
costing, etc.
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 20
Stevens-SERC Team
• Auburn University• Air Force Institute of Technology• Carnegie Mellon University• Fraunhoffer Center at UMD• Massachusetts Institute of
Technology• Missouri University of Science and
Technology (S&T)• Pennsylvania State University• Southern Methodist University
Stevens Institute of TechnologyLead University
• Texas A&M University • Texas Tech University• University of Alabama in Huntsville • University of California at San Diego • University of Maryland• University of Massachusetts• University of Southern California• University of Virginia • Wayne State University
University Partners
The DoD Systems Engineering Research Center will be responsible for systems engineering research that supports the development, integration, testing and sustainability of complex defense systems, enterprises and services. Stevens has MOUs to develop enhanced SE courseware for competency development within DoD with AFIT, NPS and DAU. Further, SERC members are located in 11 states, near many DoD facilities and all DAU campuses.
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 21
Initial SERC Research Tasks
• “Assessing Systems Engineering Effectiveness in Major Defense Acquisition Programs (MDAPs)”– USC task lead, with support from Stevens, Fraunhofer Center, MIT,
University of Alabama at Huntsville– Characterize SE effectiveness within the context of DoD acquisition and
identify methods of measurement suitable for application to project execution organizations (e.g., defense contractors), program management organizations (e.g., project managers and program executive offices) and oversight organizations (e.g., OUSD(AT&L)) (OSD)
• “Evaluation of Systems Engineering Methods, Processes, and Tools (MPT) on Department of Defense and Intelligence Community Programs”– Stevens task lead, with support from USC, University of Alabama at
Huntsville, and Fraunhofer– Examine and recommend areas for advancing current SE methods,
processes, and tools (MPTs) as they are applied across the DoD acquisition life cycle focusing on three different development environments: individual weapons systems, SoS, and network-centric systems (NSA)
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 22
Ways Your Organizations Can Become Involved…
• Identify SE research challenges• Provide funding to sponsor research as a Tasking Activity• Collaborate on DoD-sponsored research by identifying
pilot programs to participate, provide relevant acquisition information, identify subject matter experts
• Make use of research findings to improve systems engineering on acquisition programs
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 23
DoD SE Best Practice Continuum
Policy &Guidance
Teamwork &Collaboration
Developingthe Workforce& AdvancingSE Practice
Program Support& Assessment
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 24
Our World of Stakeholders
• DoD– Components and Agencies– Product Centers– Program Executive Offices– Acquisition Programs (PMs
and CSEs)– Practicing PMs and SEs– Other DAWIA Career Fields
• Industry– First tier contractors (e.g.,
Lockheed Martin, Boeing, Northrop-Grumman, Raytheon)
– Second tier contractors– Practicing SEs– SE tool vendors
• Education & Training Institutions– DoD organizations: DAU,
AFIT, NPS– US universities with SE
programs (~60)– SE short course providers
• Professional and Industrial Associations– DAU Alumni Association– INCOSE– NDIA SE Division, other assoc
Div– IEEE Standards Committee;
Systems Council– AIAA SE Committee– TechAmerica– ISO
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 25
DoD Challenge: “A”cquisition Process
Requirements Budgeting
Acquisition
• ICD, CDD, CPD• Future Threat and Op
Tempo
• Planning, Programming, and Budgeting Process
• FYDP• POM Process• Appropriation and
Authorization of Funds
• Acquisition Policy, Guidance, and Oversight.
• MDAP Decision Authority
“Sweet Spot”
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 26
Requirements Budgeting
Acquisition
“Sweet Spot”
DoD Goal: Increase the Overlap
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 27
Systems & Software EngineeringOrganization Overview
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 28
Congress
Sec Def
RequirementsDevelopers
Service AcquisitionExecutives
PEOs
Program Offices
Prime ContractorsSecond Tier Contractors
Education & Collaboration InfrastructureProfessional / Industry Associations
(NDIA, INCOSE, AIA, ITEA, TechAmerica, etc.) DAU, Academic Institutions, SERC
Opportunities for SSE Engagement
SSEPolicy & Guidance
• Systems Engineering
• DT&E
Program Support
• Program Support Reviews
• OIPT, T&E and SE WIPTs
• AOTR, Post-CDR Review & Assessment
Workforce Planning
• Competency Models
• Certification Reqts
• Education & Training
Emerging Concepts
• Systems of Systems
• SE Research
Outreach
• SE Forum
• Engagement Strategy
Statutory Direction
ICD, CDD, CPD
DAB, ITAR, DSAB, OIPT
PSR, SEP, TEMP,Technical Reviews
CSEP-Acq, Research, Industry-University SE Workforce Initiatives,
Industry-Gov’t Projects
AT&L Direction
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 29
DoD Human Systems Integration (HSI)
Problem• HSI is not being consistently
integrated within DoD acquisition resulting in increased ownership costs and lower system effectiveness.
Approach• Assign DUSD(A&T) Executive
Authority to define policy, as necessary, to implement changes.
• Maximize the good work that has been done (policies, plans, methods, tools and standards).
• Engage DoD stakeholders and strengthen industry affiliations including NDIA, TechAmerica and INCOSE.
• Identify gaps and apply necessary resources to resolve gaps or barriers to successful HSI implementation.
Cross-DoD and Technical Community Collaboration
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 30
Collaboration with INCOSE
• International Panel: “What Defines a Systems Engineer? Comparing and Contrasting Global Perspectives on Systems Engineering Competency”
– Moderator: Dr. Don Gelosh, Human Capital Strategy and Planning, OSD Directorate of Systems and Software Engineering
– Dr. Arthur Pyster, Distinguished Research Professor, School of Systems and Enterprises, Stevens Institute of Technology; Deputy Executive Director, DoD Systems Engineering Research Center; and member of the INCOSE Board of Directors
– Dr. John Snoderly, Program Director, Systems Engineering, Defense Acquisition University
– Samantha Brown, President-Elect of INCOSE and Systems Engineering Innovation Centre (SEIC), Loughborough, UK
– Mark Kupeski, Director, Complex Systems Integration, IBM Global Business Services
– Dr. U. Dinesh Kumar, Professor in Quantitative Methods and Information Systems, Indian Institute of Management Bangalore
– Professor Stephen Cook, Director, Defence and Systems Institute, University of South Australia
July 19 – 23, 2009
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 31
INCOSE Multi-LevelProfessional Certification
Foundation Level
ESEP Expert Systems Engineering ProfessionalCSEP-Acq CSEP w/ US DoD AcquisitionCSEP Certified Systems Engineering ProfessionalASEP Associate Systems Engineering Professional
Entry Level
DisciplineExtensions
Senior Level
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 32
CSEP with US DoD Acquisition Extensions
• Targeted towards Systems Engineers who support or work in a US Department of the Defense acquisition environment
• Same core CSEPs experience, education, and knowledge requirements
• Additional acquisition knowledge items tested• Available since July 2008
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 33
DoD Best Practices Clearinghouse (BPCh)
https://bpch.dau.mil/
The DoD Acquisition Best Practices Clearinghouse (BPCh) facilitates the selection and implementation of systems engineering and software acquisition practices appropriate to the needs of individual acquisition programs.
The BPCh uses an evidence-based approach, linking to existing resources that describe how to implement various best practices.
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 34
We Need You!
• Browse the Clearinghouse for applicable best practices• Provide feedback on practices• Submit additional evidence for an existing practice• Submit a lead or a possible best practice• Tell others about the Clearinghouse• Consider becoming an editor to help vet practices
submitted in your area of expertise
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 35
DoD SE Best Practice Continuum
Policy &Guidance
Teamwork &Collaboration
Developingthe Workforce& AdvancingSE Practice
Program Support& Assessment
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 36
OSD’s Program Support Reviews…“The Source of Systemic Root Cause Analysis
Data”
Defense Acquisition ProgramDefense Acquisition ProgramSupport Methodology (DAPS) vSupport Methodology (DAPS) v1.11.1
Defense Acquisition ProgramDefense Acquisition ProgramSupport Methodology (DAPS) vSupport Methodology (DAPS) v1.11.1DAPS
1.0 Mission Capabilities/Requirements Assessment AreaSub-Area 1.1 – Mission Capabilities/Requirements
2.0 Resources Assessment AreaSub-Area 2.1 – Technology Development Planning and AllocationSub-Area 2.2 – PersonnelSub-Area 2.3 – FacilitiesSub-Area 2.4 – Engineering Tools
3.0 Management Assessment AreaSub-Area 3.1 – Technology Development StrategySub-Area 3.2 – Technology Development PlanningSub-Area 3.3 – Program and Project ManagementSub-Area 3.4 – Contracting and SubcontractingSub-Area 3.5 – Communication
4.0 Technical Process Assessment AreaSub-Area 4.1 – Technology Selection AssessmentSub-Area 4.2 – Requirements DevelopmentSub-Area 4.3 – Functional Analysis & AllocationSub-Area 4.4 – Design SynthesisSub-Area 4.5 – System Integration, Test and VerificationSub-Area 4.6 – Transition to SDDSub-Area 4.7 – Process Improvement
5.0 Technical Product Assessment AreaSub-Area 5.1 – System DescriptionSub-Area 5.2 – System PerformanceSub-Area 5.3 – System Attributes
6.0 Environment Assessment AreaSub-Area 6.1 – Statutory and Regulatory Environment
DAPS1.0 Mission Capabilities/Requirements Assessment Area
Sub-Area 1.1 – Mission Capabilities/Requirements2.0 Resources Assessment Area
Sub-Area 2.1 – Technology Development Planning and AllocationSub-Area 2.2 – PersonnelSub-Area 2.3 – FacilitiesSub-Area 2.4 – Engineering Tools
3.0 Management Assessment AreaSub-Area 3.1 – Technology Development StrategySub-Area 3.2 – Technology Development PlanningSub-Area 3.3 – Program and Project ManagementSub-Area 3.4 – Contracting and SubcontractingSub-Area 3.5 – Communication
4.0 Technical Process Assessment AreaSub-Area 4.1 – Technology Selection AssessmentSub-Area 4.2 – Requirements DevelopmentSub-Area 4.3 – Functional Analysis & AllocationSub-Area 4.4 – Design SynthesisSub-Area 4.5 – System Integration, Test and VerificationSub-Area 4.6 – Transition to SDDSub-Area 4.7 – Process Improvement
5.0 Technical Product Assessment AreaSub-Area 5.1 – System DescriptionSub-Area 5.2 – System PerformanceSub-Area 5.3 – System Attributes
6.0 Environment Assessment AreaSub-Area 6.1 – Statutory and Regulatory Environment
Plan
Conduct
Analyze
Report
Tools & Materials• Tailorable, Scalable
Methodology (DAPS v1.1)• Templates• Criteria• Questions• Training Materials• Execution Guidance• Subject Matter Expertise
Rigor in process …
…ensures integrity
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 37
Program Support Review Data Set (since March 2004)
PSRs/NARs completed: 57AOTRs completed: 13Nunn-McCurdy Certification: 13Participation on Service-led IRTs: 3Technical Assessments: 13Reviews planned for CY08:
PSRs/NARs: 10AOTRs: 1NM: 1
Decision Support Reviews
DAE Review5%
OTRR9%
Other25%
Pre-MS C19%
Pre-MS A3%
Pre-MS B26%
Nunn-McCurdy
13%
Service-Managed Acquisitions
Marine Corps 10%
Army 27%
Navy 21%
Air Force 33%
Agencies 9%
Programs by Domain Area
Unmanned 4%
Land 13%
C2-ISR 13%
Ships 8%Munitions 3%
Rotary Wing 21%
Comms 3%
Space 4%
Business 2%Missiles 9%
Fixed Wing 18%
Other 2%
Data derived from
diverse & broad program set
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 38
• Policy/Guidance• Education & Training• Best Practices
• Policy/Guidance• Education & Training• Best Practices
• Other Processes (JCIDS, etc)• Oversight (DAB/ITAB)• Execution (staffing)
• Other Processes (JCIDS, etc)• Oversight (DAB/ITAB)• Execution (staffing)
DoD AcquisitionCommunity
Program Review
Findings
ProgramUnique
Solutions
ProgramEffects &
Root Cause
Systemic Issues
SystemicRoot
Causes
Corrective Actions
Tactical, Program and Portfolio ManagementTactical, Program and Portfolio Management
Strategic ManagementStrategic Management
OSD Systemic Analysis: Data Model
Use collective set of program findings…
… to identify systemic issues at the root cause level…
… and develop recommendations that mitigate
problems at their source!
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 39
• Use “Core” and “Systemic” Root Cause tagging to cull data into focused subsets• Analyze and trend using Pareto charting and Pivot Tables
PSRfindings
PSRfindings
“Negative”Findings
“Negative”Findings
AnalysisPareto &
PivotTables)
AnalysisPareto &
PivotTables)
FindingsFindingsPrelim.
AnalysisResults
Prelim.AnalysisResults
Deriving Systemic Issues…
Hypothesis testing: Re-analyze, review…44 Programs
Systemic Issues
Systemic Issues
Occurred on 50% or more of the programs
Occurred on 50% or more of the programs
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 40
SRCA Milestones & Results
OUSD/SSE root cause analysis yields systemic issues and preliminary recommendations
OUSD/SSE sponsored NDIA Task Group to develop executable acquisition recommendations
NDIA Task Group concludes effort and proposes actions for both government and industry
Sep 07 – Feb 08Sep 07 – Feb 08
Mar 08 – Oct 08Mar 08 – Oct 08
Oct 08 – Dec 08Oct 08 – Dec 08
3 Recommendation
Areas
3 Recommendation
Areas
Acquisition Strategy & Planning
(ASP)
Acquisition Strategy & Planning
(ASP)
DecisionGate
Review(DGR)
DecisionGate
Review(DGR)
Enhanced Staff
Capability(ESC)
Enhanced Staff
Capability(ESC)Early & sufficient
planningEarly & sufficient
planningClear, enforceable execution criteriaClear, enforceable execution criteria
People: the right skills and numbersPeople: the right skills and numbers
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 41
Full RateProductionDecisionReview
CBAJoint Concepts
MS CMS B
Strategic Guidance
MS A
ICD TechnologyDevelopment
CDDEngineering and Manufacturing
Development and Demonstration
CPDProduction and
Deployment O&S
New Opportunitiesfor Independent Reviews
MDD
Materiel Solution
Analysis
* PDR – Preliminary Design Review * CDR – Critical Design Review * OTRR – Operational Test Readiness Review
What’s relevant: Mandatory Milestone A for all “major weapon systems” MS B after system-level PDR* and a PDR Report to the MDA EMDD with Post-CDR* Report and MDA Assessment PSR and AOTR in policy
OTRRPotential Independent Technical Reviews - PSRs and AOTRs
Program Support Reviews (PSRs) All ACAT ID and IAM To inform the MDA on technical planning and management processes thru risk identification and mitigation recommendations To support OIPT program reviews and others as requested by the MDA
Assessments of Operational Test Readiness (AOTRs) All ACAT ID and special interest programs To inform the MDA, DOTE, & CAE of risk of a system failing to meet operational suitability and effectiveness goals To support CAE determination of materiel readiness for IOT&E
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 42
DoD SE Best Practice Continuum
Policy &Guidance
Teamwork &Collaboration
Developingthe Workforce& AdvancingSE Practice
Program Support& Assessment
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 43
The Mission:Delivering Timely and Affordable Capabilities to the Warfighter
The Defense Acquisition Community126,033 Government and Military Certified Professionals
500,000+ Defense Industry Personnel
Always Our Focus
For additional information: http://www.acq.osd.mil/sse
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 44
DoD Guidance (1/2)• Defense Acquisition
– Defense Acquisition Guidebook (DAG)– Integrated Defense Acquisition, Technology, & Logistics Life Cycle Management Framework, Version 5.3.2:
December 8, 2008 • Systems Engineering
– DAG Chapter 4, Systems Engineering– Systems Engineering Plan (SEP) Preparation Guide, Version 2.01, April 2008– SEP Frequently Asked Questions (FAQs), February 8, 2008– SE WIPT Brief: An Overview of Technical Planning and Systems Engineering Plan (SEP) Development,
Version 1.0, March 20, 2008– SE Working Integrated Product Team (WIPT) Generic Charter Template – Systems Engineering Guide for Systems of Systems, Version 1.0, August 2008– Systems Engineering Assessment Methodology, Defense Acquisition Program Support (DAPS), Version 2.0,
October 21, 2008– Guide for Integrating Systems Engineering into DoD Acquisition Contracts, Version 1.0, December 11, 2006 – Risk Management Guide for DoD Acquisition, 6th Edition, Version 1, August 2006– Integrated Master Plan / Integrated Master Schedule Preparation and Use Guide, Version 0.9, October 21,
2005– Program Manager's Guide: A Modular Open Systems Approach (MOSA) to Acquisition, Version 2.0,
September 2004 – Modular Open Systems Approach (MOSA) Program Assessment and Rating Tool (PART), Version 1.02 – DoD Guide for Achieving Reliability, Availability and Maintainability, August 1, 2005– Program Reliability and Maintainability Review Template, Version 1.0, August 15, 2008 – Designing and Assessing Supportability in DOD Weapon Systems: A Guide to Increased Reliability and
Reduced Logistics Footprint, October 24, 2003 – Technical Review Checklists
http://www.acq.osd.mil/sse/pg/guidance.html
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 45
DoD Guidance (2/2)
• Developmental Test and Evaluation (DTE) – DAG Chapter 9, Integrated Test and Evaluation– Guide on Incorporating Test and Evaluation into DoD Acquisition Contracts, FINAL DRAFT
FOR FINAL APPROVAL VERSION, September 2008 • Modeling and Simulation (M&S)
– DoD 5000.59-M, DoD Modeling and Simulation (M&S) Glossary, January 15, 1998– Recommended Practices Guide for Verification, Validation, and Accreditation (VV&A), Build
3.0, September 2006– Guide for Modeling and Simulation for the Acquisition Workforce, Version 1.01, October 2008
• Safety– System Safety — ESOH Management Evaluation Criteria for DoD Acquisition, Version 1.1,
January 2007 – Joint Systems Safety Review Guide for USSOCOM Programs, Version 1.1, October 12, 2007– Unmanned System Safety Guide for DoD Acquisition, Version 0.92, June 27, 2007– Joint Weapons and Laser Safety Review Guide (Draft), Version 0.92, September 4, 2007– ESOH in Acquisition: Integrating ESOH into SE, Version 3.0, January 2008
• System Assurance – DAG Chapter 7, Acquiring Information Technology and National Security Systems – DAG Chapter 8, Intelligence, Counterintelligence, and Security Support – DoD 5200.1-M, Acquisition Systems Protection Program, March 16, 1994 – Engineering for System Assurance, Version 1.0, October 2008
http://www.acq.osd.mil/sse/pg/guidance.html
SYSTEMS & SOFTWARE ENGINEERING DIRECTORATE, DUSD(A&T) 46
DoD Best Practices Clearinghouse (BPCh) Program Goals
A Single Source For answers about
practices, how to apply them, when they are good to use; lessons learned; and risks to
avoid
Useful Information Help finding,
selecting, and implementing
practices appropriate to User’s situation; fill the gap between “what” and “how”
Validated practices Consistent, verifiable
information
Repository
Active Knowledge BaseNot just another practice
list; experience data updated, expanded refined; encourages
organic growth
Living KnowledgeIntegration with
DoD communities of practice (ACC)
and experts