Lesson12: Supportability Design Reviews · Web viewBefore delving into each specific Supportability...

LOG 211 Supportability Analysis Student Guide

Lesson12: Supportability Design Reviews

Conte

nt

Slide12-1. Lesson 12: Supportability Design Reviews

Welcome to Lesson 12 on Supportability Design Reviews.

January 2013Final v1.3

1 of 57


Topic 1: Introduction

Content

Slide 12-2. Topic 1: Introduction

2 of 57 January 2013Final v1.3


Content

Slide 12-3. Life Cycle Management Framework: Where Are You? What Influence Do You Have?

Supportability design reviews offer Life Cycle Logistician’s (LCL’s) opportunities to impact a system, from “Influencing the Design” to “Supporting the Design” in the life cycle. There are numerous design reviews throughout the Life Cycle Management Framework, but for the purposes of this lesson, the Supportability design reviews listed below will be covered.

Materiel Solution Analysis Phase

1. Alternative Systems Review (ASR)

Technology Maturation and Risk Reduction (TMRR) Phase

2. System Functional Review (SFR)3. Preliminary Design Review (PDR)

Engineering & Manufacturing Development Phase

4. Critical Design Review (CDR)5. Developmental Test and Evaluation (DT&E)6. Functional Configuration Audit (FCA)7. Production Readiness Review (PRR)

Production & Deployment Phase

8. Initial Operational Test and Evaluation (IOT&E)9. Physical Configuration Audit (PCA)


3 of 57

Technology Maturation & Risk Reduction


Content

Content

Slide 12-4. Topics and Objectives



Content

Content

Slide 12-5. Supportability Design Reviews Lesson Approach

Key questions to ask for each review in this lesson are:

What inputs are necessary for a Supportability design review at this phase in the Life Cycle Management Framework?

How does this particular design review impact the system’s Supportability?

What must be achieved before this review is considered complete?


5 of 57


Topic 2: Overview of Supportability Design Reviews

Content

Slide 12-6. Topic 2: Overview of Supportability Design Reviews



Content

Slide 12-7. What Are Supportability Design Reviews?

Before delving into each specific Supportability design review, it is helpful to understand how reviews are used at the program level. From the perspective of the Program Manager, these reviews are an opportunity to ascertain whether or not certain criteria (described as “exit criteria” in this lesson) have been met and whether a decision to proceed is warranted. In general, the LCL supports these reviews through the Supportability analyses discussed throughout this course. The LCL should participate in a review to explain how those analyses support a decision as to whether the exit criteria have been met and the program can proceed.

In summary, these reviews may:

Serve to confirm major technical efforts within the Life Cycle Management Framework

Verify the outputs from each phase within the Life Cycle Management Framework

Ascertain progress in defining system technical requirements Ensure that the system under review has a reasonable expectation of

being judged operationally effective and suitable Be required by contract and/or DoD policy to include review processes

and requirements


7 of 57



Content

Content

Slide 12-8. Supportability Design Reviews: Overview

In general, Supportability design reviews follow the same basic pattern:

1. An event serves as an impetus for a review conference to convene. Note: Although most reviews are part of a master schedule, the reviews are conducted to determine if the criteria have been met, not when the criteria have been met. This can be refered to as “event-driven” as opposed to “schedule-driven.”

2. The review consists of the IPTs and other stakeholders applying various criteria and Supportability questions.

3. Upon successfully completing all actions associated with a particular review, the system moves to the next review, development phase and/or project milestone. Often the outputs are used in the next review.

Although each design review requires specific entry/exit criteria to make particular decisions, in general, they include the inputs, processes and outputs described below.




Content

Supportability Design Review Requirements

Entry Criteria: Phase-specific accomplishments required to be completed before a program is allowed to enter a particular phase in the Life Cycle Management Framework

The program may have to fulfill criteria, including:

Maturity Performance Documentation

Inputs: Standup Integrated Product Teams (IPTs)

Identify role: Who is doing what? Define design review goal Define schedule/timeline

Supportability Considerations/Questions:

Affordability? Effective Design? Supportability? Functionality? Design Readiness? Design Capability? Producibility?

Outputs: Integrated Product Teams

Assess the system’s Supportability requirements and characteristics for compliance to the review’s entrance and exit criteria

Identify Supportability deficiencies, identify potential impacts to the design and make recommendations regarding their resolution

Identify requirements and mechanism for exchanging information, to include analyses, reports, and data

Assign action items, specifying responsibilities, schedule and expected outcome(s)

Exit Criteria: Program-specific accomplishments that must be satisfactorily demonstrated before a program can progress further in the current Life Cycle Management Framework or transition to the next phase.

Specific Supportability Design Review Requirements, the Defense Acquisition Guidebook and Assessment Checklists

Inputs/outputs, entry/exit criteria, and Supportability questions/considerations specific to a particular review are discussed in the next topic. In addition Chapters 4, 5, and 9 of the Defense Acquisition Guidebook (DAU) provide detailed discussion of technical reviews requirements for Systems Engineering, Life Cycle Logistics and the Test & Evaluation communities.


9 of 57


Content

Content

For example, Section 4.3, Systems Engineering Activities In the Life Cycle, outlines the Technical Review process. To assist in the preparation for and conduct of technical reviews technical review risk assessment checklists are available for each of the reviews. These checklists are designed as a technical review preparation tool, and should be used as the primary guide for the risk assessment during the review. The checklist itself can be both an input to, and an output of, the review. The Integrated Product Team can do a self-assessment as input, but the review participants should take ownership of the assessed risk as an output of the review. These checklists are available on the Systems Engineering Community of Practice and are accessible in the DAU Technical Reviews Continuous Learning Module, CLE-003.



Content

Slide 12-9. Design Reviews and Affordability

The logistician’s Supportability and life cycle goals during Supportability design reviews should:

Leverage and use performance-based logistics strategies to reinforce optimization of total system availability while minimizing cost and logistics footprint

Merge design effectiveness and process efficiency to gain the maximum mission effectiveness as the goal for performance-based logistics

Offer opportunities to check how the system is developing along effective lines of Affordability and Supportability


11 of 57


Topic 3: Supportability Design Reviews and Criteria

Content

Slide 12-10. Topic 3: Supportability Design Reviews and Criteria



Content

Slide 12-11. Alternative Systems Review (ASR)DefinitionIn the Materiel Solutions Analysis phase, the Alternative Systems Review (ASR) is a technical review that demonstrates the preferred concept is cost effective, affordable, operationally effective, and suitable, and can be developed to provide a timely solution to a need at an acceptable level of risk. This review also determines whether the system is ready to proceed to the next phase, Technical Development.By reviewing alternative materiel solutions, the ASR helps ensure that sufficient effort has been given to conducting trade studies that consider and incorporate alternative system designs that may more effectively and efficiently meet the defined capabilities. A successful review is predicated on the Integrated Product Team's determination that the operational capabilities, proposed solution(s), available technologies, and program resources (funding, schedule, staffing, infrastructure, and processes) form a satisfactory basis for proceeding into the Technology Maturation and Risk Reduction (TMRR) phase.Key ASR QuestionThe Alternative Systems Review answers many questions that will be addressed next. The key ASR question asks whether one or more of the proposed systems are cost-effective, affordable, operationally effective and suitable with an acceptable level of risk. The LCL can address this key question using the inputs, entry criteria, Supportability questions, exit criteria and outputs discussed with the following slide.


13 of 57



Content

Content

Slide 12-12. ASR Process

Inputs

The Alternative Systems Review begins with the following inputs:

Successful completion of:o Initial Technical Review (ITR)o Program System Review (PSR)

A data repository that includes:o Preferred System Solution(s) concept/descriptiono Draft Request for Proposal (RFP)o Analyses results and definitions

System Requirements documento Including interoperability and system distributed services

requirements Initial Capabilities Document (ICD) Alternative maintenance and Sustainment concepts Updated cost and schedule data



Content

In addition to these inputs, the following key entry questions and/or issues need to be addressed:

Is there a plan to conduct an ASR, or has one been completed in preparation for a Milestone A Decision?

In preparation for the ASR, what alternative systems were evaluated during the Capabilities Assessment phase?

Do the stakeholders have an understanding of how available system concepts meet capabilities described in the ICD, and of the Affordability, operational effectiveness and technology risks inherent in each alternative concept?

Supportability Considerations/Questions

During the ASR, you should consider questions or criteria under the following key questions:

1. Affordability? Including support costs, is the system cost effective?2. Effectively Designed? Is it operationally effective and suitable?3. Supportability? Can it be supported in the field to do its mission?

You can find more specific Supportability questions and considerations for the Alternative Systems Review in the Lesson 12 Job Aid that is located after the lesson Summary.

Outputs

ASR is successful when it accomplishes and/or produces the following exit criteria and outputs:

Preliminary system specifications Systems Engineering Plan (SEP) Support and maintenance concepts

The following questions must also be successfully answered:

1. Is the system software scope and complexity sufficiently understood and addressed in the Acquisition Strategy to enable low software technical risk?

2. Has the system technical baseline been captured in a preliminary system specification that is consistent with technology maturity and the proposed program cost and schedule?

(Source: DAG, p. 230)

Successful completion of the Alternative Systems Review leads to the passing of MS A and the start of the Technology Maturation and Risk Reduction (MRR) phase.


15 of 57


Content

Slide 12-13. System Functional Review (SFR)

Definition

In the Technology Maturation and Risk Reduction (TMRR) phase, the System Functional Review (SFR) is a formal review of the conceptual design of the system to establish its capability to satisfy requirements. It establishes the functional baseline and seeks to confirm that the system has a reasonable expectation of satisfying requirements.

Key SFR Question

The System Functional Review answers many questions that will be addressed next. The key SFR question asks whether the system can reasonably satisfy the requirements of the Initial Capabilities Document.

The LCL can address this key question using the inputs, entry criteria, Supportability questions, exit criteria and outputs discussed with the following slide.




Content

Slide 12-14. SFR Process

Inputs

The System Functional Review begins with the following entry criteria and inputs:

Draft Capability Development Document (CDD)o Requirements traced from CDD to system specifications, functional

specifications and subsystem specifications Approved Materiel Solution

o Preliminary Technical Performance Measurements (TPMs) identified and traceable to Measures of Effectiveness (MOE) and Measures of Performance (MOP)

o Functionality assigned to Key Performance Parameters (KPPs)o Verification and validation methodology defined for each

specification requiremento System functional specification completed to subsystem levelo Functional requirements for operations and maintenance assigned

to subsystems, hardware, software, or support after detailed reviews of the design’s architecture

Updatedo Systems Engineering Plano Software Development Plan


17 of 57


Content

Logistics and training requirements identified in system and subsystem specifications

Additional inputs:o Updated Integrated Architecture Viewpointso Updated use case analysiso Completed Preliminary subsystem specification and Interface

Design Documentso Allocated Information Assurance Controls over to system

componentso Created Technology maturation plans for critical technologies.o Defined Technology Maturation and Risk Reduction (TMRR) Exit

Criteria.


During the SFR, you should consider questions or criteria under the following key questions:

1. Affordability: Is the program with the approved functional baseline executable within the existing budget?

2. Functionality: Are the system functional requirements sufficiently detailed and understood to enable system design to proceed?

3. Supportability: Are the Supportability requirements to achieve the support strategy included in the performance specifications?

You can find more specific Supportability questions and considerations for the System Functional Review in the Lesson 12 Job Aid.

Outputs

SFR is successful when it accomplishes and/or produces the following exit criteria and outputs:

Establishes:o System functional baseline with traceability to lower-level

performance requirementso Preliminary system-level maintenance plan

Updates:o Cost Analysis Requirements Description (CARD), or equivalento Test and Evaluation Management Plan (TEMP), or equivalento Program development schedule including system and software

critical path driverso Risk assessment for the EMD phase

(DAG, pp. 243-5)

Successful completion of the System Functional Review leads to the Preliminary Design Review.



Slide 12-15. Preliminary Design Review (PDR)

The Preliminary Design Review (PDR) is the next Pre-Milestone B review we cover in the Technology Maturation and Risk Reduction (TMRR) phase. The PDR influences system design and support by ensuring that the allocated baseline is established. Each function in the functional baseline is allocated to one or more system configuration items to ensure traceability.

Definition

The PDR is a formal review that confirms the preliminary design logically follows the SFR findings and meets the requirements. It normally results in approval to begin detailed design.

Key PDR Question

The Preliminary Design Review answers many questions that will be addressed next. The key PDR question asks whether the system has a reasonable chance of being judged operationally effective, suitable and affordable before becoming a Program of Record (POR) at Milestone B.



19 of 57



Slide 12-16. PDR Process

Inputs

The Preliminary Design Review begins with the following entry criteria and inputs:

Approved Life Cycle Sustainment Plan with updated program Sustainment development efforts and schedules

Subsystem specifications for hardware, software, and associated Interface Control Documents. These enable detailed design or procurement of subsystems.

System requirements/capabilities Test and Evaluation Master Plan (TEMP), including:

o Sell-off criterion for each producto Witness requirementso Documentation

Cost Analysis Requirements Description (CARD), updated as required Risk and opportunity management, reviewed and managed regularly Configuration Management Board running with metrics



In addition to these inputs, the following key entry issues need to be addressed. In preparation for the PDR, are the following activities/actions completed or documents/information available?

Requirementso Design Reliabilityo Design Maintainabilityo Requirements Traceability Matrix

Datao Supplier data describing specific componentso Design data defining major subsystems, equipment, software and

other system elements Design & Analyses

o Reliability & Maintainability Analyses, reports, trade studieso Supportability analysis data/Logistics Product Databaseo Design documentationo Equipment and part standardization

Costs/Riskso Developmental Test & Evaluation planso Spares and Government-Furnished Property (GFP)


During the PDR, you should consider questions or criteria under the following key questions:

1. Affordability: Is the program with the approved functional baseline executable within the existing budget?

2. Design Readiness: Have the majority of manufacturing processes been defined and characterized?

3. Supportability: Does the system meet all logistics and Supportability requirements?

You can find more specific Supportability questions and considerations for the Preliminary Design Review in the Lesson 12 Job Aid.


21 of 57


Content

Outputs

PDR is successful when it accomplishes and/or produces the following exit criteria and outputs:

Establishes:o Preliminary Design Review (PDR) Reporto A system allocated baselineo The Requirements Traceability Matrix (RTM)

Updates:o Cost Analysis Requirements Description (CARD) or CARD-like

document based on the system allocated baselineo Risk assessment for Engineering and Manufacturing Development

(EMD)o Approved Life Cycle Sustainment Plan updating programo Sustainment development efforts and scheduleso Program schedule, including system and software critical path

drivers, and hardware, software, human/support systems

(DAG, pp. 247-50)

Successful completion of the Preliminary Design Review is a prerequisite for a successful Milestone B decision, which permits the system to enter the Engineering and Manufacturing Development phase.



Slide 12-17. Critical Design Review (CDR)

Definition

In the Engineering and Manufacturing Development phase, the Critical Design Review (CDR) is a formal review conducted to evaluate the completeness of the design and its interfaces. In addition to establishing the initial product baseline and placing it under configuration control, the CDR also ensures the system under review has a reasonable expectation of satisfying the requirements of the Capability Development Document (CDD) within the current budget and schedule.

Key CDR Question

From a Supportability perspective, the Critical Design Review answers the question of whether the system meets all logistics and Supportability requirements from the Capability Development Document within the current allocated budget and schedule.



23 of 57



Content

Slide 12-18. CDR Process

Inputs

The Critical Design Review begins with the following entry criteria and inputs:

Approved Life Cycle Sustainment Plan that updates program Sustainment development efforts and schedules

System requirements and capabilities CARD, updated as required Risk and opportunity management reviewed and managed regularly Product baseline under configuration control



Content

In addition to these inputs, the following key entry issues need to be addressed. In preparation for the CDR, are the following activities/actions completed or documents/information available?

Designo Prepare production baseline (“Build To” documentation)o Determine if the system design documentation (product baseline,

including Item Details Specs, Material Specs, Process Specs) is satisfactory to start initial manufacturing

o Capture product baseline in the detailed design documentationo Review test plans to assess if test efforts are developing sufficiently

to indicate the Test Readiness Review will be successful Costs/Risk

o Produce published agenda (several days prior to the conference) Prior Reviews

o Completed all action items related to the previous conference (PDR) successfully


During the CDR, you should verify the achievement of the following criteria:

1. Affordability: Is the program executable with the existing budget and the approved product baseline?

2. Design Capability: Does the detailed design satisfy the Capability Development Document or any available draft Capability Production Document?

3. Supportability: Does the system meet all logistics and Supportability requirements?

You can find more specific Supportability questions and considerations for the Critical Design Review in the Lesson 12 Job Aid.

Outputs

CDR is successful when it accomplishes and/or produces the following exit criteria and outputs:

Establishes:o Post-CDR Reporto System initial product baseline is established and places it under

configuration controlo Issues and Actions Summary


25 of 57


Content

o Completed FMECA/FTA/ Safety and Maintainability analyseso Estimates of System R&Mo Supportability requirement enablers including Human Systems

Integration (HSI) design features, with Environment, Safety and Occupational Health (ESOH) risk reduction features included in the design

Updates:o Cost Analysis Requirements Description (CARD) or CARD-like

document based on the system product baseline.o Risk Assessment for the EMD phase, including issues/risks that

could result in a breach to the program baseline or substantively impact cost, schedule or performance.

o Approved Life Cycle Sustainment Plan, including: Updated program Sustainment development efforts Schedules based on current budgets Test evaluation results

o Supportability design features Developmental Testing Results confirmed for key Sustainment enablers or drivers requirements are complete. If the test results to date do not indicate the operational test is likely to succeed or risk has increased, new developmental and operational testing criteria and plans should be considered, along with fallback plans.

(DAG, pp. 262-5)

Successful completion of the Critical Design Review leads to the next review in the Engineering and Manufacturing Development phase.



Content

Slide 12-19. Developmental Test and Evaluation (DT&E)

Developmental Test and Evaluation (DOT&E) activities are principally conducted during the Technology Maturation and Risk Reduction (TMRR) and Engineering and Manufacturing Development (EMD) phases to ensure the system design satisfies desired capabilities.

Definition

DT&E is an engineering test process conducted to provide data for the assessment of a design’s achievement of critical system performance parameters. The testing is performed at the component, subsystem and system-level configurations of hardware and software, and ensures the verification and validation of the Systems Engineering process.

DT&E is conducted as part of the design process. The testing may be conducted by the Contractor under Government oversight, independently by the Government, or jointly by the parties. DT&E differs from Initial Operational Test and Evaluation (IOT&E) in that the IOT&E is conducted by the Service’s Operational Test Agency (OTA) and determines the system’s operational effectiveness and suitability in the operational environment.

Key DT&E Question

Developmental Test and Evaluation (DT&E) answers the question of whether the system design solution satisfies desired capabilities. The LCL can address this key question using the inputs, entry criteria, Supportability questions, exit criteria and outputs discussed with the following slide.


27 of 57



Content

Content

Slide 12-20. DT&E Process

Inputs

Developmental Test and Evaluation begins with the following entry criteria and inputs:

Areas needing test to complete FMECA/FTA/RCM Analysis Testing required as part of Reliability improvement program Performance estimates and specifications

In addition to these inputs, the following key entry questions need to be addressed:

Are Modeling and Simulation (M&S) uses in Developmental Test and Evaluation (DT&E), Operational Test and Evaluation (OT&E), Live Fire Test and Evaluation (LFT&E), System of Systems (SoS) interoperability testing and information assurance testing integrated in an efficient continuum?

Has the system demonstrated sufficient technical maturity into Initial Operational Test and Evaluation (IOT&E) regarding Critical Technical Parameters (CTPs), including interoperability, and is it documented in the Test and Evaluation Master Plan (TEMP)?



Content


During DT&E, you should focus on the following key question:

Supportability: Are Supportability metrics being met?

You can find more specific Supportability questions and considerations for the Developmental Test and Evaluation in the Lesson 12 Job Aid.

Outputs

DT&E is successful when it accomplishes and/or produces the following exit criteria and outputs:

Areas requiring redesign Improvements in Reliability

Successful completion of the Developmental Test and Evaluation leads to the next review in the Engineering and Manufacturing Development phase, the Functional Configuration Audit (FCA).

(DAG, pp. 802-3)


29 of 57


Content

Slide 12-21. Functional Configuration Audit (FCA)

Definition

The Functional Configuration Audit (FCA) is the second review in the EMD phase and may be conducted concurrently with the System Verification Review (SVR). SVR is a multi-disciplined product and process assessment that ensures the system under review can proceed into Low-Rate Initial Production.

The FCA is a formal review conducted to verify that all subsystems can perform all of their required design functions in accordance with their functional and allocated configuration baselines.

Key FCA Question

The Functional Configuration Audit answers the question of whether the functionality or performance of hardware and software configuration items meets their specifications.





Content

Content

Slide 12-22. FCA Process

Inputs

The Functional Configuration Audit begins with the following entry criteria and inputs:

Accomplishment of program Supportability objectives detailed in acquisition documents (CDD, LCSP)

A completed system Technical Data Package Developmental Test & Evaluation (DT&E) results

In preparation for the FCA, are the following activities/actions completed or documents/information available?

Requirements:o Functional and allocated baselineso Verification is comprehensive and complete

Testing:o Test procedures and resultso Preproduction and production test results

Design & Analyses:o Configuration audits, including completion of all change actions,

have been completed for all configuration itemso Systems engineering planning is updated for production


31 of 57


Content

Costs/Risks:o Critical achievements, success criteria and metrics have been

established for productiono Risk management planning has been updated for productiono Readiness issues for continuing design, continuing verifications,

production, training, deployment, operations, support and disposal have been resolved


During FCA, you should focus on the following key question:

Functionality: Do all the subsystems do what they are supposed to do according to the functional configuration?

Outputs

FCA is successful when all FCA-related actions and risks are identified, resolved or effectively mitigated to an acceptable level, and the EMD product is sufficiently mature for entrance into Low-Rate Initial Production.

Successful completion of the Functional Configuration Audit leads to the next review in the Engineering and Manufacturing Development phase, the Production Readiness Review (PRR). (DAG, pp. 267-8)



Content

Slide 12-23. Production Readiness Review (PRR)

Definition

The Production Readiness Review (PRR) is the last review covered in the EMD phase. The PRR is a formal examination of a program to determine if:

The design is ready for production Production engineering problems have been resolved The producer has accomplished adequate planning for the Production

and Deployment phase

Key PRR Question

The Production Readiness Review answers the question of whether the system design is ready for production.



33 of 57



Content

Content

Slide 12-24. PRR Process

Inputs

The Production Readiness Review (PRR) begins with the following entry criteria and inputs:

Long lead-time items identified and ordered Mature software Manufacturing processes identified and validated Special tooling: jigs, fixtures and other manufacturing aids available Quality assurance procedures and methods in place Manufacturing personnel trained

In addition to these inputs, the following key entry issues need to be addressed. In preparation for the PRR, are the following activities/actions completed or documents/information available?

Designo The developer’s design is complete.o Product design is low risk from the standpoint of producibility and

has stabilized.o The design is in consonance with the operational, maintenance and

support concepts, including meeting inter-service and foreign interoperability requirements.



Content

Costs/Risko Verification of the design has been accomplished, including

qualification of subsystems and components as appropriate, and demonstration of performance and R&M characteristics.

o Long lead-time materials identified, and action initiated for advance procurement where appropriate. Sole source items are identified, and continuity of supply is ensured.

o Production cost projections have been made and are well supported.

Datao The Technical Data Package will permit competitive acquisition and

domestic and foreign co-production, where appropriate. Prior Reviews

o A Critical Design Review has been accomplished and discrepancies resolved.

Training and Manpowero Skilled production manpower will be available in sufficient numbers

for the planned term of production.Supportability Considerations/QuestionsDuring the PRR, you should focus on the following key question:Producibility: Are we ready to go into production? Producibility is the degree to which “Design for Manufacturing” concepts have been used to influence system and product design. Producibility facilitates timely, affordable, and optimum-quality manufacture, assembly, and delivery of system to the field. You can find more specific Supportability questions and considerations for the Production Readiness Review in the Lesson 12 Job Aid.OutputsPRR is successful when it accomplishes and/or produces the following exit criteria and outputs:

IPT reviews readiness of manufacturing process, including:o Quality management systemo Production planning (i.e., facilities, tooling and test equipment

capacity, personnel development and certification, process documentation, inventory management, etc.)

IPT determines:o System requirements are fully met in the final production

configuration.o Production capability forms a satisfactory basis for proceeding into

Low-Rate Initial Production (LRIP) and Full-Rate Production. Remaining risks have been captured and risk mitigation plan is in place.

Successful completion of the Production Readiness Review leads to the next review in the Engineering and Manufacturing Development phase. (DAG, pp. 268-9)


35 of 57


Content

Content

Slide 12-25. Life Cycle Management Framework: Where Are You?

What Influence Do You Have?

You have reached the Production and Deployment phase of the course. You will learn about two Supportability design reviews that occur during this phase. First, let’s examine the general characteristics of the Production and Deployment phase.




Content

Slide 12-26. Production and Deployment Phase

The Production and Deployment phase commences at Milestone C. During the Production and Deployment phase, the system should achieve operational capability that satisfies mission needs.

Inputs to Production and Deployment include:

Evaluation results from testing Identification of:

o Exit criteria for the Production and Deployment Phaseo Entry criteria for the Operations and Support Phase

Acquisition program baseline Capability Development Document and Capability Production

Document Systems Engineering Plan Test and Evaluation Master Plan Programmatic Environment, Safety, and Occupational Health Evaluation Product support element requirements System Safety Analyses to include updated ESOH Risk Analysis (section

4.4.7.5), update to Safety Requirements Criteria Analysis, System Safety Hazard Analysis


37 of 57

SAE GEIA-STD-0007


Content

Content

Two work efforts take place during the Production and Deployment phase:

Low-Rate Initial Production (LRIP) Full-Rate Production and Deployment (FRP&D)

(DAG, p. 271)



Content

Slide 12-27. Initial Operational Test and Evaluation (IOT&E)

Definition

Initial Operational Test and Evaluation (IOT&E) is the last review covered in this lesson. Earlier we discussed DT&E, which the design team conducted. IOT&E is an evaluation of operational effectiveness and Suitability made by an independent operational test agency, with user support as required.

Key IOT&E Question

The Initial Operational Test and Evaluation answers the question of whether the system performs in an operationally effective and suitable manner under realistic operational conditions.



39 of 57



Content

Slide 12-28. IOT&E Process

Inputs

The Initial Operational Test and Evaluation (IOT&E) begins when the developing organization provides the following inputs to an independent testing authority:

System Reliability, Maintainability and Supportability performance requirements, demonstrated in as realistic an operating environment as possible

Completed:o Life Cycle Sustainment Plano Systems Engineering Reporto Logistics Product Databaseo Test and Evaluation Master Plano Technical manualso Training plan/programo Provisioning plano Product support strategy



Content

Content


The Initial Operational Test and Evaluation process works differently than the reviews discussed previously. In IOT&E, the developing organization hands off the inputs listed above to an independent agency who reports to Congress. The independent agency will compile a list of deficiencies and trouble tickets. The LCL must address and resubmit the fixed deficiencies to the independent agency.

During the IOT&E, the independent agency will be answering the following questions:

Supportability: Does the system satisfy the user’s needs?

Suitability: Does it work as advertised including Supportability?

You can find more specific Supportability questions and considerations for the Initial Operational Test and Evaluation in the Lesson 12 Job Aid.

Outputs

When the independent agency finds that the deviances and related issues have been solved the system can proceed out of IOT&E with the following outputs:

Issued the Beyond Low Rate Initial Report (BLRIR) to Congress Measured the system’s operational effectiveness and suitability

performance in operational use.

Successful completion of the Initial Operational Test and Evaluation leads to the next review in the Production and Deployment phase. (DAG, pp. 811-3)


41 of 57


Content

Slide 12-29. Physical Configuration Audit (PCA)

DefinitionThe Physical Configuration Audit (PCA) is a formal audit that establishes the final product baseline as reflected in an early production configuration item, and is conducted before the decision for Full-Rate Production.A configuration item (CI) is any item that satisfies an end-use function and is designated for separate configuration management. Any item required for Product Support and designated for separate procurement is a CIThe Physical Configuration Audit answers the question of what is the actual configuration of an item being produced; it is the final check before hand-off to Sustainment.

Please note: The PCA risk assessment checklist is designed as a technical review preparation tool, and should be used as the primary guide for assessing risk during the review. This checklist is available on the Systems Engineering Community of Practice.

Key PCA Questions

What is the actual physical configuration of the item being produced? Is the item accurately reflected in the LPD?

The LCL can address these key questions using the inputs, entry criteria, Supportability questions, exit criteria and outputs discussed with the following slide.




Content

Content

Slide 12-30. PCA Process

InputsPCA begins with the following inputs, entry questions and/or issues:

Complete Technical Data Package, describing the product baseline Completed manufacturing and quality control plans


During PCA, you should focus on the following key questions:

Producibility: Do we have a clear picture of the total configuration of the system? Do we know all the parts in the system? Can we produce at full rate production?


43 of 57


Content

Outputs

PCA is successful when it accomplishes and/or produces the following exit criteria and outputs:

All PCA-related actions and risks are identified, resolved or effectively mitigated to an acceptable level.

The design and manufacturing documentation match the item as specified in the contract.

Signed PCA certificate of completion or approval between the developing and the sustaining organizations that is submitted to the Milestone Decision Authority (MDA) for Full Rate Production(FRP)

(DAG, pp. 274-5)



Content

Slide 12-31. IPS Elements and IPTs

The Integrated Product Teams conduct Supportability Design Reviews to:

Assess the system’s Supportability requirements and characteristics for compliance to the review’s entrance and exit criteria

Identify Supportability deficiencies, identify potential impacts to the design and make recommendations regarding their resolution

Identify requirements and mechanism for exchanging information, to include analyses, reports, and data

Assign action items, specifying responsibilities, schedule and expected outcome(s)

The IPTs most involved in developing the design interface for Supportability include:o Human Systems Integrationo Test & Evaluationo Product Support Managemento Systems Engineering

These teams will be responsible for providing updates to documents such as the Life Cycle Sustainment Plan, the Systems Engineering Plan, the Test and Evaluation Master Plan, the RAM-C Rationale Report and others.


45 of 57


Topic 4: Exercise

Content

Slide 12-32. Topic 4: Exercise



Content

Slide 12-33. Exercise

Purpose

The purpose of the exercise is for students to apply their understanding of logistics Supportability issues during design reviews. Students will differentiate impacts that Supportability design reviews have on Supportability and Supportability Analysis.

Instructions

1. Log into Blackboard and navigate to the Lesson 12 Exercise.2. Answer the questions.3. Submit your answers in Blackboard.


47 of 57


Topic 5: Summary

Content

Slide 12-34. Topic 5: Summary



Content

Content

Slide 12-35. Takeaways


49 of 57



Content

Content

Slide 12-36. Summary



Supportability Design Reviews Job Aid

The purpose of this job aid is to provide the student with an organized collection of specific Supportability questions and considerations for each of the Supportability Design Reviews covered in Lesson 12: Supportability Design Reviews.

Supportability questions and considerations for the following Supportability design reviews are covered:

Materiel Solution Analysis Phase

Alternative Systems Review (ASR)

Technology Maturation and Risk Reduction (TMRR) Phase

System Functional Review (SFR) Preliminary Design Review (PDR)

Engineering & Manufacturing Development Phase

Critical Design Review (CDR) Developmental Test and Evaluation (DT&E) Functional Configuration Audit (FCA) Production Readiness Review (PRR)

Production & Deployment Phase

Initial Operational Test and Evaluation (IOT&E) Physical Configuration Audit (PCA)


51 of 57



Alternate Systems Review

Affordability: In addition to answering the question about the cost-effectiveness of the system, you should also ask:

Are the risks for competitive prototyping and initial development (through to the allocated baseline) known and manageable?

Have required investments for technology maturation, to mature design and manufacturing related technologies, been identified and funded?

Effectively Designed: In addition to asking whether the system is operationally effective and suitable, you should also ask:

Is the proposed materiel solution(s) sufficiently detailed and understood to enable entry into Technology Maturation and Risk Reduction (TMRR) with low technical risk?

Are the system software scope and complexity sufficiently understood and addressed in the planning for the Technology Maturation and Risk Reduction (TMRR) phase to enable an acceptable/manageable level of software technical risk?

Is the Technology Maturation and Risk Reduction (TMRR) work effort properly staffed? Is the Technology Maturation and Risk Reduction (TMRR) work effort executable within the

existing budget and schedule? Has a preliminary system specification, consistent with technology maturity and the

proposed program cost and schedule, been captured in the system technical baseline?

Supportability: In addition to asking whether the system is supportable in the field, you should also ask:

Have the preliminary manufacturing processes and risks been identified for prototypes? Have initial producibility assessments of design concepts been completed? Is the program schedule for the Technology Maturation and Risk Reduction (TMRR) Phase

executable (technical/cost risks)? Are the hazards sufficiently understood and addressed to achieve an

acceptable/manageable level of ESOH risk in the Technology Maturation and Risk Reduction (TMRR) phase?

(Source: DAG P. 230)



System Functional Review

Affordability: In addition to answering the question about the existing budget of the system, you should also ask:

Does the updated cost estimate fit within the existing budget? Are adequate processes and metrics in place for the program to succeed? Are the risks known and manageable for development? Is the program schedule executable (technical/cost risks)? Is the updated Cost Analysis Requirements Description (CARD) consistent with the approved

functional baseline?

Functionality: In addition to asking questions about whether system design can proceed you should also ask:

Has the system Functional Baseline been established to enable preliminary design to proceed with proper configuration management?

Can the system functional requirements, as disclosed, satisfy the draft Capability Development Document?

Has a draft preliminary Software Requirements Specification been defined, with complete verification requirements?

Has an initial software architecture design been defined? Is the software functionality in the approved functional baseline consistent with the updated

software metrics and resource-loaded schedule?

Supportability: In addition to asking questions about whether Supportability requirements are included in specifications you should also ask:

Are the program development efforts required to achieve the sustainment key performance parameters, key system attributes, and enabler metrics, along with their corresponding schedules, included in the program documentation and Life-cycle Sustainment Plan?

Have the system-level hazards been reviewed and mitigating courses of action been identified?

Is the program properly staffed?

(DAG, pp. 243-5)


53 of 57


Preliminary Design Review

Affordability: In addition to asking questions about the existing budget you should also ask:

Are the risks known and manageable for integrated testing and developmental and operational evaluation?

Is the program schedule executable (technical/cost risks)? Is the program properly staffed? Has the program‘s cost estimate been updated? Is the preliminary system level design producible within the production budget? Is the updated CARD consistent with the approved allocated baseline?

Design Readiness: In addition to asking questions about manufacturing processes you should also ask:

Does it follow functional review requirements? Does the status of the technical effort and design indicate operational test and evaluation

success (operationally effective and suitable)? Can the preliminary design, as disclosed, satisfy the draft Capability Development

Document? Has the system allocated baseline been established and documented to enable detailed

design to proceed with proper configuration management? Have the majority of manufacturing processes been defined and characterized? Are initial manufacturing approaches documented? Have producibility assessments of key technologies been completed? Has a production cost model been constructed?

Supportability: In addition to asking questions about meeting logistics requirements you should also ask:

Are adequate processes and metrics in place for the program to succeed? Have sustainment and human integration design factors been reviewed and included, where

needed, in the overall system design? Can the industrial base support production of development articles? Have long-lead and key supply chain elements been identified? Can the risks associated with ESOH hazards be mitigated to an acceptable risk level within

the existing budget?

(DAG, pp. 247-50)



Critical Design Review

Affordability: In addition to asking questions about the existing budget you should also ask:

Is the detailed design producible within the production budget? Is the updated Cost Analysis Requirements Description (CARD) consistent with the approved

product baseline? Does the updated cost estimate fit within the existing budget?

Design Capability: In addition to asking questions about the design satisfying the Capabilities Development Document you should also ask:

Does the status of the technical effort and design indicate operational test and evaluation success (operationally effective and suitable)?

Has the system product baseline been established and documented to enable hardware fabrication and software coding to proceed with proper configuration management?

Is the software functionality in the approved product baseline consistent with the updated software metrics and resource-loaded schedule?

Have key product characteristics having the most impact on system performance, assembly, cost, Reliability, and sustainment or safety been identified?

Have the critical manufacturing processes that affect the key characteristics been identified and their capability to meet design tolerances determined?

Have process control plans been developed for critical manufacturing processes? Have manufacturing processes been demonstrated in a production representative

environment? Are detailed trade studies and system producibility assessments underway? Are materials and tooling available to meet pilot line schedule? Has the system production cost model been updated, allocated to subsystem level, and

tracked against targets? Are long lead procurement plans in place and has the supply chain been assessed? Are the ESOH residual risks known and manageable?

Supportability: In addition to asking questions about meeting logistics requirements you should also ask:

Supportability requirement enablers, such as Human Systems Integration (HSI) design features, inclusive of the environment, safety and occupational health risk reduction features, are included in the design.

Failure Mode Effects and Criticality Analysis have been completed and any remaining subsystem requirements for the product support package elements design are complete.


55 of 57


Key sustainment characteristic drivers (including critical manufacturing processes to achieve them) have been identified, and an estimate of system Reliability based on demonstrated Reliability rates and other sustainment drivers are being used in developing the product support package.

Development testing results are used to update the sustainment metric projection estimates and any planned corrective actions to hardware/software deficiencies have been identified.

Test success criteria for any remaining development testing and operational testing plans (for testing both operationally effective and suitable) for key sustainment enablers or drivers requirements are complete. If the test results to date do not indicate the operational test success is likely or risk has increased, new developmental and operational testing criteria and plans should be considered, along with fallback plans.

Program sustainment development efforts with corresponding schedules, including system fabrication, test, and software critical path drivers, are included in LCSP updates.

Has the detailed design satisfied sustainment and Human Systems Integration requirements?

Are adequate processes and metrics in place for the program to succeed? Are the risks known and manageable for testing in support of developmental and

operational evaluation objectives?

(DAG, pp. 262-5)

Developmental Test and Evaluation

Are critical system design requirements, including Supportability metrics, being met? Is there a negative or positive influence on Supportability from the Reliability predictions

and assessments based on the test data

(DAG, pp. 802-3)

Functional Configuration Audit

Functionality: Do all the subsystems do what they are supposed to do according to the functional configuration?

(DAG, pp. 267-8)

Production Readiness Review



Additional Producibility questions you should consider are:

Do we have long lead items?o Are all technologies mature enough for production? o Is the supply chain established and stable with materials available to meet planned low

rate production? Have we identified the skills and machines?

o Is the program properly staffed? o Are the production facilities ready and required workers trained? o Are the ESOH residual risks known and manageable?

Have we laid out processes and process controls?o Has the system product baseline been established and documented to enable hardware

fabrication and software coding to proceed with proper configuration management? o Are adequate processes and metrics in place for the program to succeed? o Are the risks known and manageable? o Is the detailed design producible within the production budget?

(DAG, pp. 268-9)


57 of 57

Date post:	14-Mar-2020
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times