Technical Reviews Moduleorigins.sese.asu.edu/ses405/Class Notes/Technical... · Exploration Systems...

Exploration Systems Engineering: Technical Reviews Module

Technical Reviews Module

Exploration Systems Engineering, version 1.0

Exploration Systems Engineering: Technical Reviews Module 2

Module Purpose: Technical Reviews

♦  To understand the purpose and value of conducting technical reviews.

♦  To discuss the timing of technical reviews over the course of a project’s life cycle.

♦  To consider the entrance criteria and success criteria for the standard project technical reviews.

♦  To understand when a technical review is complete.


Definition of Technical Reviews

♦  Technical reviews are scheduled within a project or program to communicate an approach, demonstrate an ability to meet requirements, or establish status.

♦  Technical reviews often serve as control gates for management to make a go/no-go decision on a project. •  Control gates involve formal examination of a project’s status in

order to obtain approval to proceed.

♦  Technical reviews may be held for subsystems too; depending on their size and complexity. Subsystem reviews precede their corresponding system review.

♦  When do technical reviews occur? •  Throughout the entire life cycle of a project, as shown on the

following life cycle chart.


Reviews in a NASA Project’s Life Cycle


The Value of Technical Reviews

♦  Technical reviews are key development milestones used to measure progress, assess project maturity and to infuse lessons from the past. They… •  Provide confirmation of completed effort and readiness to commit

additional resources for the next phase. •  Encourage and establish project discipline with an integrated project

team perspective. •  Identify risks and review mitigation options. •  Describe plans and priorities for the next phase.

♦  Technical reviews give everyone (design team, non-advocate discipline experts, customer and consumer) the opportunity to agree on the current project baseline (requirements, interfaces, allocations, margins, schedules, risks, budgets, etc.). •  Functional, resource and performance allocations and assumptions

are described and confirmed. •  System development constraints and interfaces are described and

agreed to by both sides. •  Risks and development problems are identified and mitigation options

discussed.


More Values of Technical Reviews

♦  A significant, and perhaps unexpected, value to the development team is the preparation for a review. Usually everyone is busy in their own domain - preparing for a review forces the team to describe what they have done and understand what others are doing.

While there is great value in the preparation, there is also unique value in the execution. One Air Force officer once took Dwight D. Eisenhower’s famous quote:

“Plans are nothing. Planning is everything.”

too seriously and on the day before it was scheduled cancelled the PDR of a $100 million space mission. After a minor revolt, the PDR was reinstated and the team benefited from the preparation and the review.

♦  Periodic project reviews are held to demonstrate that the appropriate products, accomplishments and plans have been completed before proceeding to the next phase.

Appropriate products, accomplishments and plans are based on the lessons of hundreds of past projects (best practices). NASA, for example, describes standard entrance and success criteria for all standard milestone reviews.


NASA’s Minimum Set of Technical Reviews

♦  Mission Concept Review (MCR), ♦  System Requirements Review (SRR) and/or Mission Definition

Review (MDR), ♦  Preliminary Design Review (PDR), ♦  Critical Design Review (CDR), ♦  System Integration Review (SIR), ♦  Test Readiness Review (TRR), ♦  Operational Readiness Review (ORR), ♦  Flight Readiness Review (FRR), ♦  Post-Launch Assessment Review (PLAR), ♦  Critical Event Readiness Review (CERR), and ♦  Decommissioning Review (DR).

While this may seem like a large number of technical reviews, each has its own focus and proven value.

In addition, since mission lifetimes are several years, typically there is 6 months or more between major reviews.


The Primary Questions of Each Review

♦  Mission Concept Review - Does the proposed concept meet the mission need and objectives?

♦  System Requirements Review and/or Mission Definition Review - Do the functional and performance requirements and the selected concept satisfy the mission?

♦  Preliminary Design Review - Does the preliminary design meet all the system requirements within acceptable cost, schedule, and risk?

♦  Critical Design Review - Is the system design mature enough to proceed with full-scale fabrication, assembly, integration and test?

♦  System Integration Review - Are the system, facilities, personnel, plans and procedures ready for system integration?

♦  Test Readiness Review - Is the project ready to commence with verification testing?

♦  Operational Readiness Review - Are all systems hardware, software, personnel, and procedures in place to support operations?

♦  Flight Readiness Review - Is the system ready for launch? Are the ground facilities and personnel ready to support launch?

♦  Post-Launch Assessment Review - After launch and deployment, are the spacecraft systems ready to proceed with routine operations?

♦  Critical Event Readiness Review - Is the project ready to execute the mission’s critical activities during flight operation?

♦  Decommissioning Review - Is the system ready to be terminated?


Pre-Phase A Reviews

Mission Concept Review (MCR) ♦  A validation that the mission has clearly established needs, objectives,

and top-level functional/performance requirements, and that at least one way of conducting the proposed mission is realistic and attainable within existing or projected technology and ROM cost.

♦  An internal Center review(s) of all Pre-Phase A activities and products should be conducted prior to forwarding the Pre-Phase A report to Headquarters. Technical, management, resources, and scientific personnel should conduct the review.

Peer Reviews ♦  Used informally during Pre-Phase A and Phase A. ♦  The peer group is composed of individuals selected from outside the

project according to their expertise in the applicable disciplines. ♦  Throughout Pre-Phase A and Phase A, peer reviews should informally

check the evolving mission concept against objectives, requirements, and constraints.

♦  Peer reviews help you take advantage of other engineering experience from colleagues who have worked on different missions. They can point out issues they confronted that may be similar for your mission concept.


Phase A Reviews

System Requirements Review (SRR) ♦  The primary focus of the SRR is to verify the realism of the functional

and performance requirements, ensure their congruence with the mission and system configuration, and ensure the mission objectives can be satisfied.

♦  The SRR encompasses all major participants (NASA and contractors), and is chaired by the Project Manager.

♦  A product from the SRR is the project system specification that is formally baselined and placed under configuration management control.

Mission Definition Review (MDR) ♦  A validation that the mission objectives can be satisfied, the partitioning

of the functionality to each of the systems is adequate, the top-level performance requirements for each system have been defined, and the technology required to develop the systems and implement the mission is attainable.

♦  The MDR is keyed to the end of Phase A and evaluates the mission definition, system design, operational concepts, schedule, and cost estimates.


1. Successful completion of the Mission Concept Review (MCR) and responses made to all MCR Requests for Actions (RFAs).

2. A preliminary SRR and/or MDR agenda, success criteria, and charge to the board have been agreed to by the technical team, project manager, and review chair prior to the SRR and/or MDR.

3. The following technical products for hardware and software system elements are available to the cognizant participants prior to the review:

a. System Architecture. b. System requirements document. c. System software functionality description. d. Updated concept of operations. e. Updated mission requirements, if applicable. f. Baselined SEMP (Systems Engineering Management Plan). g. Preliminary system requirements allocation to the next lower level system. h. Updated cost estimate. i. Technology Development Maturity Assessment Plan. j. Preferred system solution definition including major trades and options. k. Updated risk assessment and mitigations. l. Updated cost and schedule data. m. Logistics documentation (preliminary maintenance plan, etc.). n. Preliminary human rating plan, if applicable. o. Software Development Plan (SDP). p. System safety and mission assurance plan. q. Configuration management plan. r. Project management plan. s. Initial document tree. t. Verification and validation approach. u. Preliminary hazard analysis (PHA).

Example Entrance Criteria — System Requirements Review


Example Success Criteria — System Requirements Review

1.  The resulting overall concept is reasonable, feasible, complete, responsive to the mission requirements, and is consistent with system requirements and available resources (cost, schedule, mass power, etc.).

2.  The project utilizes a sound process for the allocation and control of requirements throughout all levels, and a plan has been defined to complete the definition activity within schedule constraints.

3.  Requirements definition is complete with respect to top level mission and science requirements, and interfaces with external entities and between major internal elements have been defined.

4.  Requirements allocation and flow down of key driving requirements have been defined down to subsystems.

5.  System and subsystem design approaches and operational concepts exist and are consistent with the requirements set.

6.  The requirements, design approaches, and conceptual design will fulfill the mission needs within the estimated costs.

7.  Preliminary approaches have been determined for how requirements will be verified and validated down to the subsystem level

8.  Major risks have been identified, and viable mitigation strategies have been defined.


Constellation Program Office Schedule of SRRs

March April 2007

MO SRR

GO SRR

EVA SRR

CEV SRR

CLV SRR

May Issue ID, Form Entry 3/26

3/26 Issue ID, Form Entry

Issue ID, Form Entry 4/9



Rev. Issue Table, POC, & Sched. 3/29

4/6 Rev. Issue Table, POC, & Sched.




Provide Status on Issue Resolution (every Mon. @ Cx SE&I 8:30 telecon) 4/6 4/9 4/16 4/23 4/30

Provide Status on Issue Resolution (every Mon. @ Cx SE&I 8:30 telecon) 4/9 4/16 4/23 4/30

(send updates to Cx SE&I COB every other day in May)

Provide Status on Issue Resolution (send updates to Cx SE&I COB every other day in May)

Provide Status on Issue Resolution (send updates to Cx SE&I COB every other day in May)



5/10 Provide Issue list w/ Resolution Plan &

Impacts


Impacts


Impacts

Provide Issue list w/ Resolution Plan &

Impacts 5/10

Provide Issue list w/ Resolution Plan &

Impacts 5/10

5/14 Develop PBS Presentation





PBS 5/23

Provide Status on Issue Resolution (every Mon. @ Cx SE&I 8:30 telecon) 4/23 4/30 4/16

You are Here!


Constellation Program (CxP) Office Status – Closure of SRR

Board AI (Action Items) ♦ Total AI Count: 48 Total ♦ 9 Closed ♦ 39 Open

•  30 are “Past Due” •  9 are in work, “Not Due Yet”

RIDs (Review Item Discrepancies) ♦ Total RID Count: 6,283 ♦ 6225 Closed ♦ 58 Open

•  Majority closed prior to PBS •  All RIDs should be closed prior to CxP SDR

Document Document Title Being Worked

CxP 70008 Master Integration and Verification Plan (MIVP) 4CxP 70023 CxP Prg Design Spec Natural Environments (DSNE) 9CxP 70024 CxP HSIR 1CxP 70036 Cx Environ Qual & Accept Testing Reqs (CEQATR) 5CxP 70050 Vol 2 Electrical Power System Spec Vol 2 1CxP 70061 C3I Strategic Plan 1CxP 70067 Vol 1 CxP Program Human-rating Plan Vol 1 19CxP 70080 Electromagnetic Environmental Effects (E3) Req'ts 6CxP 70086 Software Verification and Validiation Plan 5CxP 70135 Structural Design & Verification Requirements 7Total 58

TBD/TBR ♦ Total Count: 2,532

•  Received plans for 2,064 TBRs/TBDs in 72 of 95 documents (76% of docs.)

•  Need burn-down plans for 196 TBRs/TBDs in 16 documents (17% of docs.)

•  7 documents (274 TBRs/TBDs) will not be updated/baselined until post CxP PBS. These TBRs/TBDs will not be closed prior to CxP PBS


Preliminary Design Review (PDR)

♦  The PDR is not a single review but a number of reviews starting with the specific component PDRs, followed by the system-level review.

♦  The PDR establishes the “design-to” baseline and ensures that it meets the program, project , system, subsystem, or specific component baseline requirements.

♦  The PDR process should: •  Establish the ability of the selected design approach to meet the

technical requirements (i.e., Verifiability/ Traceability); •  Establish the compatibility of the interface relationships of the

specific end item with other interfacing items; •  Establish producibility of the selected design; •  Establish the operability of the selected design; •  Assess compliance with reliability and system safety requirements; •  Establish the feasibility of the approach; •  Address status, schedule and cost relationships.


When is a Review Complete?

Reviews are considered complete when the following is accomplished:

a)  Agreement exists for the disposition of all Review Item Discrepancies (RID) and Request for Actions (RFA).

b)  The review board report and minutes are complete and distributed. c)  Agreement exists on a plan to address the issues and concerns in the

review board’s report. d)  Agreement exists on a plan for addressing the actions identified out of

the review. e)  Liens against the review results are closed, or an adequate and timely

plan exists for their closure. f)  Differences of opinion between the project under review and the review

board(s) have been resolved, or a timely plan exists to resolve the issues.

g)  A report is given by the review board chairperson to the appropriate management and governing program management committees charged with oversight of the project.

h)  Appropriate procedures and controls are instituted to ensure that all actions from reviews are followed and verified through implementation to closure.


Example Agenda for a Project System Design Review by a Standing Review Board (SRB)

♦  Purpose of Review & Charge to SRB SRB Chair ♦  Project Overview & Status Project Manager ♦  System Engineering & Status Project SE

•  Requirements & V&V plans •  Trade studies •  Technical margins

♦  WBS-level 2 Design State & Status for each area WBS Managers •  System Design •  Key Requirements •  Trade Studies •  Technology Readiness •  Acquisition Strategy & Long Lead •  Logistics & Facilities •  Challenges & Risks

♦  Integrated System (e.g., power) State & Status for each area Discipline Leads ♦  Integration & Test Integration Manager ♦  Safety & Mission Assurance (S&MA) S&MA Manager ♦  Human Rating Project HR Rep ♦  Risk Risk Manager ♦  Schedule Project Planner ♦  Cost Cost Manager


Pause and Learn Opportunity

Student role-play: You are the chief systems engineer for a New Frontiers-class

mission to Europa. Your PDR is scheduled in 3 months. ü What do you do? ü What do you ask the development team to do? ü What benefits would you expect from having a PDR? ü How might it waste time?

Or Read and discuss the Crosslink article: The Role of

Independent Assessments for Mission Readiness (Crosslink_Independent Review.pdf)


Module Summary: Technical Reviews

♦  Technical reviews are key development milestones used to measure progress, assess project maturity and to infuse lessons from the past. They… •  Provide confirmation of completed effort and readiness to commit

additional resources for the next phase. •  Encourage and establish project discipline with an integrated

project team perspective. •  Identify risks and review mitigation options. •  Describe plans and priorities for the next phase.

♦  There are 11 reviews in the minimum set of technical reviews for a NASA robotic mission. These reviews cover the entire mission life — assessing the concepts and designs early; readiness for test, flight and operations in mid-life and plans for disposal at the mission’s end.

♦  These reviews are held to demonstrate that the appropriate products, accomplishments and plans have been completed before proceeding to the next phase. •  Appropriate products, accomplishments and plans are based on the

lessons of hundreds of past projects (best practices).


Backup Slides for Technical Review Module


Major Project Reviews Precede Each Key Decision Point

Key Decision Points

FORMULATION IMPLEMENTATION

Major Reviews

A C

D E

Project Phases

Concept Studies

Concept & Technology Development

Preliminary Design & Technology Completion

Final Design & Fabrication

System Assembly, Test, & Launch

Closeout Operations & Sustainment

A B

B

C

F

D E F Pre-A

Mission Concept Review

Systems Requirements Review

Mission/System Definition Review

Critical Design Review

Systems Integration Review

Operational Readiness Review

Flight Readiness Review Post Launch Assessment Review

Decommissioning

Review

Preliminary Design Review

Independent Cost Estimates


Purpose of Technical Reviews

A technical review is an evaluation of the project, or element thereof, by a knowledgeable group for the purposes of:

a)  Assessing the status of and progress toward accomplishing the planned activities.

b)  Validating the technical tradeoffs explored and design solutions proposed.

c)  Identifying technical weaknesses or marginal design and potential problems (risks) and recommending improvements and corrective actions.

d)  Making judgments on the activities’ readiness for the follow-on events, including additional future evaluation milestones to improve the likelihood of a successful outcome.

e)  Making assessments and recommendations to the project team, Center, and Agency management.

f)  Providing a historical record that can be referenced of decisions that were made during these formal reviews.

g)  Assessing the technical risk status and current risk profile.


Need for Management Reviews

The progress between life-cycle phases is marked by key decision points (KDPs). At each KDP, management examines the maturity of the technical aspects of the project. For example, management examines whether the resources (staffing and funding) are sufficient for the planned technical effort, whether the technical maturity has evolved, what the technical and non-technical internal issues and risks are, or whether the stakeholder expectations have changed. If the technical and management aspects of the project are satisfactory, including the implementation of corrective actions, then the project can be approved to proceed to the next phase.


4/2470132

3/27MS SRR Board

4/1070087

4/3070009

5/1EVA SRR Board

4/3070022 Vol 1

4/24GO SRR Board

2/1570073 Vol 3

4/116 weeks

Prior to PBS

4/2670022 Vol 2

4/11Afford . Plan,

70070-ANX 05 Bk 2

4/970064

5/23Program Baseline Sync

4/2470133

3/1CEV SRR Board

4/2470141

5/8701305/1

70019Cx Nomen . Plan

TBD

3/2070135

4/270022 Vol 5 4/24

70080

4/270044

3/2070025 FAD

N/A

4/2470036

3/2370012

3/2370009Anx 1

5/970000

5/470024

5/770022 Vol 8

5/1570143

4/2570022 Vol 3

5/770022 Vol 4

4/2070138

5/370078

5/2570008

4/1970137

4/2370070 -ANX05 Bk3

4/1870016

4/1870086

4/1670136

5/970007TBD

4/1370013

4/970023

4/25700614/24

70145

4/1670085

4/12 70065 ,

70070 -ANX05 Bk 1

5/16 70050 Vol 1 , 70050 Vol 2

4 /2470144

3/2970151

2/2 6/172/4 2/11 2/18 2/25 3/4 3/11 3/18 3/25 4/1 4/8 4/15 4/22 4/29 5/6 5/13 5/20 5/27 6/3 6/10

SE&I/ T&V Change Requests submitted to Configuration Management Dates 4/3/07

Elapsed Time

Blue text indicates changes from last update

Submitted to CM

On plan and will be baselined prior to PBSPlan confirmed but cannot be baselined prior to PBS (CR submitted within 2 weeks of PBS) or requires IMS update. No plan confirmed

CxP Doc # TitleSubmit CR

to CM Comments Org Doc Owner Change Flag

70000 Constellation Architecure Requirements Document, Revision A 5/9/2007 Per update from PRIMO 2/26/07 PRIMO C. Adamek

70007 DRM & Operations Concept Document 5/9/2007 Submit date reflected in the IMS inputs for 3/30. PRIMO B. Teague70008 Master Integration & Verification Plan 5/25/2007 Date changed in 3/8 verison of IMS - changed color to yellow T&V R. Cox70009 System Integrated Analysis Plan 4/30/2007 Added. Per IMS Update of 3/14/07 ATA A. Zuniga

70009 Anx 1 System Integrated Analysis Plan, Annex 1 for IDAC-3 3/23/2007 Submit date reflected in the IMS inputs for 3/30. ATA A. Zuniga

70012 Constellation Reference Architecture Document for IDAC3 3/23/2007Updated per e-mail on 3/27. Expected to be reflected in the IMS inputs for 4/6. ATA D. McKissock

70013 Systems Engineering Management Plan 4/13/2007 Confirmed by James A. on 2/26 and Dave C. on 3/8. COS-T J. Afarin70016 Requirements Engineering Management Plan 4/18/2007 Updated per IMS updates for 3/23 PRIMO M. DiGiuseppe

70022 Vol 1 C3I Interoperability Standards Book 4/30/2007 Updated per IMS updates for 3/23 CSI K. Muery70022 Vol 2 CSI Spectrum and Channel Plan 4/26/2007 Updated with CSI input rec'd 3/19 CSI C. Sham70022 Vol 3 CSI Master Link Book 4/25/2007 Updated with CSI input rec'd 3/19 CSI J. Brase70022 Vol 4 CSI Information Representation Spec 5/7/2007 Updated per IMS updates for 3/23 CSI J. Differding70022 Vol 5 CSI Data Exchange Protocol Specification 4/2/2007 Updated with CSI input rec'd 3/19 CSI M. Stagnaro70022 Vol 8 Common Command & Control Requirements 5/7/2007 Updated with CSI input rec'd 3/19 CSI M. Severence

70023Constellation Program Design Specification for Natural Environments (DSNE) 4/9/2007 Updated per 3/19 IMS inputs E&C K. Hwang

70024 Constellation Human Systems Integration Requirements 5/4/2007IMS input on 3/9 shows CR drop 5/4 (but this is prior to all updates in). Will confirm before updating here. HFSIG K. Ess

70025 Functional Analysis Document N/AFAD through RWG only, no board baseline required. B/L date in schedule of 6/27 is currently out of sync with PBS need dates PRIMO J. Williams-Byrd

70036Constellation Program Environmental Qualifications and Acceptance Testing Requirements (CEQATR). 4/24/2007 Update from Ed Strong, 2/21 T&V E. Strong

70044Constellation Program Natural Environment Definition for Design (NEDD) 4/2/2007 Updated per 3/19 IMS inputs which confirmed date E&C L. Smith

70050 Vol 1Electrical Power System Specification, Volume 1: Electrical Power Quality Performance for 28 VDC 5/16/2007

Schedule update received 3/9. Doc will be submitted but not baselined in time for PBS Power R. Scheidegger

70050 Vol 2Electrical Power System Specification, Volume 2: User Electrical Power Quality Performance for 28 VDC 5/16/2007

Schedule update received 3/9. Doc will be submitted but not baselined in time for PBS Power R. Scheidegger

70061 C3I Strategic Plan 4/25/2007Updated per e-mail on 3/27. This will need to be updated in the IMS inputs for 3/30. (was 3/30) CSI P. Paulsen

70064 Supportability Plan 4/9/2007 Updated per IMS inputs from 3/30 (was 3/23/07) SOA K. Watson

!Slipped

Date post:	04-Nov-2020
Category:	Documents
Upload:	others
View:	1 times
Download:	0 times

Technical Reviews Moduleorigins.sese.asu.edu/ses405/Class Notes/Technical... · Exploration Systems...

Documents