Charles.armstrong

OrionPDR Process -

What in the World Happened?

1

Charles ArmstrongNational Aeronautics and Space AdministrationOrion VI&D Plans & Processes [email protected]

Tony ByersLockheed Martin Space Systems CompanySystems Engineering – Program Integration [email protected]

NASA PM Challenge 2010

Used with Permission

mailto:[email protected]�

mailto:[email protected]�

Project OrionProject Orion

Agenda

2

• Orion PDR Management Challenges• Planning• Execution

• Tenants of Operational Excellence• PDR Process• PDR Results• Key Lessons Learned• Tentative CDR Plan• Closing Remarks


Management Challenges

3

• Planning Challenges• Develop and execute a Preliminary Design Review (PDR) process

that ensures NASA’s next generation spacecraft can meet all mission objectives and satisfy requirements with acceptable risk

• Develop a multi-tiered approach that ensures the integrated subsystems, modules and vehicle are reviewed to ensure the design approach is at a sufficient level of maturity to proceed to the detailed design phase.

• Balance thoroughness and schedule. The desire to review every aspect of the design down to the lowest possible level must be balanced against the available time to execute the review as to not delay the development activities unnecessarily.

• Balance inclusion and efficiency. Develop a process that ensures all stakeholders have an adequate time to participate and provide input helping to shape the design, ensure mission success, and ultimately provide a safe crew exploration vehicle.


Management Challenges

4

• Execution Challenges• Communicate the expectations to a diverse 2000+ personnel

organization consisting of NASA, prime contractor (LM), and many support contractors

• Balance the desire to improve the design and need for a design baseline. From the time the PDR baseline is struck, a lot of potential energy develops to improve the requirements, architecture, design, and configuration. This energy must be managed to ensure that PDR products align as close as possible to the PDR baseline.

• Monitor and control process execution throughout the duration of the effort


Operational Excellence

• Teamwork– Created a joint NASA/LM PDR Integration Team (PIT) to ensure common expectations are

communicated across the NASA, LM, and support contractor teams• Ensured key personnel from each Module and IPT were included on team• Enlisted the support of the Orion planning organizations

• Communication– Focused on communicating early and often– Conducted a series of TIMs with each of the subsystem leads to develop common

expectations and identify issues needing resolution prior to PDR– Instituted multiple statuses and forums targeting all stakeholders

• Commitment– With the goal of “Get it right the 1st time”, the Orion Management team routinely stressed

the importance of attaining the PDR level of design maturity– Management routinely communicated the PDR expectations and monitored status weekly–

• Accountability– Each Subsystem Lead took ownership of their respective Subsystem Design Review

preparation and execution– A detailed schedule was developed to manage the program DRD review and delivery– Detailed metrics were developed for tracking all deliverables by System and Subsystem


PDR Process

6


PDR Process & Data Package

• PDR Definition– A technically-focused design review which demonstrates that the preliminary system,

module and subsystem design of the Orion integrated vehicle, its concept of operations, associated processes and applicable technical plans meet all system requirements with acceptable risk and within cost and schedule constraints.

• PDR Process– The PDR process included a series reviews which provide a solid foundation for

determining if the current design is at PDR level of maturity. • 184 Technical Reviews• 104 Peer Reviews• 10 Combined Tech/Peer Reviews• 18 Subsystem Design Reviews• 1 System and Module Review• 8 RID Teams responsible for RID Screening & Disposition• 1 PDR Pre-Board• 1 PDR Board

• PDR Data Package– 170 DRDs delivered and reviewed– 53 “Drafts Available” DRDs provided to NASA for comment


PDR Objectives

8

• Demonstrate that the Orion Project’s design baseline meets all Constellation Program and Orion system requirements to the subsystem level and does so with acceptable risk and within the cost and schedule constraints

• Ensure that all system requirements have been allocated to the component level, the requirements are complete, and flow down is adequate to verify system performance

• Ensure all physical and functional interfaces for Orion system and subsystem design have been identified and defined

• Demonstrate sufficient definition of integration and verification activities to allow the Project to proceed with detailed planning of test, integration, and the manufacturing of test articles

• Confirm that verification method requirements have been developed to the subsystem level and are consistent with the requirements and the vehicle design

• Demonstrate that the PDR vehicle configuration is a closed vehicle and within all specified margins

• Demonstrate that the vehicle hazards and hazard control methods are identified and assessed, including safety features of the design and crew survival features of the vehicle


PDR Objectives

9

• Establish the preliminary design, thus enabling the Project to baseline subsystem specifications

• Ensure that technical and programmatic risks are identified, are categorized by criticality, and that risk mitigation plans are approved and are being managed

• Ensure that the Project is meeting cost and schedule baselines and has the infrastructure necessary to support management and reporting of cost and schedule data

• Show appropriate maturity in the proposed design approach to proceed to critical design

• Establish the basis and a viable path for proceeding to the Critical Design Review (CDR) including interim milestones


PDR Entry & Success Criteria

10

• Performed a detailed analysis and mapping of Entrance and Success Criteria• Orion is required to meet both NASA NPR 7123.1A and LM 2.1.6-T1-PgmMgt-1.2-P-

C3 PDR Entrance and Success/Exit Criteria• Evaluated both sets of entrance and exit (success) criteria• Integrated into an “Orion” set of PDR Entrance and Success Criteria

• Documented in CxP 72212 Orion PDR Process Plan• Mapped PDR Products/DRDs to criteria• Criteria reviewed by all NASA and LM Module/IPT Leads, Subsystem Leads, and

key others prior to formal documentation in CxP 72212• Criteria reviewed and approved by LM SSC Central Systems Engineering

• NASA & LM IPTs developed a detailed self assessment against the entry and success criteria

• Allowed team to better resolve issues prior to the PDR Board• Assisted program management in documenting open issues needing to be resolved

post PDR

• First evaluation of PDR Entry and Success Criteria occurred at the PDR Pre-Board• Final evaluation of the Success Criteria occured at the PDR Board


CxP 72212 Orion PDR Plan

11

• Comprehensive PDR plan documented in CxP 72212 and placed under configuration control

• All significant stakeholders participated in review of document prior to release

• Formalized the overall PDR process

• Documented the requirements and associated documentation baseline for PDR

• Incorporated tailored NPR 7123.1A criteria into plan

• Served as the primary method of documenting all agreements regarding method to meet the SSDR and PDR criteria

• Used as a reference throughout the execution to ensure the process was adhered to

12

2008 2009Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May June July Aug Sep

Project Orion

12

606E Closeout ERB

9/29,30& 10/2

Module MELs

Complete8/1

DAC-2

SBR

10/23 10/24

Test and Verification Activities

Integrated vehicle products on ProjectLink

1 week prior to ERB

10/29T-009, T-010 DAC-2

Analysis Reports Due to NASA

Avionics & Software Activities

S/C & Module Spec (DAC-2) 12/5

Safety & Mission AssuranceCSERP Groundrules TIM 10/1

Spiral Data Pkg Delivery

11/20Cradle Snapshot 8/15

8/15

Struct, Mech, Pyro, & TPS MEL Update

LOC/LOM ERB

C&T SRR 10/29C&T SS Review 12/4-5 Spiral

Review 12/7-11

Spiral Data Rev

Board 1/22/10

6-wk Asmts &DAC/PDR Planing DAC-3

DAC-3 POD ERB 11/19-209/9

Sys Integ Sys Integ

SWELs Complete

5/15

CloseoutERB 606G6/11&12

MidtermERB 606F

2/12-13

Integrated vehicleproducts available

Subsystem Design Reviews

S/C & Module Spec Development

Subsystem IRD & ICD Development

Master Verification Plan DevelopmentSubsystem Design and Databook Development

Tech Reviews & Peer Reviews

Sys Product Develop

T-008 SAPDelivered12/22

Subsystem Spec Development

11/3-7 Ph 1 TIM #1Ph 1 TIM #2 1/20-23 2/9-13 Ph 1

TIM #3

2 wk Sys & Mod Presentations

7/24

7/7Final PDR DRDs delivered to NASA

7/8T-009, T-010 DAC-3 Analysis Reports Due to NASA

Sizing / Subsystem FEMs LA4 FEM Develop

Ares LA4 Loads Analysis

Orion Uncoupled

Ares LC4 Loads AnalysisAres LC4 Results Final Drop 12/20

Orion LC-4 R&USzg

DAC-3 Initialization Loads (DAC-2 Loads + 6 wk Update for Landing, & Sec Environments)

Loads Update (Test Data Updates)

PDR LoadsStatus

Orion Uncoupled

Lunar Data Drop 8/14

SMR

1-wk DRDDelivery Margin

S/C, Mod, & AV&SW Spec(C&T & Mid-DAC-3) – 5/5

Integrated vehicleproducts available

Orion Uncoupled

Road to August 2009 PDR

LC5 FEM DevelopSizing / SS FEMOrion Rev & Update

Orion Uncoupled

KO7/13

2/28 Sys & Subsystems Cradle Snapshot

8/31All RIDs Dispositioned

DRD Rev & RID

2/24-27GN&C SSDR

Drawing Release

PDR Board8/31

Pre-Board

Loads Update (LC-4, Landing)

Update

Update

LoadsERB4/14

1/23 1/30

606F IntegratedProducts

3/24

3/30-4/3 Ph 1 TIM #4

5/4-8 Ph 1 TIM #5

6/15-26 Ph 1 TIM #67/27-8/7 CSERP

DeltaReview

AA-2 7/16PTR2

PA1 DD250 3/3EGSE DD250 2/11

4/10 T&V Sys B/L RevCAIL PR 6/30

RemainingData Dropped8/31 (TBR)

Will be updated after Orion/Ares Coordination


June July August Sep Oct

DRD Update & Redeliver(Type 1 + Final Type 2)

DRD Waterfall / Early Review(Tech Reviews & Peer Reviews)

Orion PDR ProcessProject Orion

SMR

PDR KO7/13 7/21

8/10 NASA ScreeningTeam Concludes RID Screening

All RIDsDispositioned8/20

4 wk DRD Review(RID Creation &

Sponsor)

System &Module

Presentations

Decision to Proceed to DAC-4

RIDs Captured Against Design

DRDs Delivered7/7 PDR -45

NASAScreening

TeamNASA Lvl III Review & Filter RIDs

(Work disconnects with external stakeholders)(LM SEIT led SME team available to answer questions)

RID Tool OpenTBD

RID Tool Close (all RIDs Sponsored)7/31

DRDsRedeliveredNLT 2/1/10Pre

Board5 1/2 Day Pre-Board to work issues in parallel

with completion of RID Disposition

Orion PDR

All RIDs receive product owner or lead recommendedDisposition8/17

Starts 8/3

Starts8/12

Product OwnerRecommendDisposition

Starts 8/17

8/4 BoardDecision Gate

8/10 ForwardDaily Board Decision Gates

S

IntegTeam

PIT Lead with Module/IPT SupportStarts 8/27

PIT War Room

8/31-9/1 PDR Board

Review TeamDisposition

Dry-Runs

Subsystem DesignReviews (SSDRs)

2-WkSSDRs

SMR Prep


Monday Tuesday Wednesday Thursday Friday1 2 3 4 5

8 9 10 11 12

15 16 17 18 19

22 23 24 25 26

29 30 July 1 2 3 - Holiday

June 2009Project Orion

Orion Subsystem Design Reviews

Environmental Control and Life Support SSDR

Structures SSDR

Landing and Recovery System SSDR

LAS Abort Motor SSDR

DAC-3 Closeout ERB

Avionics and Software SSDR

LAS Attitude Control Motor SSDR

Electrical Power System SSDR & Voltage/Distributed Architecture Updates

LAS Jettison Motor SSDR

Wiring SSDR

Passive Thermal Control SSDR

Mechanisms SSDR

Propulsion SSDR

Thermal Protection System SSDR

Pyrotechnics SSDR

Crew Systems SSDR

2 w

k A

ppro

ach

Sacramento, CA

Promontory, Utah

Elkton, MD

Houston

Denver

Denver

Denver

Denver

Denver

Denver

Denver

Denver

Denver

Denver Denver

PDU ReviewHouston

SCRAM ReviewHouston

Ends Sat 6/27

Structures SSDRIncludesSat 6/20

LAS focus 6/20 & 6/22 AM Overflow

SSDR KO/ROE (2 ea.)

SSDR KO/ROE (2 ea.)

Chairs ROE (Makeup)2:00 PM MT

Chairs ROE (Makeup)2:00 PM MT

Ends Sat 6/20

GN&C SSDR Conducted 2/24-27LAS SEIT, Avionics & Power SSDR Conducted 4/14-15


RID Criteria & Ground Rules

15

Criteria• A design-related issue intended to document:

• Inability of the preliminary design to meet requirements (e.g., design compliance issues)

• Incomplete or incorrect requirement flow down or traceability• Incomplete or inadequate maturity for PDR (e.g., incomplete/inadequate trades or

models)• Undefined or inadequately defined technical interfaces• Inconsistency between design, analysis, requirements, or operations concepts• Incorrect or inadequate verification methods in support of the preliminary design• Inconsistency with the primary characteristics that should factor into each design

decision as identified in CxP 70000, Constellation Architecture Requirements Document, Section 3.1.2. These primary characteristics are: Safety and Mission Success, Programmatic Risk, Extensibility and Flexibility, and Life Cycle Cost

• Unsafe design features• Unidentified or inadequately mitigated risk

• Inconsistency with Project objectives



16

Ground Rules• RIDs can be written against RIDable documents or against any aspect of the Orion

vehicle design and technical plans within the purview of the Orion Project• RIDs will be recommended for withdrawal if the initiator is identifying a deficiency that

is out of scope of this review or does not meet the RID criteria listed above• An assertion via RID that a design feature is inconsistent or deficient with respect to

the primary characteristics expressed in the Orion Requirements Baseline should be supported by rationale that provides a credible case for the existence of a more favorable alternative

• Where possible, RIDs should be tied to functional or performance requirements or constraints

• RIDs will also be recommended for withdrawal if the requestor is identifying an area of “forward work”

• “Forward work” is defined as work that is necessary for successful completion of the Orion vehicle but not required to satisfy the success criteria for PDR

• Lack of sufficient information is a basis for a RID only if the RID initiator has exhausted all reasonable means to obtain the information or if the information has not been made available as required by the entrance criteria



17

Ground Rules• Editorial RIDs will be demoted to comments and forwarded to the document

developer as information at the end of the review• Initiators are expected to review the errata sheet associated with a specific DRD prior

to reviewing the DRD and/or prior to entering a RID• RIDs are intended to identify design issues and are not intended to document known

issues as documented in the errata• RIDs may be written if the errata description or details are insufficient to provide

confidence that the issue has been addressed or will be addressed in the appropriate manner and timeframe


What is a Good RID?

• What is a good RID?– A “new” design related issue– Accurately describes the design related issue consistent with the RID criteria– Provides the details of where the issue was found

• For DRD related RIDs (section, page, etc)• For Architecture Element RIDs with no DRD referenced (chart, conversation, etc.)

– Provides a sufficiently detailed recommended resolution– Stays within the scope of PDR expectations– For requirements, provides correct “from” language and recommends “to” language

• What is a bad RID?– Incomplete issue identification– No recommended resolution– Recommended resolution of “fix this”– Editorial comment (spelling, grammar, etc.)– RID that expands on errata or duplicates a Pre-Declared RID– Post PDR level of maturity / unrealistic expectations– RID written against a baselined or reference document (Examples: SRD, S/C Spec)– RID entered by someone other than the original initiator without documenting the original

initiator– RID written against a previously made project decision (VICB, CPCB, etc.) without new data

or evidence to challenge the decision

Don’t tell us something we already know!!!

An issue that has not previously been identified.

Project Orion

19

PDR Results

19

Project Orion

PDR Assessment

• Design Issues– A detailed pre-assessment was performed by each of the module prior to entering PDR– All known issues were presented at each SSDR and the SMR

• No new significant issues surfaced during the reviews– Significant focus was applied to ensure known issues had agreed-to plans to proceed– Top issues contributing to non-green assessments

• Mechanisms – FBC R&R – FBC Recontact, CM/SM R&R Maturity• LRS – FBC Recontact / Parachute Architecture & Testability• TPS & Structures – Parachute Architecture Impacts to Baseline• EPS – 120 V Incorporation• AV&SW – C&T Incorporation, BFCS Maturity• ECLS – ARS Suit Loop, Procurement Schedule• CM Prop – Procurement Schedule• Pyro – Devices Dependent on Mechanism Maturity• LAS ACM – Production Motor Maturity

• 18 Subsystem Design Reviews (SSDRs) executed successfully– Each SSDR assessed against a common set of entry and success criteria plus a joint set of NASA/LM agreed to objective

evidence customized for each Subsystem– 59 Total Review Days & approximately 500 review participants

• PDR executed successfully– Each module assessed against a common set of entry and success criteria; ITAs assessment integrated – 56 days of DRD review, 7 day System & Module review, 1 week pre-Board and a Board– Approximately 1200 reviewers

Yellow / Green• Project Management assessed PDR a success and authorized the program to proceed

to the detailed design phase• Assessment based on a Red, Yellow, Green scale

Parachute Architecture Study was underway prior to entering PDR. Assessment concluded in October 2009.

Driven by significant post SDR requirements changes

Procurement schedules were being updated to align with project funding profile

Project Orion

PDR Data Package• NASA/LM jointly assessed PDR

deliverables to determine which DRDs were necessary to meet PDR expectations and satisfy criteria

• PDR –• 146 Deliverable Documents

• SSDR & PDR –• 26 Design & Data Books

• Data Package Total• 172 Total PDR Documents• 53 “Drafts Available” DRDs

provided to NASA for comment

Waterfalled Review and Delivery• Managed via the PDR Integration

Team• Captured in a 2000+ line MS

Project Schedule• Metrics generated & reported on

a weekly basis• Deliveries culminated on 7/7/09,

PDR-45 Days

PDR Data Package Delivery

As of July 8, 2009

PDR -45 days

Project Orion

Action Item & RID Closure

PDR RIDsLAS 150

CM 241

SM 90

Av&Pwr&Wire 444

Software 38

T&V, Fac, GSE 411

AI&P 18

SR&QA 91

Systems 566

Total 2049

SSDR ActionsAV&SW 140

Crew Systems 39

ECLSS 49

EPS 97

GN&C 26

LAS AM 60

LAS ACM 96

LAS JM 60

LAS SE&I AVP 12

LRS 42

Mechanisms 56

Propulsion 150

PTC 33

Pyro 26

Structures 141

TPS 20

Wiring 14

Total 1061

• Detailed Action and RID screening occurred throughout the SSDR & PDR process

• All actions and RIDs dispositioned and closure aligned with project post PDR work plan

• All RIDs approved by PDR Board

• Project approved closure plan implemented

• Approximately 100,000 pages of design, process, and plan documentation reviewed

• Included requirements, interface definition, verification details, general plans, etc.

Project Orion

23

Key LessonsLearned

23

Project Orion

Key PDR Lessons Learned

• SSDRs– SSDRs executed in parallel and in a common location worked well

• Key personnel were in the building and available and could easily bounce between reviews when needed

– Focused review teams allowed the reviews to move at a sufficient pace while ensuring all significant information was reviewed to an adequate level

• On-site review teams of 50 to 100 personnel is appropriate• Overflow rooms and remote participation were not an ideal way of participating in the on-site review

– Data Packages consisted of the core information necessary to evaluate design maturity• Minimizing the data package to only those DRDs necessary to establish design maturity supported a

streamlined review process– Early technical and peer reviews facilitated improved design awareness prior to the review– Need to strengthen Software integration into SSDRs

• Evaluate what additional products need to be added to the SSDR Data Packages• Note software was addressed to level required for NPR 7123.1a (project wants additional focus for CDR)

– Need to ensure adequate spacing between tech reviews and SSDRs to allow for sufficient time to close TR actions

• Need SSDR co-chairs to review open TR actions prior to SSDRs– Transferring SSDR actions to RIDs created a dual paper trail that required unnecessary effort to resolve– SSDR Dry-runs need to be scheduled further in advance to allow subsystem team to make adjustments

prior to the review– SSDR Caucuses were most effective when kept to a small attendance– SSDR color ratings took more time than expected given that issues were identified ahead of the SSDR

Project Orion

• PDR– Data Package Preparation, Delivery, & Review

• Separate Technical and Peer Reviews were not the most efficient way to review DRDs– Peer Reviews were not viewed as where the detailed review occurs– Integrating Technical and Peer Reviews into a single review that meets the expectation of both

reviews• Decouple DRDs not needed for design review from the design review

– Review and accept these documents through NASA’s normal DRD acceptance process• Waterfalled delivery allows for early review of material prior to the face-to-face reviews

– System & Module Review (SMR)• SMR value did not match level of effort required to prepare and execute

– Most technical review and discussion occurred at the SSDRs– Weak in-person attendance

• Need to change this review to an integrated vehicle review focusing on the integrate module and integrated stack design for CDR

– Include Assembly, Integration & Production– Include Test & Verification– Pull the integrated analysis and design products and review them up-front prior to the SSDRs

• Create a one-day CDR Kick-off that gives the requirements baseline, environments overview, etc. prior to the CDR SSDRs


Project Orion

• PDR– RID Tool, ProjectLink, Portal, and Wiki

• Use one tool from the beginning of the process through completion– Minimizes duplication of issues and the overhead of transferring from one tool to another

• Test the RID tool earlier in the planning process• Choose a RID tool that allows for a detailed closure process tracking• Portals and Wiki were a valuable resource to all reviewer

– Assign a NASA IT person to the PIT for CDR• PDR hotline (outlook email box) served as a ongoing resource when reviewers did not know who to

contact regarding a process or technical issue– RID Screening and Review Teams

• IPTs who integrated the screening/review effort completed activities earlier and reduced iteration when dispositioning / resolving issues

• Improved direction regarding “Approve” and “Approve w/mod” needs to be developed and communicated to review teams prior to start

• Training needs to better prepare the reviewers for what to review the design against (Rqmts, Docs)– Consider adding an Orion familiarization to the training

– Pre-Board and Board• Overall Board execution proceeded to plan

– Increased Pre-Board time needed in CDR schedule if a similar number of RIDable documents are to be reviewed during CDR

• Ensure RIDs dispositioned by pre-board or board are updated in the tool needs to occur• Do not push issue decisions to later forums (VICB, CPCB, etc.). Make the decisions prior to leaving

the Board even if this extends the Board


Project Orion

27

Initial CDRTop-level Plan

27

Project Orion

Top-level CDR Plan

Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 Week 10 Week 11 Week 12

Orion CDR Kickoff

DRD Review

Interactive Reading Room (DRDs, Drawings, Etc.)

SSDRs

SSDR Caucuses

RID Disposition (Product Owners & Leads)

System Analysis Review (1 to 2 days)

RID Tool Opens

RID Review Teams

Pre-Board

Board

RID Tool Closes

Integrated Vehicle Design Review

DRAFT

Issue Identification (RID generation)

IVDR

DRAFT

Project Orion

Top-level CDR Plan

Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 Week 10 Week 11 Week 12

Orion CDR Kickoff

DRD Review

Interactive Reading Room (DRDs, Drawings, Etc.)

SSDRs

SSDR Caucuses

RID Disposition (Product Owners & Leads)

System Analysis Review (1 to 2 days)

RID Tool Opens

RID Review Teams

Pre-Board

Board

RID Tool Closes

Integrated Vehicle Design Review

DRAFT

Issue Identification (RID generation)

IVDR

• All reviewers participate in the Orion CDR Kickoff (Orion Familiarization) to educate reviewers on the:

• Design configuration• Requirements baseline• Environments baseline• Design Work products• Significant design issues and their closure plans

• Coordinate what metrics are needed with RID/Action Tool prior to CDR• RID closure will be handled within IDE• Errata and Pre-Declared RIDs all in one location (minimize number)• Caucus serves the function of RID Screening• Align tool with Process

• Don’t duplicate SSDR actions and RIDs. • Issues captured in standard format throughout process (Action/RID Tool)• Standardize process for handling cost/schedule issues• Ensure Action/RID template aligns with IDE tool • Summarize SSDR content that has been covered in component CDRs

• Replace SMR with an Integrated Vehicle Design Review• Cover integrated module, integrated stack, AI&P, T&V, etc.

Notes from Orion Streamlining TIM 12/09 DRAFT

Project Orion

30

Closing Remarks

30

Project Orion

PDR Supporting Documentation

• PDR Plan– CxP 72212 CxP Orion PDR Plan is available on PDR Wiki or PDR

Portal/Dashboard

• PDR Resources– PDR Wiki

• Used as a resource to all Orion personnel and PDR stakeholders to communicate the current planning status

• https://ice.exploration.nasa.gov/confluence/pages/viewpage.action?pageId=13828791&maxRecentlyUpdatedPageCount=20

– PDR Portal• Used to formally provide all SSDR and PDR details, data package, and review results• Federal record of review• https://ice.exploration.nasa.gov/ice/site/cx/reviews_orion_pdr/

https://ice.exploration.nasa.gov/confluence/pages/viewpage.action?pageId=13828791&maxRecentlyUpdatedPageCount=20�

https://ice.exploration.nasa.gov/confluence/pages/viewpage.action?pageId=13828791&maxRecentlyUpdatedPageCount=20�

https://ice.exploration.nasa.gov/ice/site/cx/reviews_orion_pdr/�

Project Orion

PDR Integration Team

In addition to the core PIT members above, there were members from each IPT, Module, Planning, and Administration incorporated into the planning effort and considered active PIT members.

Project Orion

Questions?

33


34

Back-up


Entrance Criteria

Orion Entrance Criteria (CxP 72212)1) Successful completion and close-out of SDR. All prior SDR Actions and RIDs have been closed, or approved closure plan exists for those remaining open.

2) PDR plan has been approved by the CPCB, which includes the PDR entrance and success criteria, RID/Action process, Board Members, stakeholders and schedule.

3) All PDR applicable and supporting products have been delivered and made accessible for stakeholders prior to review.

4) Updated baselined documentation, as required.

5) Preliminary design is established for each subsystem and supporting subsystem specifications are final with supporting analysis and trade-studies. Spacecraft, Module and Subsystem Specifications are complete and traceable. Subsystem specifications include Component definition. Component specifications delivery and review schedules have been developed and aligned with procurement strategy, including long-lead items. Preliminary software design and supporting subsystem specifications should include a complete definition of the software architecture and preliminary database design description (if applicable) which should include all analysis and data used for the development/maturation of the software architecture.

6) Technology development plans are updated.

7) The project risks and opportunities have been developed. Acceptable risk mitigation plans have been developed in accordance with the Orion Risk Management Plan.

8) Cost/schedule data are available to appropriate reviewers.

9) Updated logistics documentation, including initial support equipment list.


Entrance Criteria

Orion Entrance Criteria (CxP 72212)

10) Applicable technical plans (e.g., technical performance measurement plan, contamination control plan, parts management plan, environments control plan, EMI/EMC control plan, producibility/manufacturability program plan, reliability program plan, quality assurance plan).

11) Updated Applicable standards.

12) Integrated CEV flight safety analysis (including software safety analyses) and other applicable S&MA analysis of preliminary design (Failure Modes and Effect Analysis, flight system reliability and maintainability analysis, Probabilistic Risk Assessment, etc.) are complete and requirements are implemented in system, module, subsystem, and component specifications (component specifications are developed per the acquisition strategy). All associated plans have been developed.

13) Updated Engineering drawing tree.

14) Preliminary Functional and physical interfaces are developed and documented in the Interface Control Documents (ICDs).

15) Test and verification and validation plans are defined.

16) Plans to respond to regulatory requirements (e.g., Environmental Impact Statement, as required)

17) Disposal Plans defining safe disposal of assets.

18) Technical resource utilization estimates and performance margins are updated.

19) Preliminary limited life items list is updated (LLIL).

20) Operational planning is proceeding to schedule.


Success Criteria

Orion Success Criteria (CxP 72212)1) The Spacecraft, Module, and Subsystem Requirements, TPMs and any CxP-imposed requirements are agreed upon, finalized, stated clearly and are consistent with the preliminary design.2) The flow down of verifiable requirements to the subsystems and to the appropriate components are complete. An adequate plan exists for timely resolution of all open items. Requirements are traceable to Orion SRD, CxP IRDs and other applicable CxP requirements including any errata.3) System, module and subsystem preliminary designs are complete and meet requirements at an acceptable level of risk. All liens have been identified and closure plans developed.4) Technical interfaces are defined, deemed adequate, and are consistent with the overall technical maturity and provide an acceptable level of risk.5) Adequate spacecraft technical margins exist with respect to TPMs.6) Any required new technology has been developed to an adequate state of readiness or back-up options exist and are supported to make them a viable alternative.7) The project risks and opportunities are understood and have been credibly assessed. Appropriate risk mitigation plans for all top risks. 8) Safety and mission assurance (e.g., safety, reliability, maintainability, quality, and EEE parts) have been adequately addressed in preliminary designs and any applicable S&MA products (e.g., PRA, system safety analysis, and failure modes and effects analysis) have been reviewed and concurred.9) The operational concept is technically sound, is supported by the vehicle design, and includes (where appropriate) human factors. Appropriate requirements identified by the ConOps have been flowed down including requirements for ConOps execution.10) Preliminary test and verification plans are updated. Manufacturing plans are developed.11) System costs are within targets.


Product to Criteria Mapping

• Performed a detailed analysis and mapping of Entrance and Success Criteria• The PDR Data Package is mapped to the criteria and documented in Appendix C of

CxP 72212 Orion PDR Process Plan

Project Orion




Orion PDRrev 8/26/09 – 2200 MT

Project Orion

SMR

PDR KO7/13 7/21




Sponsor)

System &Module

Presentations




NASAScreening



RID Tool OpenTBD


DRDsRedelivered

NET 11/6

Redelivery date will be recommended on 8/21 based on number and complexity of

RIDs.

PreBoard

Dry-Runs

SMR PrepOrion PDR


Starts 8/3

Starts 8/12


Starts 8/17



S

IntegTeam


PIT War Room

8/31-9/1 PDR Board



2-WkSSDRs

5 1/2 Day Pre-Board to work issues in parallel with completion of RID Disposition

Subsystem Design Reviews (SSDRs)

18 Subsystem Reviews• 3 SSDRs Executed Prior to the 2-wk SSDRs

• GN&C SSDR Completed on 2/24-27• LAS SEIT, Avionics & Power SSDR Completed on 4/14-15• LAS Jettison Motor Completed on 6/8-10• Helped iron out the process and allowed the team to take advantage of

lessons learned• 13 SSDRs completed 2-wk set of SSDRs on 6/15-26

• At any given time 4 SSDRs were occurring in parallel in the Lockheed Martin Denver WB2 facility

• 2 remaining SSDRs completed the last week of June• SSDR Products

• 28 Design & Data Books Delivered & Reviewed• 23 Draft Subsystem Specs, 21 Subsystem IRDs, 21 Subsystem ICDs, and

12 Subsystem Volumes of MVPs Available for Comment• 59 Total Review Days• Over 1,000 review participants including support from NESC, SRB, NASA Crew

Office, NASA MOD, NASA Engineering, NASA S&MA, NASA Health & Medical, Constellation, Orion Project, & LM Central Engineering

• 1,056 Action Items Generated• All actions recorded in either the ConCERT RID Tool or LM IDE

Project Orion





Project Orion

SMR

PDR KO7/13 7/21




Sponsor)

System &Module

Presentations




NASAScreening



RID Tool OpenTBD


DRDsRedelivered

NET 11/6


RIDs.

PreBoard

Dry-Runs

SMR PrepOrion PDR


Starts 8/3

Starts 8/12


Starts 8/17



S

IntegTeam


PIT War Room

8/31-9/1 PDR Board



2-WkSSDRs


18 Subsystem Reviews• 3 SSDRs Executed Prior to the 2-wk SSDRs

• GN&C SSDR Completed on 2/24-27• LAS SEIT, Avionics & Power SSDR Completed on 4/14-15• LAS Jettison Motor Completed on 6/8-10• Helped iron out the process and allowed the team to take advantage of

lessons learned• 13 SSDRs completed 2-wk set of SSDRs on 6/15-26

• At any given time 4 SSDRs were occurring in parallel in the Lockheed Martin Denver WB2 facility

• 2 remaining SSDRs completed the last week of June• SSDR Products

• Design & Data Books Delivered & Reviewed• Draft Subsystem Specs, Subsystem IRDs, Subsystem ICDs, and Subsystem

Volumes of MVPs Available for Comment• Over 1,000 review participants including support from NESC, SRB, NASA Crew

Office, NASA MOD, NASA Engineering, NASA Orion, LM Orion, & LM Central Engineering

• _________ Action Items Generated• _____ Actions classified as “Pre-declared RIDs”• _____ Actions classified as “Complete Prior to PDR”• _____ Actions classified as “Forward Work”• All actions recorded in either the ConCERT RID Tool or LM IDE



Each SSDR assessed against a common set of entry and success criteria plus a joint set of NASA/LM agreed to objective evidence customized for each Subsystem (Assessed either Green, Yellow, or Red)

3. The subsystem risks and opportunities are understood and have been credibly assessed. Acceptable risk mitigation plans have been developed where applicable.

4. The preliminary subsystem design is reliable and safe for use. Reliability and safety analyses of the preliminary design have been performed and resulting requirements flowed down to the system, module, and subsystem. Preliminary plans for tests, analyses, demonstrations, and inspections will verify the reliability and safety of the subsystem. Safety, Reliability, and Quality Assurance (SR&QA) has been adequately addressed in preliminary designs, and applicable SR&QAproducts (e.g., PRA, system safety hazard analysis, and failure modes and effects analysis) satisfy PDR product maturation requirements (e.g. Phase 1 Safety Review criteria)

Mechanism

s

LAS A

MTPSEPS

LAS A

CM

Pyro

AV &

SWEC

LSStructures

SM Prop

CM

PropLR

SPTC

Wiring

Crew

Systems

LAS JM

LAS SEI, AVP

GN

&CSubsystem Criteria

1. Requirements are complete and flowed to the appropriate products. Subsystem specifications are updated.

2. Subsystem design is complete to a level of detail that is sufficient to demonstrate that all requirements are expected to be met at an acceptable level of risk. Subsystem preliminary design baseline is complete and does not exceed subsystem mass and power allocations. Subsystem preliminary design includes hardware and software considerations. Test and verification plans are defined. Technical interfaces are defined, deemed adequate, and are consistent with the overall technical maturity at an acceptable level of risk. Resolution plans for any non-compliance have been developed.

5. Test, verification and manufacturing planning are underway.

3. The subsystem risks and opportunities are understood and have been credibly assessed. Acceptable risk mitigation plans have been developed where applicable.

4. The preliminary subsystem design is reliable and safe for use. Reliability and safety analyses of the preliminary design have been performed and resulting requirements flowed down to the system, module, and subsystem. Preliminary plans for tests, analyses, demonstrations, and inspections will verify the reliability and safety of the subsystem. Safety, Reliability, and Quality Assurance (SR&QA) has been adequately addressed in preliminary designs, and applicable SR&QAproducts (e.g., PRA, system safety hazard analysis, and failure modes and effects analysis) satisfy PDR product maturation requirements (e.g. Phase 1 Safety Review criteria)

Mechanism

s

LAS A

MTPSEPS

LAS A

CM

Pyro

AV &

SWEC

LSStructures

SM Prop

CM

PropLR

SPTC

Wiring

Crew

Systems

LAS JM

LAS SEI, AVP

GN

&CSubsystem Criteria

1. Requirements are complete and flowed to the appropriate products. Subsystem specifications are updated.

2. Subsystem design is complete to a level of detail that is sufficient to demonstrate that all requirements are expected to be met at an acceptable level of risk. Subsystem preliminary design baseline is complete and does not exceed subsystem mass and power allocations. Subsystem preliminary design includes hardware and software considerations. Test and verification plans are defined. Technical interfaces are defined, deemed adequate, and are consistent with the overall technical maturity at an acceptable level of risk. Resolution plans for any non-compliance have been developed.

5. Test, verification and manufacturing planning are underway.

Project Orion





Project Orion

SMR

PDR KO7/13 7/21




Sponsor)

System &Module

Presentations




NASAScreening



RID Tool OpenTBD


DRDsRedelivered

NET 11/6


RIDs.

PreBoard

4 Day Pre-Board to work issues in parallel with completion of RID Disposition

Orion PDR


Starts 8/3

Starts 8/12


Starts 8/17



S

IntegTeam


PIT War Room

8/31-9/1 PDR Board


Dry-Runs


2-WkSSDRs SMR Prep

Data Requirements Description (DRD) Review and Waterfall

PDR Data Package• NASA/LM jointly assessed PDR deliverables to determine which DRDs were

necessary to meet PDR expectations and satisfy criteria • PDR –

• 146 Deliverable Documents(Includes multiple volumes of specific DRDs, i.e. MVP, Specs, IRDs)


• Data Package Total• 172 Total PDR Documents

• 145 RIDable Documents (listed on PDR Portal)• 12 Commentable Documents• 17 DRDs reviewed through NASA DRD Review Process Early

(2 Documents Redelivered Prior to 7/7)• 53 “Drafts Available” DRDs provided to NASA for comment

• Total SSDR and PDR Data Package consisted of nearly 100,000 pages of requirements, interface definition, verification details, general plans, and supporting design detail

Waterfalled Review and Delivery• Managed via the PDR Integration Team

• Captured in a 2000+ line MS Project Schedule• Included Tech Reviews, Peer Reviews, C&DM Processing, and Delivery

• Metrics generated on a weekly basis• Reported to Management team through various forums

• Deliveries culminated on 7/7/09, PDR-45 Days

Project Orion





Project Orion

SMR

PDR KO7/13 7/21




Sponsor)

System &Module

Presentations




NASAScreening



RID Tool OpenTBD


DRDsRedelivered

NET 11/6


RIDs.

PreBoard


Orion PDR


Starts 8/3

Starts 8/12


Starts 8/17



S

IntegTeam


PIT War Room

8/31-9/1 PDR Board


Dry-Runs


2-WkSSDRs SMR Prep


PDR Data Package• NASA/LM jointly assessed PDR deliverables to determine which DRDs were

necessary to meet PDR expectations and satisfy criteria • PDR –

• 146 Deliverable Documents(Includes multiple volumes of specific DRDs, i.e. MVP, Specs, IRDs)


• Data Package Total• 172 Total PDR Documents

• 145 RIDable Documents (listed on PDR Portal)• 12 Commentable Documents• 17 DRDs reviewed through NASA DRD Review Process Early

(2 Documents Redelivered Prior to 7/7)• 53 “Drafts Available” DRDs provided to NASA for comment

• Total SSDR and PDR Data Package consisted of nearly 100,000 pages of requirements, interface definition, verification details, general plans, and supporting design detail

Waterfalled Review and Delivery• Managed via the PDR Integration Team

• Captured in a 2000+ line MS Project Schedule• Included Tech Reviews, Peer Reviews, C&DM Processing, and Delivery

• Metrics generated on a weekly basis• Reported to Management team through various forums

• Deliveries culminated on 7/7/09, PDR-45 Days


PDR Data Package• Metrics Summary – July 8th, 2009

Project Orion





Project Orion

SMR

PDR KO7/13 7/21




Sponsor)

System &Module

Presentations




NASAScreening



RID Tool OpenTBD


DRDsRedelivered

NET 11/6


RIDs.

PreBoard

Orion PDR


Starts 8/3

Starts 8/12


Starts 8/17



S

IntegTeam


PIT War Room

8/31-9/1 PDR Board


Dry-Runs


2-WkSSDRs SMR Prep


System & Module Review (SMR)

The purpose was to kick off the Orion Preliminary Design Review and to provide a detailed overview of the DAC-3 close-out (606G) vehicle configuration and system architecture. The goals of SMR were to provide:

• Common understanding of the system architecture and PDR technical baseline

• Common understanding of ground systems and operations requirements and plans

• Common understanding of flight operations by mission phase

• Status on major open system, module, and subsystem issues

• Clarification of what documents are available at PDR and the applicable NASA documents, by version, used in developing the Data Requirements Document (DRDs)

• RID criteria and ground rules

• 7 Day Review• Participants included NESC, SRB, NASA Crew Office,

NASA MOD, NASA Engineering, NASA S&MA, NASA Health & Medical, Constellation, Orion Project, & LM Central Engineering

Project Orion





Project Orion

SMR

PDR KO7/13 7/21




Sponsor)

System &Module

Presentations




NASAScreening



RID Tool OpenTBD


DRDsRedelivered

NET 11/6


RIDs.

PreBoard


Orion PDR


Starts 8/3

Starts 8/12


Starts 8/17



S

IntegTeam


PIT War Room

8/31-9/1 PDR Board


Dry-Runs


2-WkSSDRs SMR Prep

Documentation Review

RID and Comment Generation• RIDs Written Against

• 145 RIDable Documents• Architecture Elements (IPTs, Modules, Subsystems)• All PDR Related Presentation Material

• Comments Written Against• 53 “Drafts Available” DRDs provided to NASA for comment

• 1200+ Reviewers (Initiators & Sponsors) Participated in Review of Documentation

• Detailed metrics will be provided in the RID Metrics & Reclama Status portion of the Board

Training• Completed 12 RID Training Sessions Focusing on RID Intiation and

Sponsorship• RID Training Package Located on PDR Wiki and PDR Portal

(See link below)

Data Package Access• NASA ICE PDR Portal

• https://ice.exploration.nasa.gov/ice/site/cx/reviews_orion_pdr/

https://ice.exploration.nasa.gov/ice/site/cx/reviews_orion_pdr/�

Project Orion





Project Orion

SMR

PDR KO7/13 7/21




Sponsor)

System &Module

Presentations




NASAScreening



RID Tool OpenTBD


DRDsRedelivered

NET 11/6


RIDs.

PreBoard

Orion PDR


Starts 8/3

Starts 8/12


Starts 8/17



S

IntegTeam


PIT War Room

8/31-9/1 PDR Board


Dry-Runs


2-WkSSDRs SMR Prep

RID Review and Disposition

• Detailed multi-level approach to RID review and disposition.

• Process included initiators, sponsors, IPT leads, subsystem leads, systems engineering, and project management



Subsystem Design Reviews

• Subsystem Design Reviews (SSDRs) provided the Orion stakeholders the opportunity to review each subsystem’s preliminary design

• Products reviewed at SSDRs provided objective evidence that the preliminary subsystem design meets requirements with acceptable risk

• Additional Objectives included:• Reviewing the preliminary design as documented in the technical design

products for each subsystem • Participating in a face-to-face open forum, raising issues/concerns,

answering questions or clarifying design details• Providing forward plans for significant issues from preceding reviews• Demonstrating horizontal integration with other subsystems• Reviewing the design against functional and performance specifications and

ground and mission operability characteristics • Evaluating the technical adequacy and maturity of design• Ensuring correct design options have been selected• Ensuring interfaces have been identified and defined• Reviewing the verification approach• Reviewing associated risks and risk management strategies• Evaluating the Subsystem readiness to proceed to PDR

Project Orion

System & Module Review (SMR)

47

• The purpose of the System & Module Review (SMR) was to kick off the Orion Preliminary Design Review and to provide a detailed overview of the DAC-3 close-out (606G) vehicle configuration and system architecture. The goals of SMR were to provide:• Common understanding of the system architecture and PDR

technical baseline• Common understanding of ground systems and operations

requirements and plans• Common understanding of flight operations by mission phase• Status on major open system, module, and subsystem issues• Clarification of what documents are available at PDR and the

applicable NASA documents, by version, used in developing the Data Requirements Document (DRDs)

• RID criteria and ground rules

Project Orion

Orion RID Process FlowProject Orion

DRAFT


Initiators

• RIDs can be written against a DRD or against an architectural element (i.e. CM Prop). When writing a RID against an Architectural Element a reference to the document, drawing, or presentation that triggered the concern must be included in the “Other Affected Documents” field.

Roles & Responsibilities• Review PDR products and document issues in the

RID Tool• Respond to dispositions (agree/disagree) provided

by the Screening Team, Product Owner, Review Team Leads, and/or Pre-Board (within 2 working days)

• Submit reclamas against RID dispositions to the next higher level in the process, if an agreement cannot be met regarding the recommended disposition (including the Sponsor level)

• Prepare presentation information, as needed, for RIDs reclama’d to the Pre-Board and/or Board level


DRAFT

Sponsors

• A technically cognizant manager knowledgeable of the Orion design.

• Sponsors represent Projects, Directorates, and Functional Areas.

Roles & Responsibilities• The Sponsor reviews all RIDs from their

organization’s RID Initiators prior to submitting the RID to the Orion Project.

• The Sponsor will first review the RID to ensure that it meets the appropriate RID Criteria.

• If the RID Criteria are not met or it is out of scope of this review, then it is expected that the Sponsor will request withdraw or ask the RID Initiator to modify the RID appropriately so that it meets the RID Criteria.

• Next the Sponsor should determine if the RID is representative of their organization’s position on the RID issue. If it is not, then it is expected that the Sponsor will request withdraw or ask the RID Initiator to modify the RID appropriately.

• If the Initiator is not available to reclama or defend their RID, the Sponsor has the authority to act on behalf of the Initiator.


DRAFT

Integration Team

Members• Lead by the PDR Integration Team

(MRI, PIT will act as super users in the Tool)

• Reps from IPTs• Relevant subsystem experts, as required per

team

• Roles & Responsibilities• Triage all RIDs to appropriate Screening Teams

(performed outside the RID tool)• Integration team will download to a spreadsheet and

flag screening/review teams to ensure all RIDs are bucketed correctly.

• Ensure that RIDs are complete and not missing relevant/key info

• Scan every RID to ensure proper Screening , Product Owner, and Review Team mapping

• For example, determine if RIDs against specs go to Requirements Team or the Module Teams

• Ensure the RIDs are assigned to proper technical authority (resource owner) for disposition

• Ensure RIDs that need horizontal integration have been addressed.

• Increase coordination on hot issues that span multiple team areas by flagging RIDs that will have to be transferred between Screening teams


DRAFT

Integration Team




team










Integration Team (Continued)

Roles & Responsibilities (cont.)• Flag RIDs that need coordination from multiple

Screening and Review Teams• Flag RIDs that need to be merged

• Upon integration team direction, MRI will merge the RIDs

• Merge duplicate RIDs to reduce work load on Review Teams

• Super users will move RIDs around “informally” as needed

• While Screening/Review Teams are meeting, team will transition to “War Room Mode”

• Team will help manage RID review schedules across review teams

Schedule• Early meeting(s) during the week of July 27-30

• Daily meetings starting at 8:30 AM CT (Go until we have finished what could be processed)

• Target 3-4 Aug 09 in Houston for wrap-up of RIDs dropped on final days prior to 7/31

• Review of integrated RIDs requiring F2F discussion

• Integration team will transition to a full time effort lead by the PIT in parallel to the screening and review teams. (War Room Mode)


DRAFT

Integration Team




team










Integration Team (Continued

Roles & Responsibilities (cont.)• Flag RIDs that need coordination from multiple

Screening and Review Teams• Flag RIDs that need to be merged

• Upon integration team direction, MRI will merge the RIDs

• Merge duplicate RIDs to reduce work load on Review Teams

• Super users will move RIDs around “informally” as needed

• While Screening/Review Teams are meeting, team will transition to “War Room Mode”

• Team will help manage RID review schedules across review teams

Schedule• Early meeting(s) during the week of July 27-30

• Daily meetings starting at 8:30 AM CT (Go until we have finished what could be processed)

• Target 3-4 Aug 09 in Houston for wrap-up of RIDs dropped on final days prior to 7/31

• Review of integrated RIDs requiring F2F discussion

• Integration team will transition to a full time effort lead by the PIT in parallel to the screening and review teams. (War Room Mode)

Integration Team (Continued)

Location• Houston, Space Park

Team Logistics and Procedures• MRI will assist in giving each team a .xls export of

all of the RIDs in their area• Teams will scan every RID assigned to their

Screening/Review Team area • Mini-teams will be created if the volume of RID

throughput is significant• Assign a POC to check off that every RID was

scanned (PIT, MRI)• Teams will flag any RID that needs to be re-assigned

or merged• MRI will re-assign flagged RIDs in the ConCERT RID

Tool

RID Tool Procedures• The integration team will be “invisible” to the RID

Tool• PIT and MRI Super Users will move RIDs around as

needed


DRAFT

Screening Teams

Members• Team Leads (NASA IPT/Mod Lead)• Reps from IPTs as required (T&V, SR&QA,

Requirements, Vehicle, etc.)• Relevant subsystem experts, as required per team• LM Focus team assigns 1 to 2 people per team to

support• PIT and MRI

Roles and Responsibilities• Check that RIDs meet RID Criteria and screen for

quality• Assign recommended disposition to each RID

• “Recommend Withdraw” or “Approve with Mod”• Communicate with RID initiator as necessary

• “Recommend Approve” indicates a good RID and passes on to LM

• Merge duplicate RIDs• Clarify poor RIDs before they are elevated to the

Product Owner/Review Team• Flag RIDs on subjects that have already been

decided at CPCB, ERB, etc (Best effort)


DRAFT

Screening Teams





quality• Assign recommended disposition to each RID• “Recommend Withdraw” or “Approve with Mod”• Communicate with RID initiator as necessary• “Recommend Approve” indicates a good RID and

passes on to LM• Merge duplicate RIDs• Clarify poor RIDs before they are elevated to the



Screening Teams (Continued)

Roles and Responsibilities• Screening Team should attempt to dialogue with

initiator if disposition is other than straight Approve in order to clarify intent and streamline later disposition and review team function (if this bogs us down too much we may need to modify)

• If a large number of RIDs are assigned to a Screening Team, the Lead can assign team members a set of RIDs to screen based on subject and experts needed

• Team Members will recommend disposition to Screening Team Lead

• Can recommend transfer to another Screening Team• MRI will work the tool• Merge parent/child RIDs in the ConCERT RID Tool• Input recommend disposition in the ConCERT RID

Schedule• Planned for August 3-10• May begin meeting and screening RIDs during the

week of 27-31 July 09 at discretion of screening team• RID Schedule and Priority

• Pre-Declared RIDs, As RIDs are sponsored, As team is available

• Team Leads need to gauge the quantity of RIDs to determine how much meeting time is required


DRAFT

Screening Teams





quality• Assign recommended disposition to each RID• “Recommend Withdraw” or “Approve with Mod”• Communicate with RID initiator as necessary• “Recommend Approve” indicates a good RID and

passes on to LM• Merge duplicate RIDs• Clarify poor RIDs before they are elevated to the



Screening Teams

Roles and Responsibilities• Screening Team should attempt to dialogue with

initiator if disposition is other than straight Approve in order to clarify intent and streamline later disposition and review team function (if this bogs us down too much we may need to modify)

• If a large number of RIDs are assigned to a Screening Team, the Lead can assign team members a set of RIDs to screen based on subject and experts needed

• Team Members will recommend disposition to Screening Team Lead

• Can recommend transfer to another Screening Team• MRI will work the tool• Merge parent/child RIDs in the ConCERT RID Tool• Input recommend disposition in the ConCERT RID

Schedule• Planned for August 3-10• May begin meeting and screening RIDs during the

week of 27-31 July 09 at discretion of screening team• RID Schedule and Priority

• Pre-Declared RIDs, As RIDs are sponsored, As team is available

• Team Leads need to gauge the quantity of RIDs to determine how much meeting time is required

Screening Teams (Continued)

Location• Houston, Space Park • Glenn Research Center for SM and Requirements• MSFC, Huntsville for LAS

RID Tool Procedures• MRI will work the tool, with PIT support as needed• Assign recommended disposition and comments in

“Review” Team field• Add a date and “Screening Team” stamp to

comments• If disposition is “Recommend Withdraw” or

“Approve w/ Mod”• Fill out recommended disposition and comments

field• Send back to the initiator using the feature in the tool• If disposition is “Approve” or passing a RID up to

the next level• Fill out recommended disposition and comments

field• DO NOT USE ACTIONS IN THE TOOL (i.e. Send to

Initiator or Send to Next Review Level)• Instead, transfer the RID to the appropriate Review

Team


DRAFT

Product Owner

• Product Owners are primarily LM personnel• When a RID is written against a document, the

product owner is the DRD author• When a RID is written against an Architectural

Element, the product owner is the applicable lead (System, Module, or Subsystem)

Definition• Product owner is responsible for developing

proposed disposition (RID impacts and recommendations)

• Assignment of RIDs to Product Owner validated/reassigned as needed (by the integration team)

• DRD POC is the Book Author• Architecture Element POC is the Subsystem

Lead• LM PIT for coordination and help with the RID tool,

as needed• NASA POCs will help facilitate disposition with

initiators, as applicable

Roles and Responsibilities • Review the content of the RID, including

recommended disposition


DRAFT

Product Owner













Product Owner

Roles and Responsibilities (Cont.)• Coordinate with appropriate product teams/SPTs as

necessary (requirement changes, changes to design baseline, design impacts, schedule impacts, and cost impacts)

• Assess design impacts and propose recommended solution

• Make initial determination of whether the RID is funded and/or in-scope

• Identify if there is a schedule impact. If so, characterize believed impact.

• Identify if there is a cost impact • Elevate to IPT Management to determine if a

LCC is required• Coordinate with RID Initiator to clarify issues

• Enter proposed disposition for review by Review Team (See Product Owner Guidance chart)

• NOTE: Product Owner does not give an “official” disposition in the tool, nor does their disposition get to be reclama’d by the initiator

NoteProduct Owners are not required to contact the RID

initiator when recommending a disposition. However, it is recommended that when further clarification is needed the Product Owner contact the Initiator for additional dialogue.


DRAFT

Product Owner













Product Owner












Product Owner

Schedule• Begin RID recommended disposition as follows

• Pre-declared RIDs – Now• All other RIDs – As soon as Screening Team has

assessed and passed along • Product Owner comments are needed by August 17• Runs in parallel with the Screening/Review teams• May query the RID Tool for RIDs in their functional

area starting July 7

Location• LM personnel in Houston, TX or Denver, CO• Coordination with RID Initiators will be virtual

Procedures• ConCERT RID tool will automatically assign RID to

the appropriate DRD Owner• Product Owner should coordinate with the relevant

technical experts in LMCO• Product Owner should comment on whether the RID

is in-scope or not• Product Owner inputs their recommended

disposition


DRAFT

Product Owner













Product Owner












Product Owner

Schedule• Begin RID recommended disposition as follows

• Pre-declared RIDs – Now• All other RIDs – As soon as Screening Team has

assessed and passed along • Product Owner comments are needed by August 17• Runs in parallel with the Screening/Review teams• May query the RID Tool for RIDs in their functional

area starting July 7

Location• LM personnel in Houston, TX or Denver, CO• Coordination with RID Initiators will be virtual

Procedures• ConCERT RID tool will automatically assign RID to

the appropriate DRD Owner• Product Owner should coordinate with the relevant

technical experts in LMCO• Product Owner should comment on whether the RID

is in-scope or not• Product Owner inputs their recommended

disposition

Product Owner

RID Tool Procedures• The Product Owner does not put in an “official”

disposition into the tool• The RID Initiator cannot reclama the Product

Owner comments• The Product Owner inputs their comments into the

appropriate field• The Review Team will use the Product Owner

comments and recommendations• Product Owner should include the following

information in their comments:• In Scope: Y or N• Schedule Impact: Y or N• Cost Impact: Y or N


DRAFT

Review Teams

Members• Team Leads (NASA & LM IPT/Module Leads)• Reps from other IPTs/Modules, impacts SPTs, • Reps from MOD, Crew Office, CxP, other key

stakeholders (invited but not mandatory)• Relevant subsystem experts, as required per team• RID Initiators/”Champions”• PIT and MRI

Roles and Responsibilities• Review and evaluate technical content of RID• Make recommended disposition on RIDs that are with-in

scope and funding• Concur with or change the recommended disposition• Select “Recommend Approve”, “Approve w/Mod”, or

“Recommend Withdraw”• Elevate RIDs on key issues to the Pre-Board• Communicate with initiator if discussion is needed

Schedule• Planned for August 12-19• May begin meeting early if the team leads choose to do

so as RIDs become available:• As RIDs are submitted or as team is available• Team Leads need to gauge the quantity of RIDs to

determine the total meeting time required


DRAFT

Review Teams









Review Teams

Location• Space Park, look for Room Assignments on the PDR

Portal• SM Team and Requirements Team held at Glenn

Research Center, OH • LAS at MSFC, Huntsville , AL• No meetings will require travel – advisable for Team

leads to be co-located

Procedures• Review Team Leads will chair the meeting• Chairs can run the meetings however they see fit,

whatever works to get through their RIDs – however, some level of consistency will be good to not confuse initiators

• Team Members will provide technical expertise on RID issues

• RID Initiators may come to the meeting in person or dial in via telecon

• Review Team Lead will make a recommend disposition• MRI will merge parent and child RIDs in the ConCERT

RID Tool• MRI will input recommended disposition in the

ConCERT RID Tool


DRAFT

Review Teams









Review Teams

Location• Space Park, look for Room Assignments on the PDR

Portal• SM Team and Requirements Team held at Glenn

Research Center, OH • LAS at MSFC, Huntsville , AL• No meetings will require travel – advisable for Team

leads to be co-located

Procedures• Review Team Leads will chair the meeting• Chairs can run the meetings however they see fit,

whatever works to get through their RIDs – however, some level of consistency will be good to not confuse initiators

• Team Members will provide technical expertise on RID issues

• RID Initiators may come to the meeting in person or dial in via telecon

• Review Team Lead will make a recommend disposition• MRI will merge parent and child RIDs in the ConCERT

RID Tool• MRI will input recommended disposition in the

ConCERT RID Tool

Review Teams

RID Tool Procedures• MRI will run the tool during the meeting, with PIT

support as needed• Review Team will edit the recommended disposition

given by the Screening Team (the same field is used)• Review Team will add their comments and rationale to

the same field as the Screening Team• Provide a date and name stamp (“Review Team”) for

all comments• If disposition is “Recommend Withdraw” or “Approve w/

Mods”• Send RID back to Initiator using feature in Tool

• If disposition is “Approve” or elevating to Pre-Board• Assign disposition and input comments• Use Tool to send RID to next level (Pre-Board)

• This is the step where the RID is elevated

Project Orion


DRAFT

Pre-Board

• NASA Orion VI&D chaired board• Provides initial evaluation of the Project’s success in

meeting the PDR entrance and success criteria

Roles & Responsibilities• Resolve and disposition elevated or disputed RIDs• Identify issues that need to go forward to the Board

(due to significant cost, schedule, or technical impact/risk)

Project Orion


DRAFT

Board

• NASA Orion Project Manager chaired• Evaluates the Project’s success in meeting the PDR

success criteria• Approving the basis of and viable path for

proceeding to the Critical Design Review

Roles & Responsibilities• Hears all reclama elevated to the board level and

makes the final decision regarding RID disposition• Evaluates any issue that has been brought forward

to the Board (due to significant cost, schedule, or technical impact/risk)

• Is the final decision authority for dispositioning of on all RIDs

• Ensures the closure of RIDs resulting from PDR

Project Orion

Orion PDR RID Metrics

Orion vs. External Sponsored RIDs

Orion, 2043

External, 1582

RIDs Withdrawn ApprovedOrion 2043 860 1183External 1582 787 795

Project Orion

67

Project Orion

Orion PDR RID Metrics

Discipline # of RIDsAv & Sw 605Av: Comm & Track 22Av: C&DH 30ICDs 107CM 452LAS 267Con Ops 204Vehicle 142ICCS 120Mech 115SM 126ECLSS 124EPS 103Propulsion 76Pyrotechnics 88GN&C 65Structures 72Hazard Analysis 45T&V 204Crew Systems 88MGSE 49Drawings & Models 37LRS 57EGSE 43Manufacturing & Assembly 23PTC 26Wiring 40TPS 23Other 272

Orion PDR Sponsored RIDs by Discipline

CM, 452

LAS, 267

Mech, 115

SM, 126

ECLSS, 124

EPS, 103

Propulsion, 76

Pyrotechnics, 88

ICDs, 107

Av: C&DH, 30

Av: Comm & Track, 22

Con Ops, 204

LRS, 57

EGSE, 43

Vehicle, 142

Wiring, 40Other, 272

Av & Sw , 605

Crew Systems, 88

Hazard Analysis, 45

T&V, 204

GN&C, 65

ICCS, 120

Date post:	05-Sep-2014
Category:	Technology
Upload:	nasapmc
View:	12,702 times
Download:	1 times