Are You Getting the Most from Your Quality Engineer?

Presented by

Charles B RobinsonCertified Lead Auditor / Senior Quality Engineer

Quality Assurance & Risk Management Services, Inc.

This session is intended to provide an outline of how Quality Engineering (QE) should be adding value during the Life Cycle phases of a Project.

This session will:

Correlate the Life Cycle model with NASA Procedural Directives (NPD’s), NASA Procedural Requirements (NPR’s), AS9100, and the QE Disciplines

Explain a QE’s mission assurance contribution to processes and procedures during the project life cycle

Provide a reference tool that QE’s can use during the Life Cycle of a project.

Provide an understanding of how a project has embraced the mission assurance aspects of AS9100D from a QE's perspective

Software Disciplines are not addressed in this Session

Purpose

AS9100Life Cycle

Mission AssuranceRequirements

NPD’s & NPR’s

Competency Wheel

NASAQuality Engineers

Supportive Systems

PLAN DO CHECK ACT4

Context of Organization

5Leadership

6Planning

7Support

8Operation

9Performance Evaluation

10Improvement

No Requirements1

Scope2

Reference3

Terms & Definitions

Conduct QualityEvaluations &Data Analysis

PerformQuality Assurance

Management& Planning

Define & PerformInspection Testing& Acceptance

Activities

Conduct Procurement

QualityEngineering

Define &ImplementProduction,Material &

Process Controls

ImplementProblem Reporting& CorrectiveActions (PRACA)

AssuresAdequateDesign &

DevelopmentControls

QualityEngineer

For more detail contact:Michael P. Kelly

NASA Safety Center302-286-0662

Michael.P.Kelly@nasa.gov

NASA Life-Cycle Phases

Phase CFinal Design

& Fabrication

Phase FCloseout

Phase EOperations

& Sustainment

Phase BPreliminary Design &

Technology Completion

Phase DSystem

Assembly,Integration &

TestLaunch

Phase AConcept & Technology

Development

Pre-Phase AConcept Studies

Project Life-Cycle Phases

OR

R

FRR

PLA

R

CER

R

Key Decision Points

MC

R

SR

R

PD

R

CD

R/

PR

R SIR

TRR

DR

SA

R

PFA

R

Launch

Formulation ImplementationApproval

MCR – Mission Concept ReviewSRR – System Requirements ReviewMDR – Mission Definition ReviewSDR – System Definition ReviewPDR – Preliminary Design ReviewCDR/PRR – Critical Design Review / Production Readiness ReviewSIR – System Integration Review

TRR – Test Readiness ReviewSAR – System Acceptance ReviewORR – Operations Readiness ReviewFRR – Flight Readiness ReviewPLAR – Post Launch Assessment ReviewCERR – Critical Events Readiness ReviewPFAR – Post-Flight Assessment ReviewDR – Decommissioning Review

MD

RS

DR

NPR 7123.1NASA Systems

EngineeringProcesses and Requirements

NPR 7120.5NASA Space

Flight Program & Project Mgmt. Requirements

SRR – System Requirements ReviewTo evaluate whether the functional and performancerequirements defined for the system are responsive tothe program's requirements on the project andrepresent achievable capabilities.

The Quality Engineer understands the System Definition to assure their future activity is focused on mission success

Understand initial requirements - End Items functional baselineUnderstand clarity of system definition - architectural level of designCreate the SMA quality level requirements, e.g., develop the SMA Plan. Identify workmanship standards, critical technology, long lead items.NASA Advisories (NPR 8735.1) and GIDEP are addressed in the Safety & Mission Assurance PlanCommunicate concerns (schedule, technology, long lead items, etc.)

I – Initiate, P – Preliminary, B – Baseline, U – Update, F- Final -&- 1 – Develop, 2 – Research & Review Requirements

Life Cycle Phase A B C D E FAS9100

↓ NPR 7120.5 (Sections) ↓ SRR MDRSDR PDR CDR

PRR SIR TRR ORR FRR DR

Risk Management Plan (G-3.3) P/2 B/1 U/1 6.1, 8.1.1Risk Mitigation Plan (F-7.0) I/2 U/2 U/2 U/2 U/2 U/2 U/2 8.2.2Safety & Mission Assurance Plan (G-3.2) P/1 B/1 U/1 U/1 U/1 6.0Material Requirements Plan (NPR 8730.2) I/2 U/2 U/2 U/2 8.1, 8.1.4Configuration Management Plan (G-3.14) P/2 B/2 U/2 8.1.2Acquisition Plan (G-3.4) P/2 B/2 8.4Technology Development Plan (G-3.5) P/2 B/2 8.2.2.d

MissionSuccess

Environment

Ground

Suppliers

University

Workmanship

Training

Mission AssuranceStaff

Material

Length of Mission

Instruments

New Supplier

Documents

Spares

Software

Reliability

GIDEP

SOW

LOD

Durationof Mission

Components

Safety & Mission Assurance Plan

Problem / Failure Reports from similar Projects

Experienced?

SupplierHistory

Equipment

Mandatory Inspection

Points

Areas to consider to prepare for your involvement

Launch vehicle

Spacecraft

On‐Site Personnel ?

Safety

Materials

High Risk Electronic

Parts [ERAI, Inc.]

GIDEP Information

[NANADARTS]

NASA Advisories

[NANADARTS, NARS]

Customs & Border

Protection C/F Parts Seizures

[CBP]Commodity Information

[SAM]

General Supplier

Information [DLIS]

PQA “Surveyed

Supplier” Info [GRC, JSC]

Supplier Information

Surveys [Supplier Provided]

Audit Reports [NASA Audits]

Debarred Suppliers

[DLIS]

Supplier QMS Certifications[Audit Teams,

Suppliers, OASIS]

Supplier Ratings

[DCMA, Navy, SAS]

Supplier Assessment System (SAS)

NPR 8735.2, ¶2.6.3.5

SAS Administrator - jsc-sasAdmin@mail.nasa.govSAS Web site - http://sas.nasa.gov/

MDR (R) – Mission Definition ReviewSDR (H) – System Definition Review

The Quality Engineer.Knowing the project's mission/system definition and associated plans,analysis the risks, identifies the resources needed for mission assurance,and starts a compliance matrix.As a component of the Verification & Validation Plan, Mandatory InspectionPoints are identified by the QE.Identifies any concerns prior to beginning Phase B.

To evaluate the credibility and responsiveness of theproposed mission/system architecture to the programrequirements and constraints, including available resources. To determinewhether the maturity of the project's mission/system definition and associatedplans are sufficient to begin Phase B.

I – Initiate, P – Preliminary, B – Baseline, U – Update, F- Final -&- 1 – Develop, 2 – Research & Review Requirements

Life Cycle Phase A B C D E FAS9100

↓ NPR 7120.5 (Sections) ↓ SRR MDRSDR PDR CDR

PRR SIR TRR ORR FRR DR

Risk Management Plan (G-3.3) P2 B1 U1 6.1, 8.1.1Safety & Mission Assurance Plan (G-3.2) P1 B1 U1 U1 U/1 U/1 6.0

Verification & Validation Plan (G-3.9) P2 B2 U2 U2 8.4.2, 8.4.3, 8.5.1.m

Baseline the Safety & Mission Assurance Plan (SMAP)

QEStart

GSFC ‐ 380‐WI‐7120 1 1AProject and or Program Mission Assurance Requirements (MAR) Preparation – Figure 1

PDR – Preliminary Design Review

To evaluate the completeness/consistency of theplanning, technical, cost, and schedule baselinesdeveloped during Formulation. To assess complianceof the preliminary design with applicable requirements and to determine if theproject is sufficiently mature to begin Phase C.

The Quality EngineerReviews the flow-down requirements to assure they are properly identifiedincluding workmanship standards, NASA Advisories/GIDEP, etc.Confirm the preliminary design is sufficiently mature to protect missionassurance, including schedule.Identify potential risk using FMEA as appropriate.Contribute to the initial Hazard Analysis

I – Initiate, P – Preliminary, B – Baseline, U – Update, F- Final -&- 1 – Develop, 2 – Research & Review Requirements

Life Cycle Phase A B C D E FAS9100

↓ NPR 7120.5 (Sections) ↓ SRR MDRSDR PDR CDR

PRR SIR TRR ORR FRR DR

Mishap Preparedness and Contingency Plan (NPR 8621.1) P/1 U/1 B/1 U/1 8.2.1

Safety Data Packages (NPR 8715.7) I/1 U/1 F/1 7.3, 8.1.3

ELV Payload Safety Process Deliverables (NPR 8715.7) P/1 P/1 B/1 8.1.3

Safety & Mission Assurance Plan (G-3.2) P/1 B/1 U/1 U/1 U/1 U/1 6.0

Failure Mode & Effect Analysis (FMEA) – Screening Process

FMEA

NASA, National Space Transportation System (NSTS) 22206, Req’ts. for Preparation & approval of FMEA & Critical Items List (CIL) – Figure 3-2

Goddard Space Flight Center (GSFC)-FAP-322-208-FMEA-Draft – Functional, Design, Interface, Software, GSE, and Process FMEA instructions – Figure 1-2

0%2%4%6%8%

10%12%14%16%18%

Review the Corrective Action history of the department or supplier to identify weaknesses that should be mitigated.

The above are the combined supplier audit results performed by a NASA Centerfrom 2Q2015 to 1Q2016

CDR – Critical Design ReviewPRR – Production Readiness Review

To evaluate the integrity of the project design and itsability to meet mission requirements with appropriatemargins and acceptable risk within defined project constraints, including

available resources. To determine if the design is appropriately mature tocontinue with the final design and fabrication phase.

I – Initiate, P – Preliminary, B – Baseline, U – Update, F- Final -&- 1 – Develop, 2 – Research & Review Requirements

The Quality Engineer.Input Mandatory Inspections as appropriateComplete the Hazard Analysis and confirm controls are in placeConfirm the Test Plan is in placeValidate the requirements for integration are in place (if applicable)

Life Cycle Phase A B C D E FAS9100

↓ NPR 7120.5 (Sections) ↓ SRR MDRSDR PDR CDR

PRR SIR TRR ORR FRR DR

Integration of systems P/2 U/2 4.4.1

Product Integration Process – NPR 7123.1, Figure C-7

SIR – System Integration ReviewTo evaluate the readiness of the project andassociated supporting infrastructure to begin systemAssembly Integration & Test (AI&T), evaluate whetherthe remaining project development can be completedwithin available resources, and determine if the projectis sufficiently mature to begin Phase D.

I – Initiate, P – Preliminary, B – Baseline, U – Update, F- Final -&- 1 – Develop, 2 – Research & Review Requirements

The Quality Engineer Review documentation prior to testing and/or integration to assure compliance.Verify compliance to the Test Plan

Life Cycle Phase A B C D E FAS9100

↓ NPR 7120.5 (Sections) ↓ SRR MDRSDR PDR CDR

PRR SIR TRR ORR FRR DR

Mission Operations Plan (G-3.10) P/2 B/2 U/2 6.2Rank Safety Risk Management Process Documentation (H-3.24) B/1 6.1.1, 8.1

Subject Sample of areas to review Confirmed by QE

Final Design Change Notices clearly understoodSafety Data Package Assure that mitigations are in placeFMEA Analysis complete and mitigation plans in placeDrawings Reviewed for the ability to manufacture and inspectVerification Plans Mandatory Inspection PointsPQA Supplier flow-down appropriateQA Audit Audits complete and findings satisfiedHazard Analysis Created and hazards understood / mitigatedMission Assurance Requirements

Risk Assessments addressed, As-built parts list approved…

Develop a Checklist to assure theMission Assurance areas are covered.

TRR – Test Readiness Review

Ensure that for each planned test or series of tests thetest article (hardware / software), test facility, supportpersonnel, and test procedures are ready for testingand data acquisition, reduction, and control.

A TRR for each planned test or series of tests ensures that the test article(hardware/software), test facility, support personnel, and test procedures areready for testing and data acquisition, reduction, and control.

Life Cycle Phase A B C D E F

AS9100↓ NPR 7120.5 (Sections) ↓ SRR

MDR

SDRPDR CDR

PRR SIR TRR ORR FRR DR

NPR 7123.1, Table G-10, TRR Entrance & Success Criteria B/2 8.3.4.1

8.4.2I – Initiate, P – Preliminary, B – Baseline, U – Update, F- Final -&- 1 – Develop, 2 – Research & Review Requirements

The Quality Engineer Review and/or contribute to the test proceduresIdentify Mandatory inspection points and assure they are incorporated into the test documents.Develop a plan to validate/witnessing test results (LOD for off-site, on-site SMA plan)

Documented, Accepted, Configuration Control, & Communicated

A Sample of “Things” that need to be in place.

ORR – Operations Readiness ReviewTo evaluate the readiness of the program, including itsprojects, ground systems, personnel, procedures, anduser documentation. To operate the flight system andassociated ground systems in compliance withprogram requirements and constraints during theoperations phase.

I – Initiate, P – Preliminary, B – Baseline, U – Update, F- Final -&- 1 – Develop, 2 – Research & Review Requirements

The Quality Engineer Confirm that all non-conformances have been addressed and closed.Verify that any mitigation plans have been satisfied.Validate the “as built” configuration.Confirm all GIDEP’s or NASA Advisories have been addressed.Confirm the Safety Data Package is complete.

Life Cycle Phase A B C D E FAS9100

↓ NPR 7120.5 (Sections) ↓ SRR MDRSDR PDR CDR

PRR SIR TRR ORR FRR DR

Nonconforming Product I/1 U/1 U/1 U/1 8.4Mitigation Plans P/2 U/2 U/2 U/2 8.1.1.d

As-built Configuration P/1 U/1 U/1 U/1 U/1 8.1.2, 8.3.6, 8.5.2

GIDEP ALERT, SAFE-ALERT, Problem Advisory, Agency Action Notice, and NASA Advisory

Safe AlertsAlertsProblem AdvisoriesAgency Action NoticesLessons LearnedProduct Chang NoticesProduct InformationDMSMSReliability MethodologyFailure AnalysisReliability PredictionsReliability StatisticsFacilities DocumentsComputer TechnologyProcess SpecificationsTest ReportsEngineering ReportsManagement ReportsSoldering TechnologiesTechnical ManualsGeneral MetrologyCalibration Procedures

FailureExperience

ProductInformation

Reliability

Engineering

Metrology

Advisories&

An overview of GIDEP investigations throughout the Life-Cycle

NPR 1441.1E - NASA Records Management Program Requirements,Appendix D: Agency Filing Scheme

5100 – Procurement (Contracts)5300 – Reliability and Quality Assurance5700 – Safety and Mission Assurance

The Records Life-CycleNPR 1441.1, Figure 1

National Archives & Records Administration (NARA)

Presented byCharles B. Robinson

Quality Assurance & Risk Management Services, Inc. / Navarro Engineering Solutions, 22 Highridge DriveAmerican Canyon, CA 94503www.QAandRM.com / www.NavarroES.com

Email: Crobinson@QAandRM.comCell: +1 480.276.7649