Framework for Compliance, Verification, and Non-Conformance George Angeli LSST All-Hands Meeting...

transcript

Framework for Compliance, Verification, and Non-Conformance

George Angeli

LSST All-Hands MeetingBremerton

August 17, 2015

LSST Systems Engineering 2

Outline

8/17/2015

• Non-conformance framework (George Angeli)–Quality assurance at project and subsystem level–Change Control Board–Acceptance Data Package–Case studies

• Verification and Validation on LSST (Brian Selvy)

• Subsystem Acceptance and Verification in Commissioning (Chuck Claver)

Overview

8/17/2015

• Basic procedures of Quality Assurance implemented by subsystems–Further refinement in progress (eTraveler, detailed NCR process)–Project QA should match in maturity the subsystems–Bottom-up and top-down procedures should meet in the middle

• Objectives–Establish a process for handling non-conformance at project level

•Well documented and uniform•Transparent and predictable for the project community•Do not interfere with current subsystem procedures

–Establish a process for documenting deliverables to and by the project•Uniform across all subsystems•Provide a well organized archive for the lifetime of the Observatory•Accessible for the project community and the operations team

Quality Assurance

8/17/2015

• Well established regime of inspections and tests• Well defined responsibilities• Traceable and detailed documentation of deliverables and their

verification/certification• Transparent and consistent handling of non-conformance • Every subsystem has the responsibility to establish its own QA• Lower level plans (safety, QA, verification, review, etc.) should

conform with higher level ones • Project level coordination of deliverables

–To the Project by subsystems, and–By the Project to stakeholders/operators

• KEEP IT SIMPLE! (as much as possible …)

The “Linear” Way of Acceptance

8/17/2015

Design Requirements

Documents

Interface Control

Documents

Verification Plan

Inspection

Calibration

VerificationActivities

Acceptance Data Package

Acceptance

Standards &Guidelines

Review Reports(when

applicable)

All requirements are met

Full compliance

The “Linear” Way of Acceptance

8/17/2015

Design Requirements

Documents

Interface Control

Documents

Verification Plan

Inspection

Calibration

Acceptance

Review Reports(when

applicable)

All requirements are met

Full compliance

PSE witnessing

PSE collecting

• Project Systems Engineering understanding of verification and as-built component behavior facilitates system verification and commissioning

The “Not-So-Linear” Way

8/17/2015

Design Requirements

Documents

Interface Control

Documents

Verification Plan

Inspection

Calibration

Acceptance

Review Reports(when

applicable)

Non-Conformance

Full compliance

The “Not-So-Linear” Way

8/17/2015

Design Requirements

Documents

Interface Control

Documents

Verification Plan

Inspection

Calibration

Acceptance

Review Reports(when

applicable)

Non-Conformance

Full complianceChange

Control Process

Deviation note

Non-Conformance Resolution

8/17/2015

• Using the Change Control Process is convenient– Proper people are involved – bring understanding of all aspects of the

project– Well established, exercised process– Well documented – web site captures process steps– Transparent and archived process

• Deviation approval process1. Non-Conformance Report – as soon as it is recognized2. Change Request

• Filed against requirements, as usual, but• Can be filed for a single component only• All relevant documentation posted in the DocuShare collection

3. Impact study, as usual4. Decision, as usual

• CCB is an advisory body• Project Manager disposes it, per the Project Execution Plan

Turnaround time limited by the impact study, not by the process

8/17/2015

• Relevant requirements– Specifications and interface requirements– Non-conformance reports, change requests, deviation approvals

• Compliance certification– Verification Matrix– Verification and acceptance test plans and procedures– Qualification and/or acceptance test results– Calibration and certification requirements records– Material and/or process certifications

• Safety documentation and constraints• As-built technical documentation• Shipping, handling and storage plans• Operations, maintenance, and repair plans and manuals

Case Study 1: M1M3 Acceptance

8/17/2015

• Managed by the Telescope & Site (T&S) team– Jacques Sebag, in particular

• All tests were done by Steward Observatory Mirror Lab (SOML) personnel–Witnessed by representatives of the T&S and Systems Engineering

teams• PSE was invited to all meetings with the vendor

– PSE supported the T&S team with analyses and trade studies• Documentation

– All data received from vendor posted on an open confluence page– All documentation collected on project DocuShare

• M1/M3 Cast Mirror Specification • M1M3 Mirror Verification Matrix• M1M3 Mirror Acceptance Test Plan • M1M3 Mirror final acceptance report • LSST M1M3 Measurements of Prescription, Dimensions and Alignment • LSST Final Acceptance Review 150410 compressed.pptx

Case Study 1: M1M3 Analyses

8/17/2015

• Rebuilding the “M1M3 Null Test Data Reduction Pipeline” (Document-17912)– Laying the foundation of fully understanding the most critical vendor

acceptance test• Potential non-conformance: crow’s feet on M3

– In support of the T&S team, PSE evaluated the impact on ultimate metrics (Document-17171)

• Image depth• Ellipticity (single visit)

– Positive outcome cleared the way to acceptance

Case Study 2: L1L2 Assembly

8/17/2015

• Early involvement of PSE – Invitation to all reviews (PDR, 75%, 90%, FDR)

• Verification/test plans• Compliance status

– Sharing data and analyses (FEA results, scattered light, interface volume, etc.)

• Potential non-conformance issues–Glass refraction index uniformity

• Joint optical analysis with LLNL cleared it –Surface treatment of carbon fiber

• Deviation request against an OSS requirement• Based on PSE scattered light analysis, the request is about to be approved

Summary

8/17/2015

• It’s essential to involve PSE early in

– Verification planning

– Compliance assessments

– Verification activities

• System level non-conformance is handled through the CCB

process

• Acceptance Data Packages are expected to accompany every

deliverable

– System Data Package are built on subsystem Data Packages

– Subsystem Data Packages are built on major component Data

Packages; and so on …