Company Confidential

AS 9101E

Documentation and Execution

Brian Geer (Lockheed Martin) Paul Dionne (ABS QE)

9101 Revision Team

1

AAQG Auditor Workshop

Long Beach CA July 16,17 2014

2

Agenda

• 9101E Key Revision Summary• Process Effectiveness Assessment Report

(PEAR) – more structure

• Process Evaluation Matrix - new

• Quality Management System (QMS) Process Matrix – places for Objective evidence

• Definitions- Added to support ISO 9000 definition of effectiveness

• Case study with PEAR and Process Matrix examples

• Q&A

PEARs and the QMS Process Matrix

• Objective Evidence is recorded on PEAR’s for all Clause 7 related processes.

• The Pear has 3 areas for collection of Objective evidence:

• Section 1 - Process Details: Inputs, Activities, Outputs and Interactions

• Section 2 – Process Results includes the identification of performance measures and KPIs.

• Section 3 - Process Realization summarizes audit trails and sources of evidence

3

PEARs and the QMS Process Matrix

• Objective Evidence is recorded on the QMS Matrix for non clause 7 related processes.

• Additional fields have been added at the end of clauses 4,5,6 and 8 for this purpose

• Clause 7 evidence will still be captured on the PEAR

Why the change ? The Objective Evidence Report is no longer required

4

5

PEARs and the QMS Matrix

9101D: The PEAR and OER are used for collection of Objective Evidence

and Process effectiveness information

9101E: The PEAR is used to collect objective evidence for Product realization clauses. Added fields in the Process Matrix report are used for

other clauses

6

PEARs and the QMS Matrix 9101 FORM 3: PROCESS EFFECTIVENESS ASSESSMENT REPORT

1 CB Name PROCESS EFFECTIVENESS

ASSESSMENT REPORT

2 CB Logo

3 Organisation:

4 Site(s):

5 OIN(s):

6 PEAR Number: 7 Audit Report Number: 8 Issue Date:

SECTION 1 – PROCESS DETAILS 9 Process Name:

10 AQMS Standard(s):

9100 ☐ 9110 ☐ 9120 ☐

Applicable 9100/9110/9120 clause(s):

11 Inputs:

12 Activities:

13 Outputs:

14 Interactions/Interfaces:

SECTION 2 – PROCESS RESULTS

15 Organisation’s method for determining process results:

16 Performance Measures KPI 1: KPI 2: KPI 3:

17 Auditor observations and comments supporting process result determination:

Reference Target for

Audited Period Value Measured for

Audited Period Comments

KPI 1:

KPI 2:

KPI 3:

9101 FORM 3: PROCESS EFFECTIVENESS ASSESSMENT REPORT

1 CB Name PROCESS EFFECTIVENESS

ASSESSMENT REPORT

2 CB Logo

3 Organisation:

4 Site(s):

5 OIN(s):

6 PEAR Number: 7 Audit Report Number: 8 Issue Date:

SECTION 1 – PROCESS DETAILS 9 Process Name:

10 AQMS Standard(s):

9100 ☐ 9110 ☐ 9120 ☐

Applicable 9100/9110/9120 clause(s):

11 Inputs:

12 Activities:

13 Outputs:

14 Interactions/Interfaces:

SECTION 2 – PROCESS RESULTS

15 Organisation’s method for determining process results:

16 Performance Measures KPI 1: KPI 2: KPI 3:

17 Auditor observations and comments supporting process result determination:

Reference Target for

Audited Period Value Measured for

Audited Period Comments

KPI 1:

KPI 2:

KPI 3:

7AAQG Auditor Workshop

Long Beach CA July 16,17 2014

PEARs and the QMS Matrix• A PEAR is required for all product realization

processes audited

• Objective evidence is recorded on the PEAR for product realization processes; can include objective evidence from other clauses related to the audited process

• For non product realization processes, objective evidence is recorded on the QMS Matrix report

8

Key Changes-NCRs PEAR’s and the PEM

• NCRs will be required for processes with an effectiveness level of “1” but no longer required for a level of “2”– It is expected Nonconformities would be discovered

within the process if rated a 2

– MOTL (more on that later)

• It is possible to have an NCR within the process and still have an effectiveness level of 3.

9

Effectiveness?

Extent to which planned activities are realised and planned results achieved

Key Changes: Definitions

Source: ISO9000-2005

Planned Activities?

The means, methods, and internal requirements by which an organisation intends to achieve planned results of a

given process to meet customer requirements.

Planned activities include conformity to process requirements and proceduresThis is how the organization intends to achieve planned

resultsSource AS/EN/SJAC 9101 Rev E

Key Changes: Definitions

Planned Results?

The intended performance of a process, as defined and measured by the

organisation.

Planned results include product conformity and OTD to meet customer requirements, and may include other

elements as defined by the organisation

This is what the organization plans to achieveSource AS/EN/SJAC 9101 Rev E

Key Changes: Definitions

• Designed to aid the auditor in determining process effectiveness level.

• Understanding if appropriate action is being taken is key.

13

CO

NFO

RM

AN

CE

PERFORMANCE

EFFECTIVENESS

Process Evaluation Matrix (PEM)

14

Summary• The OER is no longer required

• Objective Evidence will be recorded on the PEAR and the QMS process Matrix

• Definitions of planned activities and planned results have been added.

• No Required NCR at effectiveness level of “2”

• Forms will no longer be part of the standard and will appear online.

15

Case study and examples

What follows is a case study with an example of the QMS process matrix and a PEAR.

• The QMS Process Matrix will show how additional evidence is collected in lieu of the Objective Evidence Report.

• The PEAR will be presented with two Possible outcomes for the rating. This should allow for a good discussion of the elements of effectiveness.

16RMC Workshop

Long Beach CA July 16-17th 2014

Questions?

17

CASE STUDY EXAMPLES

18

PEAR 1

19

PEAR 1

20

21

PEAR 2

22

PEAR 2

23

Process Evaluation Matrix (PEM)

24

QMS Matrix

25

9101 FORM 2: QMS PROCESS MATRIX REPORT

Clauses (* = not applicable for 9120)

12 Conformity 13 NCR Number

1 2 3 4 5 6 7 8 9 10 11 12

4. Quality Management System

4.1 General Requirements C N/A N/A

4.2 Documentation Requirements

4.2.1 General C N/A N/A

4.2.2 Quality Manual C N/A NA

4.2.3 Control of Documents C C N/A

4.2.4 Control of Records C C N NCR# 2 mi

14 Summary of Objective Evidence: Evaluated Quality Manual #QM0001 dated November 2, 2013 Rev. 3, Evaluated Internal Audit procedure IA-101, documentation reviewed and approved by senior management. Evaluated the following records: (4) Purchase Orders and (5) Receiving Documents/ Certificates of Conformity – R12456, R32290, R65574, R23349, and R22957. Purchase Orders [refer to PEAR reports for detail document numbers]. TEI Database includes all records including the Supplier ASL, Corrective Action Reports. NCR#2 issued to Purchasing process – ASL records not maintained – approved supplier (the Casting Company) not added to latest ASL report.

5. Management Responsibility

5.1 Management Commitment C N/A N/A

5.2 Customer Focus C N/A N/A

5.3 Quality Policy C N/A C

5.4 Planning

5.4.1 Quality objectives C N/A C

5.4.2 Quality Management System Planning C N/A N/A

5.4.3 Safety Objectives (9110 only) N/A N/A N/A

5.5 Responsibility, Authority, and Communication

5.5.1 Responsibility and Authority C N/A N/A

5.5.1.1 Accountable Manager (9110 only) N/A N/A N/A

QMS Matrix

26

9101 FORM 2: QMS PROCESS MATRIX REPORT

1 2 3 4 5 6 7 8 9 10 11 12

5.6 Management Review

5.6.1 General C N/A N/A

5.6.2 Review Input N N/A N/A NCR# 1 mi

5.6.3 Review Output N N/A N/A NCR# 1 mi

5.7 Safety Policy (9110 only) N/A N/A N/A

14 Summary of Objective Evidence: Evaluated the TEI 2013 and 2014 Management Review reporting. Reporting is set up on templates that include the mandatory AS9100C inputs (5.6.1) and the output reporting is also in template format that includes results for each of the mandatory inputs and output (5.6.3). NCR# 3 minor issued – Customer Feedback information is not included in the Management Review Input data or the reporting reviewed (2013 or 2014). Refer to NCR form for specifics on this nonconformance.

6. Resource Management

6.1 Provision of Resources N/A C C

6.2 Human Resources

6.2.1 General N/A N/A C

6.2.2 Competence, Training and Awareness N/A N/A C

6.3 Infrastructure N/A N/A C

6.4 Work Environment N/A N/A C

14 Summary of Objective Evidence: Evaluated the training records for internal auditors. Adequate records – in accordance with TEI requirements for internal auditors. Facility is well maintained, clean and organized. Evaluated room for Supplier Improvement Meetings – adequate resources for these activities.

7. Product Realisation

7.1 Planning of Product Realisation N/A N/A NA

7.1.1 Project Management * N/A N/A N/A

7.1.2 Risk Management * N/A N/A N/A

7.1.3 Configuration Management (7.1.1 for 9120) N/A

N/A N/A NA

Clauses 12 Conformity 13 NCR Number

Thank you

27