Airworthiness
Audit ProceduresAudit Procedures
Presented by COSCAP SA
OutlineOutlineWe will cover :
•Introductions
•Course Objective
•Course Administration and Outline
4
ObjectiveObjectiveONE
•To provide Inspectors with the basic skills and knowledge required to participate as a Team Member in any audit.
5
ObjectiveObjectiveTWO
•To be able to write an audit finding accurately, and in accordance with the applicable Civil Aviation standard and the course material.
6
Overview of the Audit Process
Inspectors HandbooksInspectors HandbooksNote:
The procedures outlined in this presentation
are detailed in Chapter 17 of the Airworthiness
Inspectors handbook:
1 Audit Procedures
2 Policy and Procedures
3 Check Lists
4
AimAim
• To give a description of the audit process.
2
OutlineOutlineWe will cover:
•An overview of Audits
•Definition of an Audit
•Reasons for Conducting an Audit (Types)
•Classes of Audits
•Team Composition and Roles of Members
•Frequency
•Steps in the audit process. 3
What is an audit?What is an audit?
An audit is an in-depth review of the activities of an organization to verify conformance to applicableCivil Aviation regulations and standards.
5
AuditingAuditing
5
The auditing process places the relationship between the regulated organisation and its regulator into the proper context. It reinforces that essential requirement that the responsibility for safety rests with the regulated organisation. If the organisation has effective systematic arrangements for managing safety, it will have little difficulty in responding immediately and thoroughly to an audit. The organisation will be more open about incidents and its failures because its safety management procedures will enforce visibility and due process. It will also want to explain what went wrong and how the matter was dealt with to reassure the regulator.
The 4 basic activities The 4 basic activities during an audit:during an audit:
• Pre-audit• Physical audit• Post audit
6
• Audit follow-up
Audit ProgramAudit ProgramPurpose:
To ensure that a satisfactory level of
aviation safety is maintained, and
that the governments Civil Aviation’s audit
policies and procedures are applied
nationally in a uniform manner.
7
Audit Types:Audit Types:
8
Initial Certification Audit
Approximately six months after the issuance of an Air Operating Certificate or
Approved Maintenance Organization Certificate
Audit Types:Audit Types:Additional Authority Audit
•Prior to granting additional authorities an audit may be conducted.
9
Audit Types:Audit Types:Routine Conformance Audit
Every holder of an Air Operating Certificate,
Approved Maintenance Organization Certificate,
or any other type of aviation organization certificate,
should be audited on a periodic cycle ranging
from six to 60 months.
10
Audit Types:Audit Types:Special Purpose Audits
Called due to special circumstances, i.e.. an accident, safety concerns etc. These audits are sometimes referred to as "No notice Audits".
11
Initial Audit Procedures Course
Audit ClassesAudit Classes•Combined - Operations and Airworthiness conduct audits of their specialties simultaneously under the direction of one Audit Manager.
•Specialist - Airworthiness and Operations audit their specialties separately.
12
Classes of AuditsClasses of Audits
•Large Combined Audit
•Small Combined Audit
•Specialist Audit• Airworthiness• Operations
13
Outputs of an AuditOutputs of an Audit
•Audit Report
•Parallel Report
14
The Audit TeamThe Audit TeamConvening Authority
Audit Manager
Team Leader Ops
Others*
15
Team Leader A/W
Team Member Ops
Team Member A/W
Convening AuthorityConvening Authority
The person responsible for authorizing and
overseeing a regulatory audit
16
Audit ManagerAudit Manager•Plans, organizes, directs, and controls the audit process.
•Required on every audit.
17
Team LeadersTeam Leaders
•Support and assist the Audit Manager.
•Not necessarily required on every audit.
18
Team MembersTeam Members
•Conduct the audit field work
•Document findings
•Generally on every audit
19
ObserversObservers
•Accompany the audit team during the audit.
•Do not participate in the physical inspection unless specifically directed to do so.
•Not necessarily on every audit.
20
STEPS IN THE AUDIT PROCESSSTEPS IN THE AUDIT PROCESS
21
CONVENING AUTHORITY
•Sets the Audit Schedule
•Appoints the Audit Manager
STEPS IN THE AUDIT PROCESSSTEPS IN THE AUDIT PROCESS
AUDIT MANAGER
•Selects the team
•Communicates internally and externally
•Prepares the audit plan
•Establishes contact with the Auditee
•Sets the audit date(s)
22
STEPS IN THE AUDIT PROCESSSTEPS IN THE AUDIT PROCESS
AUDIT MANAGER (con't)
• Initiates the Audit Plan
• Reviews outputs from the audit
• Formulates the Audit Report and Parallel Findings
• Presents report(s) to the Convening Authority
• Ensures that the Audit Report is forwarded to the Auditee. 23
PLANNINGPLANNING
Initial Audit Procedures Course
AIMAIM
• To describe the various parts involved in the planning of an audit in accordance with the applicable Civil Aviation standards
2
Initial Audit Procedures Course
OUTLINEOUTLINE
1. AN OVERVIEW OF PLANNING
2. SCOPE AND DEPTH OF AN AUDIT
3. AUDIT PLAN
3
Initial Audit Procedures Course
AUDIT PLANAUDIT PLANPurpose
•To ensure that the objectives of the audit are met.
4
Initial Audit Procedures Course
AUDIT PLANAUDIT PLAN• MAIN USERS
- Plan Audit Manager and Team will work from
• Logistical details
• SECONDARY USERS
- To Convening Authority for Approval
• Who, When, How Long and Costs– To Auditee
• When, Areas to be audited and Special Requests 5
Initial Audit Procedures Course
Audit of Audit of Acme AeroAcme Aero
The Mind MapThe Mind MapTeam
Selection
Objective
Budget
Schedule
Preps
Scope & Depth
•Travel•Accommodation•Per Diems•Land transport•Budget Contingencies
•Company File Review•Previous Audit Report•Company Manuals•Principal Insp Briefing
Initial Audit Procedures Course
PLANNINGPLANNING
WHY
•Ensures efficiency and economics
8
Initial Audit Procedures Course
PLANNINGPLANNING
Purpose/Function
• Identifies functions
- Shows areas where flexibility is possible
- Shows areas where there is no margin for error
9
Initial Audit Procedures Course
Critical Path PlanningCritical Path Planning
Pre-Audit
Scope & DepthTeam Selection
Budget
DocumentsPreps
Team Meeting
Audit
Entry MeetingPhysical Insp.
Functional AreasFindings
Daily Meetings
Post-Audit
Company De-briefAudit Report
Wrap-up Activities
Follow-Up
Start Finish
Initial Audit Procedures Course
SCOPE SCOPE
11
What is it?
‘SCOPE’ is defined as the number of functional
areas within a company's operation that will
be audited.
Initial Audit Procedures Course
SCOPESCOPEDetermined by the class or type
of audit to be conducted
•Combined
•Specialist
•Special Purpose
12
Initial Audit Procedures Course
SCOPESCOPEFunctional Areas
13
Defined as 28 discrete areas or departments within an organization e.g.:Code TopicA-1 Maintenance Control ManualA-2 Technical PublicationsA-3 Company Quality AuditsA-4 EngineeringA-5 Support/Overhaul ShopsA-6 Aircraft Maintenance Records, etc
Initial Audit Procedures Course
SCOPESCOPEFunctional Areas
• Determined by
- Review OC & OPS Specs to determine size of auditee
- Review AMO’s, Distributors and Manufacturers
- Review Certificates of Approval, A/C limitations, Maintenance and/or Quality Control Manuals for Airworthiness
13
Initial Audit Procedures Course
DEPTHDEPTH
•The period of time over which a company’s operations will be audited, normally from the last audit until present day.
•It could also be thought of as the extent or the size of the audit.
16
Initial Audit Procedures Course
Planning FactorsPlanning FactorsCompany
• Discuss company history with appropriate CAA Inspectors
• Obtain and review previous audit report and follow-up action.
• Review company files (Looking for exemptions and special authorities)
• Obtain and review certificates of approval and limitations (AOC’s etc.)
• Review Manuals (MCM, Ops, Training, Cabin Attendant as applicable)
17
Initial Audit Procedures Course
Planning FactorsPlanning FactorsCompany
• Review accident reports
• Review company changes (personnel, aircraft, facilities, equipment, management, etc.)
• Bases
• Size of company for sampling rationale
18
Initial Audit Procedures Course
SAMPLING RATIONALESAMPLING RATIONALEMRA set out a table for sampling to
95% confidence level
• Confidence level of 95% is identified in the MRA. • In practical terms it may not be realistic to review all aircraft in the fleet. Significant findings against one type of aircraft may require additional review against other types. You must be realistic when conducting the review and look at each audit situation independently.
19
SCIENCE
SCIENCE
Sampling Rates for 95% Confidence
0
1
2
3
4
5
6
7
8
9
Fleet Size SampleRequired
A300-B4
A300-600
A330
737-300
737-400
747-200
747-400
DC10-30
MD11
F28
Initial Audit Procedures Course
Planning FactorsPlanning FactorsBases
•When was the last audit?
•Review the company and compliance history.
•What was found at the bases during the last audit?
•Was there a follow-up? How did it go?
20
Initial Audit Procedures Course
Planning FactorsPlanning Factors
Bases
•How accessible are the bases and is it worthwhile to visit?
•Can we find the information elsewhere?
•What manpower is required?
•Determine risk management.21
Initial Audit Procedures Course
Planning FactorsPlanning FactorsManpower
•Team Members. Are the right people available?
•Will Subject Matter Experts be required?
•Will all Specialties be covered?
•Do I need Support Staff?
•How can I get the best utilization of team members?
22
Initial Audit Procedures Course
Planning FactorsPlanning FactorsDuration
•Size of Company
•Record of Company
•Capabilities/diversification of A/C, etc.
•Scope and function of work carried out
•Distance, location and company scope of work
•Unforeseen circumstances (weather, travel etc.)23
Initial Audit Procedures Course
Planning FactorsPlanning FactorsDuration
•Significant areas of concern will increase sampling & duration.
•Availability of team members
•Loss of a team member (illness, accident etc)
•Experience level of auditors
24
Initial Audit Procedures Course
Audit PlanningAudit PlanningRule of Thumb
•Small 1 - 5 days
•Medium 5 - 15 days
•Large 15 + days
25
Initial Audit Procedures Course
FundingFundingConsider:
•Class of Audit
•The number of people on the team
•The scope of the audit
•Accommodation, meals, etc.
•Foreign travel, Air and ground transportation,
•Equipment/conference room, rentals.
26
Initial Audit Procedures Course
StaffingStaffingConsiderations
•Make an Organization Chart
•Select Team Leaders
•Select Subject Matter Experts
•Select Advisors (if required)
•Determine support personnel requirements
28
Lucky Airlines AMO 58/0100, AOC/40-001
Lucky Airlines AU D IT O R G AN IZAT IO N AL C H AR T1997
A d m in itra tive A ss is tan tM s L u sh
In sp ec to r
In sp ec to r
In sp ec to r
D ep u ty Team L ead erE n g in eerin g
B ob S co tt
In sp ec to r
In sp ec to r
A 3 0 0 , A 3 3 0 ,
In sp ec to r
In sp ec to r
B 7 2 7 ,B 7 3 7
In sp ec to r
In sp ec to r
D C -1 0 , M D -1 1
D ep u ty Team L ead erF lee t A n a lys is
D ick
In sp ec to r
In sp ec to r
D ep u ty Team L ead erS tan d ard s
T H ard
A irw orth in ess Team L ead erJoe H u n t
O p era tion a l Team L ead ertb a
A udit M anagerTed S m ith
C onvening AuthorityD irector, Airworthiness
C pt H arry
AIRWORTHINESSShowing Functional areas to be
audited
A 1IPM
A 2T echnica l Publica tions
A 3C om pany Inte rna l A udits
A 4E ngineering
A 5S upport/O verhaul S hops
A 14C erti fica tion of C om ponents
A 16A W D /S B C om pliance
Eng ineering
A 1IPM
A 2T echnica l Publica tions
A 3C om pany Inte rna l A udits
A 6A irc ra ft M a intenance Records
A 7S am ple A irc ra ft for C onform ance
A 9M aintenance Program s
A 10Re liabi l i ty Program s
A 11M E L
A 16A W D /S B C om pliance
A 19F ue l l ing/Re fue l l ing/ F ire Prevention
Fleet Ana lysis
A 1IPM
A 8Parts /M ater ia l C ontrol
A 12T es t F l ights
A 13M aintenance C ontrac ts
A 15A irw orth iness C ontrol C om m ittee
A 17M aintenance T ra in ing S tandards
A 18L icenc ing C erti fica tion S tandards
A 20F ac i l i t ies /G enera l
Standards A 21S ub-B ases/E n rou te
tba
Audit A reas
Initial Audit Procedures Course
Audit PlanAudit PlanPurpose
•To ensure audit goals and objectives are effectively and efficiently achieved.
29
Communication
AimAim
•To describe the lines of communication which must be established in order to conduct an effective audit.
2
OutlineOutlineWe will cover:
•Lines of communication
3
Lines of CommunicationLines of CommunicationPurpose
•To ensure the audit plan is followed;
•To ensure all team members understand with whom they will communicate;
•To ensure unforeseen problems are dealt with in a timely manner.
4
5
Convening Authority
Audit Manager
TeamLeader
A/W
TeamLeader
Ops
*Others
TeamMembers
TeamMembers
Audit TeamAudit Team
President
DFO CASODirector
ofMaint.
Quality AssuranceManager
C/P DispatchMain BaseAME
Sub-BaseAME
Sub-BaseAME
StoreKeeper
TechnicalRecords
Clerk
CCP Pilots Apprentice Apprentice Apprentice
CompanyCompany
ChiefCabin
Attendant
CabinAttendant
Internal CommunicationsInternal Communications
6
Overview
Internal Communications OverviewInternal Communications OverviewConvening Authority
Audit Manager
Team Leader(A)
Team Leader(O)
Team Member(s)
Team Member(s)
Observer(s) Observer(s)
7
Internal CommunicationInternal CommunicationAudit Manager with:
•The Convening Authority
•Team Leaders
•Team Members
•Observers
8
External CommunicationsExternal CommunicationsCAA Members:
•Observers
•Team Members
•Team Leaders
•Audit Manager
11
External CommunicationsExternal CommunicationsAudit Manager with:
•Auditee
•Airport Manager
•Foreign Affairs
•Media
12
13
External CommunicationsExternal CommunicationsContact with Company
•Typically by phone, then in writing;
•Establish a good relationship;
•Use a non threatening manner;
•Gain acceptance of your audit plan;
•Ask for a contact/liaison person;
•Set meeting dates and times;
•Request facilities for the audit team.
Initial Audit Procedures Course
CommunicationsCommunications
15
The Process
•AM should begin planning
•AM should establish internal contacts as applicable ie. Inspectors etc.
•AM should establish external contacts if necessary
•AM should establish the audit team (team leader. assist.)
•AM should contact auditee by phone and in writing
CommunicationsCommunicationsThe Process cont'd
•Audit Manager should complete planning
•Audit Manager should meet with the team
16
Pre-AuditPre-Audit
Initial Audit Procedures Course
AimAim
To describe the activitiesthat take place during the pre-audit.
2
Initial Audit Procedures Course
OutlineOutline
•Review of Pre-Audit activities
•Topics to be covered during the Pre-audit meeting
•Material to be reviewed during the pre-audit activities
3
Initial Audit Procedures Course
Pre-Audit ActivitiesPre-Audit Activities
•Audit planning
•Team assembly
•Pre-audit team meeting
•Final preparations
including document review
4
Initial Audit Procedures Course
Pre-Audit Team MeetingPre-Audit Team MeetingTopics to be covered
•Materials presented to members
•Duties and Responsibilities
•Specific audit area
•Checklists
•Observer position
•Team plan 5
Initial Audit Procedures Course
Pre-Audit Team MeetingPre-Audit Team Meeting
Other considerations
•Conflict of Interest
•Confidentiality
•Access to Information
•Schedules and Arrangements
•Forms Administration
•Parallel Report
•Budget 6
Initial Audit Procedures Course
Pre-Audit ReviewPre-Audit Review
7
Files held by the Authority
•Company Files
•Aircraft Files by Registration number
•Concession File
•Aircraft Type File
•Company Approvals File etc etc.
Initial Audit Procedures Course
Pre-Audit ReviewPre-Audit ReviewInspection Reports
•In-flight inspection reports
•Ramp checks
8
Initial Audit Procedures Course
Pre-Audit ReviewPre-Audit ReviewPrevious Audit Reports
•Previous history
•Review findings
9
Initial Audit Procedures Course
Pre-Audit ReviewPre-Audit ReviewAll Company manuals
•Maintenance/Operations manuals
•Aircraft flight manual
•Procedures/Training manuals
•Dispatch manual
•MELs
10
Initial Audit Procedures Course
Pre-Audit ReviewPre-Audit ReviewReview Documents
•Flight Permit
•ETOPS
•Fuel policy
•Authorities held (CAT II, RVR600, etc.)
•Exemptions/Concessions
•Approvals/Modifications
•Aircraft Type Approvals
•Manufacturing Limitations, etc. 11
Entry Entry MeetingMeeting
AimAim
•To describe the activities involved in the entry meeting with the company and to better appreciate the climate it may be given in.
2
OutlineOutlineWe will cover:
•Communication Rules
•Topics covered at Entry Meeting
3
ENTRY MEETINGENTRY MEETING Factors that shape outlook
•YOU
- Your task is to provide a service
- Your Authority is often not well understood by industry
•THE OPERATOR
- Often very clever at conducting commercial ventures
- Some view regulations and standards as barriers to expansion and have little time for those tasked with monitoring and enforcing them
4
ENTRY MEETINGENTRY MEETINGFactors (Con't)
•OPERATOR'S PERSONNEL
-View themselves as very good in their job
-They do not want to look bad
•OWNERS
-Have a lot invested in terms of time and money
-Audit may make findings that will be costly to fix
-Many do not like being under government observation
5
ENTRY MEETINGENTRY MEETINGFactors (Con't)
•GOVERNMENT
-Has a lot of say in the company's affairs
-Operator may view your presence as interference
-Time spent during an audit is non-profitable
•CREDIBILITY
-Your reputation may have preceded you
-Don't play games6
ENTRY MEETINGENTRY MEETINGStrategy
•Imagine yourself in Operator's shoes
•Plan and prepare for the audit as much as possible
•Identify issues that could prove difficult and develop a plan to deal with them should they arise
7
ENTRY MEETINGENTRY MEETINGSetting the stage
• Held on company’s premises
• Dress neatly
• Be on time
• Be open and friendly prior to and during the meeting
• Ensure your team is properly briefed
• Ensure operator is properly represented8
Initial Audit Procedures Course
ENTRY MEETINGENTRY MEETINGSetting the stage cont'd
• Ask Operator to hold questions till the end of your address
• One person speaks at a time
• Speak clearly and be brief
• Try to convey your message in a non-adversarial way
• Do not compare one operator to another
• Maintain control of the meeting9
ENTRY MEETINGENTRY MEETINGTopics to be covered
•Thank You's
•Explain the purpose of the meeting
•Introduction of Audit Team and Company Representatives
•Discuss audit objective
•Discuss scope and depth
10
11
ENTRY MEETINGENTRY MEETINGTopics to be covered cont'd
•Discuss communications between the Audit Team and the company, including Daily Meetings
•Discuss way audit is conducted
•Discuss the means of handling non-conformances including immediate threat to safety.
•Introduce forms that the operator may see over the course of the audit
ENTRY MEETINGENTRY MEETINGTopics to be covered cont'd
•Arrange tour of facilities
•Discuss Exit Meeting date and time
•Discuss efforts that will be made to make sure disruption to the operator is very small
12
ENTRY MEETINGENTRY MEETINGTopics to be covered (Con't)
• Confirm
-Working Space
-Photocopier, telephones, facsimile machine, etc.
-Measuring or test equipment
-Admission to facilities and files
• ask questions
• Conclude the meeting in a friendly way13
Physical Physical
InspectionsInspections
AimAim
•To describe the activities and procedures that are carried out during physical inspections.
2
OutlineOutlineWe will cover:
•Overview
•checklists
•Interviewing
•Documenting evidence
•Audit forms
•Corrective action3
What is Auditing?What is Auditing?
•Examining the operator's system, in your assigned area, to determine if it meets the applicable Civil Aviation
regulations and company procedures
as published in the CAA approved
Maintenance Procedures Manual.
5
Steps in Conducting a Steps in Conducting a Physical InspectionPhysical InspectionReview CAA
standards
Obtain checklists
Review and “test”
Does it meetthe standards
Does not OK
Write Notes
Proceed with next
inspection
Write down audit
finding(s)
6
Initial Audit Procedures Course
ChecklistsChecklists
• Use as directed
• Ensure that no area is overlooked
• Write down information
• Check company against applicable Civil Aviation regulation requirements
7
•Maintenance Control/Policy Manual
•Civil Aviation Regulation
•Aviation Act
•MPD/MRB
•Approved maintenance programs
AuditAuditAirworthiness
9
Interviewing PeopleInterviewing PeopleWhy?
• To determine if the system documented in the approved manuals is the system in use
• To assess the knowledge of the operator’s personnel. To review their duties and responsibilities
• To determine if any other areas need be audited
• To tell people what is going on
10
Basics of InterviewingBasics of Interviewing•Ensure no other Inspector has talked to the person about the same thing.
•Where necessary, determine which Team Member will talk to whom
•Be organized
•Be polite
•Do not interrupt auditee. If necessary make an appointment
11
Basics of InterviewingBasics of Interviewing Continued
•Do not threatening
•Explain to Person what you are doing and why
•Check whether it is ok to take notes
•Ask questions carefully
•Listen carefully to the response
12
Basic Do's of InterviewingBasic Do's of InterviewingContinued
•Ask questions another way if necessary
•Ensure you understand the answers
•Ask extra questions as necessary
•Try to discuss
•Remain on track
•End the interview in a friendly manner13
After InterviewAfter Interview
•Make extra notes as necessary
•Tell Team Leader if necessary
14
Do NotDo Not
•Play ‘I have you now’
•Intimidate
•Allow yourself to be intimidated
•Allow yourself to be drawn into fight
15
What if?What if?
•The person refuses to speak to you
•The person wants to say tell you a secret
•The person wants to fight
•The person wants to make a speech
16
EvidenceEvidenceWhat is it?
•Anything which supports or shows a non-conformance.
17
Immediate Threat to SafetyImmediate Threat to Safety
•Inform Team Leader
18
Not an Immediate Not an Immediate Threat to safetyThreat to safety
•What do you do?
19
Audit FindingsAudit Findings& Confirmation requests& Confirmation requests
AimAim
•To describe how to complete the Audit Finding and Confirmation request forms.
2
OutlineOutline
•Overview
•Writing Audit Findings correctly
•Practice exercises
•Test on writing an Audit Finding
•The Confirmation request Form
3
Initial Audit Procedures Course
Audit FindingAudit FindingDefinition
•"audit finding" means the determination of non-conformance of a product, process, practice or procedure or a characteristic thereof to a specified regulation or standard. This will be documented in the Audit Finding Form.
4
5
Steps in Conducting a Steps in Conducting a Physical InspectionPhysical InspectionReview
standards
Obtain checklists
Review and “test”
Does it meet standard
Does not Does
Prepare the paper work
Proceed with next task
Write the audit finding(s)
Writing Audit FindingsWriting Audit FindingsProblems
•The Regulatory requirement has been written incorrectly
•The standard and the example(s) do not match
•The example(s) would support more than one Audit Finding
6
Quoting the CASRQuoting the CASRStandardStandard
7
Exactly
•Write the regulatory requirement in terms of chapter, section, paragraph etc.
•Then incorporate the phrase "Which states"
•Then write enough of the regulation to clearly illustrate the meaning of it.•Tie one part to another by using …..… (dots)
Writing ExamplesWriting Examples•Ensure that the example matches the Regulatory
requirement
•Include a lead-in statement such as "The following are examples of..." where possible
•Write a statement of how the situation was
(past tense)
•Make reference to supporting evidence9
Audit Finding Form
Company Name Base Location File
Area of Audit: FINDING No.
Non-Conformance With:
EXAMPLES
Name and Signature of Auditor Date
Company Corrective Action: Immediate, Short and/or Long Term
Signature/Title Date
Civil Aviation Division Response
Audit Follow up
Target Completion Date Date Item Completed Inspector
AUDIT FINDING FORM
Non-Conformance With:
EXAMPLES
Company Corrective Action
Immediate, Short and/or Long Term
Area of Audit:
Finding No.
Civil Aviation ResponseAudit Follow up
Area of Audit: A1 Maintenance Control Manual Finding No:A1-1
Non Conformance with: SAR 121.137 (a) (1) which states in part - Each certificate holder shall furnish copies of the manual ………..to - its appropriate…...maintenance personnel.
AUDIT FINDING FORM
Non Conformance with: SAR 145.39 (d) which states in part - Each person who is directly in charge of the maintenance functions……... must ……….have had at least 18 months of practical experience………. in the work for which the certificate holder is authorised.
Area of Audit: A14 Certification of Components Finding No:A14-2
AUDIT FINDING FORMArea of Audit: A21 Test & Measuring Equipment Finding No:A21-1
Non Conformance with: SAR 145.47(a) which states - A certificate holder must have equipment and materials necessary to efficiently perform the functions listed on their certificate
Non Conformance with: SAR 145.47(b) which states in part - The organization shall ensure that all inspection tools and test equipment are tested at regular intervals to ensure correct calibration …………...
Area of Audit: A21 Test & Measuring Equipment Finding No:A21-2
Area of Audit: A21 Test & Measuring Equipment Finding No:A21-3
Non Conformance with: SAR 145.47(b) which states in part - The organization shall ensure that all test equipment are tested ………. to ensure correct calibration to a standard derived from the National Standards or to a standard provided by the equipment manufacturer.
AUDIT FINDING FORM
AUDIT FINDING FORMArea of Audit: O15 Flt & Duty Time Records Finding No:O15-1
Non Conformance with: SAR 700.14 (1) which states in part - ‘Every air operator shall establish a system that monitors the flight time, flight duty time and rest periods of each of its flight crew members…...
Non Conformance with: SAR 705.26 (1) Which states in part - Subject to subsection (2), no air operator shall operate a twin‑engined aeroplane on a route containing a point that is farther from an adequate aerodrome than the distance that can be flown in 60 minutes at the one‑engine‑inoperative cruise speed.
Area of Audit: O2 OPS SPECIFICATIONS Finding No:O2-1
Area of Audit: O16 Flight Inspections Finding No:O16-1
Non Conformance with: SAR705.29 (1) which states in part…….flight crew members who are on flight deck duty shall remain at their duty stations with their safety belts fastened...
AUDIT FINDING FORM
Confirmation Request FormConfirmation Request FormPurpose
6
To Reduce Inspector workload during the Initiation and audit phases by requiring the Auditee to prove compliance when non conformance is indicated.
The CRF reduces time consuming document searching, and places the onus on the company to prove conformance.
Confirmation Request FormConfirmation Request FormAdvantages
6
The auditee is afforded the opportunity to prevent aircraft ‘grounding’, and it also ensures that the auditor is possession of all the facts before proceeding with further action. In addition the auditor receives a signed document from the auditee to use as part of the supportive evidence package.
Confirmation Request FormConfirmation Request FormAdvantages (cont.)
6
• where initial evidence indicates a non conformance, the burden is on the company to show otherwise; the auditor makes the final determination based on auditee produced facts as to which direction to proceed;
Confirmation Request FormConfirmation Request FormAdvantages (Cont.)
6
• elimination of arbitrary findings based on subjective examples; the auditee is not faced with major surprises at the end of the audit because all contentious issues have been dealt with in an open manner; and the auditor has more time to devote to the business of auditing rather than chasing a paper trail.
Confirmation Request Form
CRFNo.
Company Name
Name of Auditor Area of Audit
Company Representative Title
Subject Matter
Auditor’s Signature Date and Time
Company Response Required By:Date and Time
Company Response
Company Representative’s Signature Date and Time
For Inspector Use Only
Company Response Accepted Company Response Reject Audit Finding: Yes No
Auditor’s Signature Date and Time
Confirmation Request Form
Company nameAuditors name
Area of audit
Company rep.
Company response
Company response required by:
CRF Number
Subject matter
MeetingMeetingss
DailyDaily
Initial Audit Procedures Course
AimAim
•To be able to conduct daily meetings with the team and the company.
2
Initial Audit Procedures Course
OutlineOutlineWe will cover:
1. Daily Team Meetings
2. Daily Company Meetings
3
Initial Audit Procedures Course
Daily Team MeetingDaily Team MeetingSo Team Leader can:
•Exchange, discuss and validate the day's findings and Confirmation Requests
•Reach agreement on certain items
•Be informed and measure the progress of the audit
•Plan the strategies and take decisions necessary in terms of directing and re-directing personnel
4
Initial Audit Procedures Course
So Team Leader can:
Daily Team MeetingDaily Team Meeting
•Be provided the information that will form the
content of the comments he will make during
the daily meeting with the operator
•Be provided the information to report the
progress of the audit to the AM as necessary
5
Initial Audit Procedures Course
Daily Team MeetingsDaily Team MeetingsTeam Members should brief TL on
•The area(s) they are examining
•A brief summary of what they are finding
•Identify any difficulties they are having
•Offer possible solutions6
Initial Audit Procedures Course
Daily Team MeetingsDaily Team Meetings
TMs should brief TL on cont'd
•A date/time for completion of any phase of their
assignments, (e.g. The technical library
review will be completed by this afternoon)
7
Post AuditPost Audit(Exit) (Exit)
Meeting Meeting
AimAim
•To describe the activities involved in the exit meeting with the company and to better appreciate the climate it may be given in.
2
OutlineOutlineWe will cover:
•Communication Rules
•Topics covered at an Exit Meeting
3
Initial Audit Procedures Course
7
Exit MeetingExit MeetingGeneral
This meeting must not come up with any surprises to the auditee, it will be a summary of the briefing sessions held throughout the course of the audit. If this meeting is a shock to the management it is evidence of procedures not being followed during the course of the audit.
Initial Audit Procedures Course
7
Exit MeetingExit MeetingContent
The speakers must present their findings in a clear and concise manner. The meeting should: be chaired by the audit Manager; be cordial, diplomatic, clear, and polite;take into account the personalities and background
of the company representatives; thank the auditee for their co-operation; summarise audit findings;
Initial Audit Procedures Course
7
Exit MeetingExit MeetingContents (Cont.)
state the areas of strength as well as the areas that are deficient; outline the progression of the report following
the meeting; explain the company’s obligations regarding
the corrective action plan; state how the response system works; and discuss follow-up action.
Audit Report Audit Report
Initial Audit Procedures Course
Audit Report Audit Report Format•Part I
-Overview of the Auditee
-Names and Positions of Management
-Date Audit Completed and Auditee's Name and Address
-Purpose and Scope of the Audit
-Criteria Used
-Audit Team Composition
5
Initial Audit Procedures Course
Audit Report Audit Report (Con't)(Con't)
•Part II
-Executive Summary (which includes a statement of Overall Conformance)
•Parts III and IV
-Functional Summaries (A/W and OPs)
•Appendices
-OPs and A/W Audit Findings
6
FLYRIGHT AUDITEXECUTIVE SUMMARY
(EXAMPLE)AirworthinessThe Airworthiness section of this report indicates that, with the exception of ‘Test & Measuring Equipment’, all audited areas revealed non conformance findings demonstrating:
Failure to fully comply with all applicable regulatory requirements.Ineffective management control of the ‘Maintenance Organisation’.Ineffective system control through the company maintenance control manuals.A lack of effective quality control to ensure satisfactory standards.
As a result, aircraft were operated with: a) inspections and components that had overrun maintenance specification requirements, b) non-approved parts installed, c) defect control and release certifications that did not meet regulatory requirements.
Flyright has therefore operated aircraft in commercial service with Certificates of Airworthiness not in force.
(EXAMPLE)OperationsThe Operations section of this report indicates that audited areas revealed non-conformance findings demonstrating:
Failure to fully comply with all applicable regulatory requirements.Ineffective management control of ‘Training/Check Programs’ to ensure satisfactory safety and regulatory standards.Failure to ensure that the company operations manuals provided effective system control.Failure to maintain adequate control of records.
(EXAMPLE)AIRWORTHINESS
SummariesA1 Maintenance Control ManualsThe hierarchy of the maintenance control manuals approved under the Air Carrier Operating Certificate (AOC) and Approved Maintenance Organization (AMO), requires restructuring. The Certificate Holder utilizes the Aircraft Maintenance Specifications document as the CAA approved, controlling manual. Sub tier manuals include the Technical Procedures Manual, Technical Quality Manual, Quality Assurance Regulations, Reliability Control Program Manual and the Weight and Balance Manual.In practical application the Technical Procedures Manual is the controlling manual, however, this manual has not been approved by CAA.
The Inspection Procedures Manual, utilized by the Approved Maintenance Organization, as with the AOC Technical Procedures Manual, Reliability Control Program Manual, Technical Quality Manual, Quality Assurance Regulations and Weight and Balance Manual, do not meet regulatory requirements.
Numerous ‘controlling’ manuals were found in the system that were not reflected in the IPM or TPM, e.g. Maintenance and Engineering Technical Form Manual, Quality Assurance Regulations, and the Technical Quality Manual.
A2 Technical PublicationsNumerous Manuals in the Main Technical Library are not being maintained to a current status. A review of the document control, for the user departments revealed a serious lack of control. Numerous technical forms were found to be uncontrolled and in one case three different forms had been assigned the same control number.A3 Company Quality AuditsQuality audits were not being consistently carried out of a number of areas, e.g. sub-contractors, out stations, and Engineering Quality Assurance. Staff were unable to provide details of the procedural control of the audit process. A review of the yearly audit plan for 1997/98 indicated that the schedule was not being met. It was noted that the Quality Audit department, encountered difficulty in getting responses to their audit findings.
(EXAMPLE)A4 EngineeringNumerous examples of components manufactured by outside agencies to non-approved data were found. Some of these parts may have been installed on Flyright, and other operator’s aircraft.Examples of non-approved engineering modification drawings and non-approved aircraft modification approvals were found. The Technical Procedures Manual lacked procedures for the approval process of repair data by CAA. Numerous forms were being used in the field; however, these forms were not referenced in the Technical Forms Manual.
A6 Aircraft Maintenance Records Numerous examples of inadequate defect rectification data have been documented. Release certifications by unqualified persons, missing certifications, missing required inspection items (RII, s), lack of reference to standards, missing component information, missing ATA references and missing employee information was evident.
OPERATIONS(EXAMPLES)
O-13 Minimum Equipment List (MEL)
All of the MELs have been approved by CAA. Not all MELs reflect the latest revision of the manufacturing authority’s approved MMEL. Evidence was seen of an item exceeding the MEL time category. Evidence was seen of an aircraft operating outside of the MEL performance limitations. One aircraft did not have an approved MEL on board.
O-14 Dangerous GoodsOperations did not have a copy of the IATA Dangerous Goods Manual and one could not be produced when requested .Notices to travellers, required by carry-on baggage dangerous goods regulations, were not in evidence at ticket counters, security clearance areas or holding rooms.
O-15 Flight and Duty Time LimitationsFlight Crew flight and duty time limitations and rest requirements were not in accordance with regulation 121.Flight Attendant duty period limitations and rest requirements were not in accordance with regulation 121 requirements.Flight Operations Officers duty time limitations were found not to exist.Evidence existed of pilots exceeding regulatory flight time limitations.
O-16 Flight and Ground InspectionNumerous cases were found of AFMs on board aircraft not reflecting CAA approval.Numerous cases were found of AOMs on board aircraft not reflecting CAA approval.Several instances were noted of aircraft not having the current approach plate.Several instances were noted of Flight Attendants not having the appropriate manuals in their possession.
Initial Audit Procedures Course
7
SummariesSummariesBe Professional
•Ensure the Functional Summaries are supported with Audit Findings
•Use the same format throughout
•Write the Summaries in the third person
•Follow the same order as the Audit Findings
•You may consider identifying any repeat non-conformances
Auditee Auditee Corrective Corrective
Action Action
Initial Audit Procedures Course
7
Corrective ActionCorrective ActionTYPES
• Auditee Corrective Action Proposals.– Before the Team can make a final
recommendation as a conclusion of the audit the auditee’s proposals for corrective actions must be received, reviewed and accepted.
– Depending on the nature of the audit findings, the company’s corrective action should involve:
Initial Audit Procedures Course
7
Corrective Action Corrective Action
Immediate Corrective Action
This is action taken immediately upon identification of the audit finding to remove the immediate threat to aviation safety;
Initial Audit Procedures Course
7
Corrective Action Corrective Action Short-term Corrective Action
This action should correct a non‑conformance that does not pose an immediate threat to aviation safety, which ensures that conformance is established quickly until long‑term action is completed to prevent recurrence of the problem. Short‑term corrective action will normally take place within 30 days; and
Initial Audit Procedures Course
7
Corrective Action Corrective Action Long term Corrective Action
This longer‑term action has two components:
1. Identifying the cause of the problem and indicating the measures the company will take to prevent a recurrence. These measures should focus on a system change.
2. A timetable for company implementation of the long‑term corrective action. Long‑term corrective action will normally take place within twelve months.
Initial Audit Procedures Course
7
Corrective Action Corrective Action Supporting Documents
Long and short term corrective action should be accompanied by supporting documents for review eg:
• logbook entries,• purchase orders,• memoranda or revised inspection procedure
cards.Note;It is important to verify as much supporting documentation as possible during subsequent surveillance.
Initial Audit Procedures Course
7
Audit Follow-upAudit Follow-upPurpose
When the Team Leaders are satisfied that all corrective action proposals have been received a full team meeting will be convened to formulate and agree the final recommendations. These recommendations will form a supplement to the Audit Report and should include:
Initial Audit Procedures Course
7
Audit Follow-upAudit Follow-upRecommendations
•an overall recommendation based on the particular objectives of the audit;acceptance or otherwise of the scheduled compliance time for corrective actions;frequency of follow-up to the findings; and period and extent of future surveillance.
Initial Audit Procedures Course
7
Audit Follow-upAudit Follow-upRecommendations
The Audit Manager will present the final recommendations to the Director (for Approval). Subject to approval by the Director the final result of the audit can then be forwarded to the company.
Initial Audit Procedures Course
7
Audit Follow-upAudit Follow-upFollow-upThe Team Leader(s) will control and monitor the follow-up to all corrective actions, using the applicable team member for the corrective action to the Audit Finding that he/she raised.When the necessary follow-up has been completed and is satisfactory the result will be notified to the Director.
CompletiCompletingng
the Auditthe Audit
Initial Audit Procedures Course
AdministrationAdministration
•Submit expenses as applicable,
•Ensure follow-up inspection is scheduled
•Schedule next audit
•Place all information on the audit file
•Return any borrowed equipment
4
Audit Audit CompletComplet
eded