Losa Report

2010

Air Georgian Limited Daniel Bockner Vice President, Flight Operations

AIR GEORGIAN LIMITED – 2010 LOSA REPORT Line Operations Safety Audit report

Air Georgian Limited – 2010 LOSA Report

1


2

Contents Executive Summary ........................................................................................................................ 3

Acknowledgements ......................................................................................................................... 4

ABBREVIATIONS ........................................................................................................................ 5

GLOSSARY ................................................................................................................................... 6

Air Georgian Limited Air Canada Alliance Destinations ............................................................... 7

1. INTRODUCTION ...................................................................................................................... 8

1.1 The Line Operations Safety Audit ................................................................................... 8

1.2 Air Georgian Limited ....................................................................................................... 9

1.3 The Air Georgian LOSA ................................................................................................ 11

2. LOSA Methodology .............................................................................................................. 12

2.1 The Threat and Error Management Model ..................................................................... 12

2.2 Involving the Union ....................................................................................................... 13

2.3 The LOSA Observation Flights...................................................................................... 15

2.4 The LOSA Observation Forms and Database ................................................................ 16

2.5 The LOSA Safety Survey ............................................................................................... 17

3. Findings ................................................................................................................................. 18

3.1 Threats ............................................................................................................................ 18

3.2 Errors.............................................................................................................................. 19

3.3 Undesired Aircraft States ............................................................................................... 24

3.4 Crew Performance Marker Worksheet .......................................................................... 25

3.5 The LOSA Safety Survey .............................................................................................. 26

4. Recommendations ..................................................................................................................... 28

Conclusion .................................................................................................................................... 31

APPENDIX A ............................................................................................................................... 32

APPENDIX B ............................................................................................................................... 40

APPENDIX C ............................................................................................................................... 42

APPENDIX D ............................................................................................................................... 43

APPENDIX E ............................................................................................................................... 45


3

Executive Summary

This report provides a detailed description of the Line Operations Safety Audit (LOSA) that was conducted by Air Georgian Limited during the summer of 2010. The LOSA is an audit of live line flights and follows the methodology set out in ICAO Document 9803. Air Georgian Limited is a Canadian air operator based at Toronto’s Lester B Pearson International Airport. The Company operates a scheduled regional airline service as a Tier 3 partner to Air Canada using BE02 commuter turboprop aircraft. Air Georgian operates flights from both Toronto, Ontario and Halifax, Nova Scotia on segments as short as twenty-five minutes to up to just over two hours. The end result has been an increased awareness and understanding of the many risk factors that can play a role in commercial aircraft operations thus providing a commensurate improvement in safety.

Due to the nature of commercial aircraft operations it is often quite difficult for company managers and individual pilots to gain a comprehensive, quantifiable picture of the many challenges involved in day to day operations. The LOSA is a tool that is designed to address this problem as it allows for the accumulation of a substantial amount of data from regular line flights that can then be studied in detail. This is done by using trained observers who monitor a series of flights and take detailed notes of their observations; the information is then entered into a database. Managers are able to use the database to pinpoint recurring concerns so that countermeasures can be developed.

For the Air Georgian LOSA 111 flights were observed by six trained observers. The observers provided a narrative of the main points noticed during each particular flight. Also, using the Threat and Error Model, they identified each external threat (for example, adverse weather) to the flight, each error by the crew, and any resulting undesired aircraft state. Finally, using a scoring system, the observers applied a numerical value to their assessment of the flight crew’s performance to give an overall rating to each flight.

The data collected during the LOSA has provided Air Georgian with a wealth of valuable information. Using the LOSA data the Company will be able to make improvements to the flight crew training program, improve internal Company policies and procedures, and provide pilots with a better understanding of the many threats that they may have to deal with during day to day operations. The 2010 LOSA will be followed up within the next two years with another full audit to assess the success, or lack thereof, that the Company will have in addressing the issues raised so far.


4

Acknowledgements The LOSA is a large and involved project and requires the dedicated efforts of a number of people. Most importantly, I want to thank the in-flight observers for their hard work:

Captain Tim Crits, Captain James Graham, Captain George Kyreakakos, Captain Rob Maahre, First Officer Patrick Pendergast and Captain Salman Syed.

I would like to thank the other members of the LOSA Steering Committee:

Captain Troy Stephens (Chief Pilot), Captain Dave Ongena (Chief Instructor), Captain Rob Booth (Corporate Safety Officer) and Captain Vito Nobrega (Standards Captain).


5

ABBREVIATIONS ACP – Approved Check Pilot

AGL – Above Ground Level

APG – Aircraft Performance Group (supplier of runway analysis charts)

ATC – Air Traffic Control

BE02 – Hawker Beechcraft Corporation Model 1900D Airliner

CARs – Canadian Aviation Regulations

EFIS – Electronic Flight Instrument System

FPM – Feet Per Minute

GPS – Global Positioning System

GPWS – Ground Proximity Warning System

ICAO – International Civil Aviation Organization

IMC – Instrument Meteorological Conditions

LOSA – Line Operations Safety Audit

MEL – Minimum Equipment List

MDA – Minimum Descent Altitude

NOTAM – Notice to Airmen

OREA – Ontario Regional Employee Association

PF – Pilot Flying

PM – Pilot Monitoring

SOP – Standard Operating Procedures

TCAS – Traffic Collision Avoidance System

TEM – Threat and Error Management

TORA – Take-Off Run Available

UAS – Undesired Aircraft State


6

VMC – Visual Meteorological Conditions

Vref – Reference Landing Approach Speed

GLOSSARY Air Operator – The holder of an Air Operator’s Certificate (a commercial air service).

Captain – The pilot assigned by the Company as the Pilot in Command for a particular flight. To be assigned as the Pilot in Command the pilot must hold the rank of Captain within the Company as this includes substantial additional training and experience.

Error – A non-compliance with regulations, standard operating procedures and policies, or unexpected deviation from crew, Company or ATC expectations on the part of the flight crew.

First Officer – The pilot assigned by the Company as Second in Command for a particular flight. A pilot holding the rank of Captain may be assigned by the Company to act as the First Officer for a particular flight but generally pilots holding the rank of First Officer are assigned to the role.

Minimum Equipment List – A document approved by the Minister that authorizes an operator to operate an aircraft with aircraft equipment that is inoperative under the conditions specified therein.

Snag – An inoperative item on an aircraft. Once discovered the snag is entered into the aircraft Journey Logbook. Depending on the snag an aircraft may or may not be able to remain in service (as per the aircraft Minimum Equipment List).

Standard Operating Procedures – In the context of aircraft operations by an air operator, SOPs are a comprehensive company-issued publication that details the precise procedures that the flight crew shall follow from pre-flight procedures to the post-flight duties.

Threat – An external factor that may increase risk during a flight.

Undesired Aircraft State – An occurrence where the flight crew places the aircraft into a situation that unnecessarily increases risk.


7

Air Georgian Limited Air Canada Alliance Destinations At the time that the LOSA observation flights were carried out, Air Georgian Limited operated flights to the following destinations under the Air Canada Alliance service:

From Toronto (YYZ)

Montreal, P.Q via BDL (YUL)

Kingston, On (YGK)

Sarnia, On (YZR)

Rochester, N.Y (ROC)

Syracuse, N.Y (SYR)

Albany, N.Y (ALB)

Hartford, Connecticut (BDL)

Richmond, Virginia (RIC)

Portland, Maine (PWM)

Allentown, PA (ABE)

Harrisburg, PA (MDT)

Manchester, N.H (MHT)

Grand Rapids, Michigan (GRR)

Dayton, Ohio (DAY)

Providence, R.I (PVD)

From Halifax (YHZ)

Moncton, N.B (YQM)

Charlottetown, P.E.I (YYG)

St. John, N.B (YSJ)

Fredericton, N.B (YFC)


8

1. INTRODUCTION

1.1 The Line Operations Safety Audit Through the summer of 2010 Air Georgian Limited carried out a Line Operations Safety Audit (LOSA). This report will explain the methodology used and the results returned. The LOSA is a tool that was originally developed in the United States by the University of Texas Human Factors Research Project and Continental Airlines. Air Georgian carried out its LOSA independently as a wholly ‘in-house’ exercise (though external expertise was brought in during the preparations) using the methodology prescribed in ICAO Document 9803. ICAO Document 9803 describes the LOSA as ‘a program for the management of human error in aviation operations’ and proposes it ‘as a critical organizational strategy aimed at developing countermeasures to operations errors’1. In essence the LOSA is used to identify threats and errors in the cockpit environment by proactively collecting safety data. Flight crews are closely observed in order to identify the way that they deal with the many threats inherent in the operation of commercial aircraft during day to day operations as well as the errors that all humans make. Data is collected and carefully reviewed so that adjustments and improvements to training and company procedures can be made as applicable. In addition, flight crews can be made aware of common threats and errors so that they may be better able to avoid or deal with them. The LOSA process involves having specially trained observers monitoring regular line flights from a passenger seat. The observers do not participate in any way in the operation of the aircraft and are not members of the crew. During the flight the observer makes careful notes using a specially designed form. A substantial number of flights on all or most company routes must be observed to provide a meaningful data sample. The observation flights should be conducted in a relatively compressed time-frame or changes in weather may create operating conditions that vary too widely. For example, in southern Canada, a LOSA with some observation flights carried out in August and others done in November would return some uneven data due to the different operating conditions between the hot summer months and our rather cold and icy fall weather. Each observer must be fully trained in the LOSA methodology in order to return meaningful data and each should be selected for their intelligence, work ethic, reliability, and general enthusiasm for a complex project of this nature. It is also essential to have the full support of the union or employee association before the LOSA begins; management and the labour leadership must work together to eliminate any concerns among the pilot group regarding the purpose of the LOSA. All company pilots must understand that the LOSA is not about finding individual fault or getting people in trouble for not following procedures and/or policies. Pilots will only participate in a LOSA if they are clear that it is not a game of ‘gotcha’, but rather a useful

1International Civil Aviation Organization, ICAO Document 9803: Line Operations Safety Audit (Montreal: ICAO, 2002), vii.


9

contribution to overall safety; therefore the LOSA is a ‘no-jeopardy’ program for the flight crew. It must also be clearly understood by both management and labour that a LOSA is a voluntary process; flight crews cannot be forced to participate or else the atmosphere in the cockpit will be strained and the data returned may be skewed as a result. In addition, all of the data that is gathered in a LOSA must be completely de-identified. Flight Crew members must be comfortable in the knowledge that there is no way that any of the information gathered during the observation flights could be used against them in any way. With the Air Georgian LOSA management had absolutely no knowledge of the names of the flight crew members operating any of the observed flights and had no way of gaining this information. Managers must completely resist the temptation to identify flight crew members, even when an observed flight was clearly not operated in accordance with company policies and/or procedures. Any failure to do so would poison the process and render any future LOSA within the company impossible. Rather, it is far more effective for management to learn from the deviations that were observed during the LOSA and develop effective systemic counter-measures that will be useful for years to come rather than to try to discipline individuals. The purpose of this report is to provide Air Georgian management with a comprehensive breakdown of the results returned by the LOSA in order for the Company leadership team to put in place corrective measures as required. The list of recommendations at the end of the report is meant to give managers a set of objectives that, if met, will address the shortcomings within our operation that were noted during the LOSA. However the list is not necessarily exhaustive and, using this report and the raw data contained in the Air Georgian LOSA database, managers may well come up with additional goals. It is also hoped that individual Company pilots will take the time to digest the information contained in this report as well. Finally, the Air Georgian LOSA report contains information that is likely relevant to other operators in the regional airline world.

1.2 Air Georgian Limited Air Georgian Limited is a Canadian air operator with a fleet of fifteen Beechcraft 1900D (BE02) turboprop commuter airliners as well as four business jets. Air Georgian Limited operates fourteen BE02s on behalf of Air Canada under the Air Canada Alliance name. At the time of the LOSA twelve BE02s were based in Toronto, Ontario and two were based in Halifax, Nova Scotia; crews based in both cities participated in the audit. Air Georgian Limited has extensive experience operating the BE02 and has been doing so since 1996. The company has been flying on behalf of Air Canada since early 2000 and carries out over 25000 flights per year (not including the charter aircraft). Only crews operating the BE02 on Air Canada Alliance flights were involved in the LOSA. Air Georgian, as a regional turboprop operator, has a relatively young pilot group consisting of 120 flight crew. Pilot employees stay with the company for an average of four years before being hired by major Canadian airlines such as Air Canada. Air Georgian hires First Officers with a


10

minimum of 1500 hours of total flying time and the minimum for a Captain upgrade is 2700 hours with 500 on the BE02. Earlier in its history Air Georgian was able to hire most of its pilots from other commuter or air taxi operations. Pilots from these backgrounds often have experience flying in the fairly demanding conditions common in the many sparsely settled areas of Canada. As the pool of available pilots has diminished in recent years Air Georgian has been forced to hire more of its flight crew directly from the ranks of flight instructors. This is unfortunate as pilots whose only professional employment is limited to instructing have a far narrower experience base to draw on. Having said that, Air Georgian provides comprehensive training and all of our pilots, regardless of background, bring themselves up to and maintain an acceptable standard. The company also has a tightly controlled mentorship program where a small number of pilots (a maximum of six) are hired directly from the aviation programs of either Seneca College or the University of New Brunswick with a minimum of 250 hours of flight time. The programs run by both of these schools are highly regarded within the industry and produce excellent entry-level candidates. Air Georgian pilots fly an average of approximately seventy hours per month and are provided eleven guaranteed days off per month. The typical duty day is approximately nine hours though days can be scheduled up to fourteen hours. The corporate culture at Air Georgian revolves around safety. From the owners of the Company down through the CEO and Accountable Executive safety is always paramount. Air Georgian is in business to be profitable but never by accepting unsafe conditions or even unnecessarily increased risk. This message is passed on to the employee group from their first day of work with the Company and is reinforced at every opportunity. The regulations and industry best practices are followed to the letter and, in many aspects of our operation, Air Georgian goes well beyond the minimum standard that is required. For example, Air Georgian pilots are provided with substantially more training time and Line Indoctrination than is required by the Canadian Aviation Regulations (CARS); we also have a line check program for our pilots despite the fact that it is not a requirement for a commuter operator. Air Georgian/Air Canada Alliance flights operate sectors that range in length from twenty five minutes to over two hours (120 to 460 miles). Flights based in Toronto serve medium sized markets, mostly in the northeastern United States, such as Albany New York, Portland Maine, Manchester New Hampshire etc… Halifax based aircraft fly to domestic destinations within the Maritime Provinces and most of these flights are relatively short. Almost all Air Georgian/Air Canada Alliance flights operate to and from their respective hubs (Toronto and Halifax) rather than carrying on to different stations in the course of the day. Only three of Air Georgian’s BE02 aircraft are equipped with autopilots and all aircraft come equipped with GPWS, TCAS, GPS and dual EFIS screens for each pilot.


11

1.3 The Air Georgian LOSA The LOSA at Air Georgian was conducted using six in-flight observers and was overseen collectively by the LOSA Steering Committee. The LOSA Steering Committee consisted of the author (the Vice President, Flight Operations and also the Operations Manager), the Chief Pilot, the Chief Instructor, the Corporate Safety Officer, the Standards Captain and the six in-flight observers. The six in-flight observers were all non-management BE02 pilots; one was an ACP, two were Line Indoctrination Captains (including the ACP), three were line Captains and one was a First Officers. Five of the observers were based in Toronto and one in Halifax. Prior to conducting the in-flight observations the in-flight observers were given four days of training in the LOSA methodology. Detailed instruction was provided to them on the Threat and Error model, the forms used for the observations and specific details about the observation process among other topics. Two days of the training was provided by Dr. Robert Baron of ‘The Aviation Consulting Group’ (based in Myrtle Beach, South Carolina). The LOSA Steering Committee met on a number of occasions to work out the various details involved in carrying out the audit. One of our main challenges was the size of the aircraft; being a small, eighteen passenger seat commuter aircraft, the BE02 is not equipped with a spacious cockpit. The aircraft does not have a jumpseat in the cockpit and the observers had to sit in passenger seat 1B. Extenders for headsets were provided to the observers so that they could plug in and listen to the cockpit intercom and the VHF radios during each flight. Unfortunately, from seat 1B, the observers were faced with a difficult line of sight into the cockpit. It was possible to see almost all of what was going on but this required contortions that made for a fairly uncomfortable series of flights for the observers. Despite this problem the observers were able to effectively monitor the vast majority of the goings on in the cockpit and the reports returned by them contained an enormous amount of valuable information. The targeted number of usable LOSA observation flights was 120 (twenty for each observer) though, due to some issues that came up over the course of the observation flight schedule, we were only able to accomplish 111. Still, 111 observed flights represents an acceptable sample and consists of approximately 5% of the Air Canada Alliance flights that Air Georgian operates in a given month. Of course one must bear in mind that each crew was observed twice (once for the outbound leg and once inbound); therefore a total of fifty-six crews were observed during the audit.


12

An Air Georgian Limited Beechcraft 1900D (BE02)

2. LOSA Methodology

2.1 The Threat and Error Management Model The Threat and Error Management model (TEM) was developed at the University of Texas and is best described as the aviation equivalent to ‘defensive driving for a motorist’2. In essence “the model posits that threats and errors are integral parts of daily flight operations and must be managed”3. With the TEM model the many threats, errors and undesired aircraft states that may be encountered on any given flight are defined and flight crews are trained to deal with them. It is also made clear to flight crews that most threats, errors and undesired aircraft states are regular occurrences that must be dealt with on a daily basis rather than abnormal situations. In the TEM model a threat is something that may increase risk during a flight but it is external and thus beyond the control of the crew (for example, an aircraft maintenance issue, significant weather, company schedule pressures etc…). An error is defined as a “non-compliance with regulations, Standard Operating Procedures (SOPs) and policies, or unexpected deviation from crew, company or ATC expectations”4. An undesired aircraft state is said to occur whenever the flight crew puts the aircraft into a situation that unnecessarily increases risk; this can be done either

2 Ashleigh Merritt, Ph.D. & James Klinect, Ph.D., “Defensive Flying for Pilots: An Introduction to Threat and Error Management”, Flight Safety Foundation website, http://flightsafety.org/archives-and-resources/threat-and-error management-tem. 3 International Civil Aviation Organization, ICAO Document 9803: Line Operations Safety Audit (Montreal: ICAO, 2002), 2-1. 4 Ibid 2-3


13

through the actions of the flight crew or the lack thereof. The TEM model provides a simple to understand but comprehensive method of categorizing and quantifying the many threats, errors and undesired aircraft states that can be encountered on any given flight. At the time that the LOSA was conducted formal TEM training was not a part of the training curriculum provided to Air Georgian pilots. Many aspects of the TEM were present in the Company pilot training curriculum but the overall concept was not specifically trained. The LOSA made it clear that the TEM concept is a valuable tool and one that should be thoroughly taught to all Company pilots.

2.2 Involving the Union It is essential to the success of the LOSA that pilots feel comfortable participating. There must be a high level of trust between the pilot group and management or flight crews will not be interested in having an observer sitting behind them making notes while they do their jobs. Pilots must feel assured that the LOSA is not simply a game of ‘gotcha’ in which management is trying to shift blame for operational problems into the cockpit. One of the most effective ways of doing this is by involving the pilot union or employee association in the LOSA from the earliest planning stages. At Air Georgian management and labour generally have a good working relationship. There will always be disagreements between management and labour as no marriage is ever perfect. However, the Company has never had a work disruption due to labour issues and both Company management and the labour leadership are dedicated to creating a fair and equitable workplace while still focusing on safety and profitability (in that order). At Air Georgian the Ontario Regional Employee Association (OREA) represents all Company employees though by far the largest group of employees are pilots and most of the Association leadership are pilots as well. The President of OREA was consulted at the start of the LOSA process and he immediately saw the value in it and provided his full support. The labour leadership appreciated the fact that their input and support was sought out and they were invaluable in allaying any concerns that some members of the pilot group brought to their attention. Two weeks before the first series of observation flights the following memorandum was issued to the pilots providing a full explanation of the LOSA and making it clear that the audit had the full support of the employee association:


14

Line Operation Safety Audit

Pilots,

Beginning in mid-July and continuing for about four weeks, Air Georgian will conduct a Line Operations Safety Audit (LOSA). For this audit we will use Air Georgian pilots to conduct in-flight observations. The in-flight observers will sit in seat 1B and will be plugged into the intercom for a full turn. Each in-flight observer will conduct a number of turns during the audit period. During the audit period approximately 60 turns (120 flights) will be observed.

LOSA has been in use since the mid-1990s and is recognized world-wide as an effective safety tool. Numerous airlines, including Air Canada, have conducted LOSAs with very positive results. To our knowledge Air Georgian will be only the second airline operating turboprop commuter aircraft in the world to carry out a LOSA. LOSA observations are no-jeopardy events, and all data are confidential and de-identified. LOSA data will be deposited into a data-base by the in-flight observers and none of the information regarding operating crew particulars will be provided to management. The in-flight observers will themselves select the flights that they will observe. All of the in-flight observers are non-management line pilots. The in-flight observers will not be there to critique your performance – their mission is to be an unobtrusive observer and to fill out data collection forms after the flight is completed. Once all of the LOSA observation flights have been completed and the information has been entered into the database by the in-flight observers the LOSA Steering Committee (consisting of the in-flight observers, myself, Troy Stephens, Dave Ongena and Rob Booth) will review the data and put together a detailed report along with recommendations that will be made available to the pilot group. The data, once it gets to the Steering Committee, will have been de-identified and the committee members will have no idea who the crews involved were or the dates of the observation flights. The recommendations will possibly lead to changes or additions to Company policies, procedures and training.

The purpose of the audit is to identify problem areas that affect our operation so that awareness and, where possible, countermeasures can be developed. There are three main areas that will be looked at, though the scope of the LOSA will not be limited to these. First, external threats caused by ATC, our ground handlers, airport authorities etc… Secondly we will look at the performance of our own crews to identify potential problems with training, comprehension of procedures and adherence to SOPs and policies. Third, we will be able to determine the effectiveness of SOPs and company policies and see where changes for the better can be made.

The LOSA is being carried out with the full knowledge and approval of OREA. We would like to stress again that the LOSA observation flights are no-jeopardy and will be completely de-


15

identified; there is no need for the flight crew to be concerned about failing anything or being reported to management for making errors. We strongly urge all Air Georgian pilots to participate if an in-flight observer requests that he be allowed to observe your flight. However, it is the right of the crew to deny access to the in-flight observer should they so desire. Again, the LOSA will help to improve safety for all Air Georgian pilots and for our passengers so we hope to have a high level of participation from the pilot group.

Please don’t hesitate to contact me if you have any questions about the LOSA.

Thanks,

Dan Bockner

2.3 The LOSA Observation Flights The six LOSA observers were each given seven straight days to carry out their observation flights. The observers were taken off of the normal flight schedule two at a time. On their first day each pair of observers monitored two Air Canada Alliance turns (from the hub to an outstations and back). On their second day the in-flight observers both brought their first day’s audit forms into the office for review. At this point any misunderstanding or questions that the observers had were addressed. The observers then continued with their observation flights over the subsequent five days doing two turns each day. The observers chose the flights that they wished to observe and used the following guidelines:

Over the course of their LOSA week they were to observe flights to as many different destinations as possible;

The observers had to coordinate with Crew Scheduling to check flight loads and to be listed on the flights that they chose. Obviously only flights with open seats could be observed (as mentioned above the BE02 does not have a jumpseat in the cockpit);

The observers were listed on two flights at a time in case they were turned down by the crew of their first selected flight;

Management had no involvement in the selection of the observation flights and had no knowledge of which flights were being observed.

Once at the gate the observer would approach the flight crew and inform them that their flight had been selected for the LOSA. A brief explanation of what was to take place was provided to the Captain and the First Officer and both were informed that the LOSA was voluntary and they could feel free to deny boarding to the observer (over the course of the LOSA only one crew


16

refused to participate and denied boarding to the observer). Once the flight crew agreed to take part in the LOSA the observer would board the flight with the rest of the passengers. As mentioned above, the observers sat in seat 1B on the BE02. During their observation flights they remained plugged in to the cockpit intercom system and could thus hear all communication between crew members and with ATC.

Due to the nature of our operation the observer was not able to observe the flight crew as they prepared for the flight. This was most unfortunate as the pre-flight preparation process is very important to the successful completion of a safe flight. However, over the course of the LOSA flight the observers were generally able to determine how well the flight crew had prepared for each particular flight. The observers were under strict instructions not to interact with the flight crew in any way once they had completed their introductory briefing, though they were told to review the flight crew’s paperwork at some point during or immediately after the flight. During the flight the observers made very detailed notes and these were later used to fill out the LOSA observation forms. Some of the observers brought laptops though it was quickly determined that this was not practical as these had to be stowed during some of the most noteworthy segments of the flights. Once their first turn of the day was completed the observers took a meal break and then set off for their second turn. They generally completed the LOSA forms (transcribing their notes) each evening once they had completed their flights for the day.

2.4 The LOSA Observation Forms and Database For each LOSA flight specially designed LOSA forms were filled out. The forms used in the Air Georgian LOSA were very similar to the sample forms provided in ICAO Document 9803 and in FAA Advisory Circular 120-90 (a copy of the LOSA form is attached in Appendix A). Nothing that is written on the form provides any indication of the identity of the flight crew. One set of forms is filled out for each leg and this was either done in paper format or electronically at the discretion of the observer. The LOSA form allows for a detailed description of each flight. The observer notes some basic information about the flight such as the city pair, the pilot flying (either the Captain or the First Officer, without naming names of course), the general weather conditions, among other things. The next section on the form provides space for a detailed narrative divided into phases of flight. The observers were instructed to provide a general description of the overall flight situation (i.e. any delays or other stressors) as well as an idea of the working atmosphere in the cockpit (i.e. open communications, stressed, etc…). For each phase of the flight the narrative was to include any noteworthy events that could be defined as threats, errors or undesired aircraft states; as much detail as possible was required. Below the narrative section are boxes for threats, errors and undesired aircraft states. The final section of the form is the Crew Performance Marker Worksheet; here the observer is able to provide an assessment of the overall flight by providing marks on a number of different topics in all phases of flight.


17

The completed forms were then entered into a database by the observers, who were provided office time to perform this task after all of the observation flights had been completed. The information was inputted into the database on a digital form that was identical to the LOSA forms used by the observers. The database is fully searchable and queries on single items or combinations can be made. For example, a query searching for all LOSA forms with the Captain as pilot flying, between Toronto and Albany with adverse weather present could be made and the returned forms could then be reviewed and compared. It is very important that the database be flexible and simple to use as it will be a valuable tool for the Training Department and the Safety Officer far into the future.

2.5 The LOSA Safety Survey The final component of the LOSA was the safety survey. A survey was created that was distributed to the pilot group by the LOSA observers. This was not done at the time of the observation flights because, due to the nature of our operation, the flight crew would likely have been over-tasked had they been pressed to do a survey just prior to or after operating a revenue flight. The safety survey consisted of ten questions that covered the respondent’s experience level and his/her assessment of the level of safety within the Company’s operation. The survey was deliberately kept short in order to increase the number of pilots who would bother to participate. Each copy of the survey was handed out to pilots by a LOSA observer who then, when possible, waited while the individual completed it. As with the observation flights, the survey was completely de-identified and there was no way for management to track who had written what. Some of the feedback from the survey will be included below and the survey questions are included in the appendix. The survey did not include any multiple choice answers as it was determined that written responses were more desirable in order to receive the information in the words of the pilots themselves. The possible downside to this approach is that some pilots could use the survey to air non-safety related complaints; this did turn out to be the case with some of the responses. However, filtering out the non-safety related issues is a fairly simple matter. The completed surveys will be kept for long-term reference so that the information and opinions gathered today can be compared with the concerns that Air Georgian pilots may raise in the future. It will be interesting to see if/how the pilot perception of safety within the Company may change over the next couple of years (until the next LOSA).


18

3. Findings

3.1 Threats Many of the threats encountered during the observed flights involved everyday airline issues and these were generally well-handled by the pilots. In the summer months extensive thunderstorm activity is quite common in the area of operations that Toronto based Air Georgian crews fly in. For Halifax based Air Georgian crews fog is quite common in July and August. Delays, especially in Toronto, are also a regular occurrence for a variety of reasons and Air Georgian crews take them in stride for the most part. However, there were delayed flights where the observers noted that the crew seemed quite rushed. Rushing is highly undesirable and leads to an unnecessary increase in the level of risk. These crews may have felt self-imposed pressure to make up some of the time lost due to the delay.

Taxi instructions in Toronto were misunderstood by the flight crew on a number of occasions. Toronto is a large, busy international airport and taxi routes can involve a number of turns onto different taxiways and across runways. The observers noted instances where ATC used non-standard or otherwise unclear phraseology that caused some confusion among the crew. There were also missed calls during the taxi due to frequency congestion. In some cases the Air Georgian crew simply misunderstood or did not hear a clearly conveyed instruction or clearance.

ATC was noted as a threat on a considerable number of the observed flights. Generally the ATC threat involved congested frequencies though the use of non-standard phraseology (as mentioned in the previous paragraph) and unclear transmissions were also noted as ATC threats.

Air Georgian aircraft are maintained to a very high standard by our professional maintenance staff. However, due to the realities of airline work, aircraft are often dispatched with deferred items. Air Georgian management insists that flight crews follow the MEL to the letter thus mitigating most of the risk involved. However, there were instances during the observed flights where risk, while still being acceptable, was heightened due to the deferral of one or more aircraft system; the most substantial item being the pressurization system.

One surprising threat that came to light from the observations was the scheduling of two Captains together as a crew. When two Captains flew together the observers noted that cockpit discipline was generally substantially lower than when a Captain and First Officer were paired together. For the most part the Captain listed as the Pilot in Command was too deferential to the when another Captain was acting as Second in Command. Also, in some cases, the Captains acting as Second in Command had not flown from the right seat for some time and were not as familiar with the right seat duties as they should have been. However, the real threat here was the heightened level of complacency involved; two Captains often seem to be far too relaxed when operating together, thus heightening the level of risk.


19

The repetitive nature of regional airline flying is in itself a threat to a certain degree. On the one hand risk is lowered by the fact that crews quickly become very familiar with the routes and destinations. Unfortunately the routine and the generally simple operating conditions involved with flying in southern Canada and the northeastern quarter of the United States breeds complacency. Almost all of the crew errors noted below involved a complacent attitude among the flight crew. For example, a charter crew taxiing at an unfamiliar airport would almost certainly not taxi at an excessive speed; this is something crews only do when they know their way around an airport. The lack of compliance with the sterile cockpit SOP also indicates that a complacent attitude is fairly widespread though it varies in degree among individuals.

The above chart shows the main threat types and the percentage of flights where these were encountered (some flights encountered multiple threats).

3.2 Errors Over the course of 111 flights (the number of observed flights during the LOSA) there are bound to be some noteworthy events. No operation or group of pilots is perfect; hence the reason for the LOSA in the first place. It is natural that some of the observed events will be more significant than others and this section will start with a list of the most unusual occurrences noted by the observers. These are events that could be considered to have increased risk though there were no serious consequences in any of these cases.

On a YYZ-ROC flight thunderstorms were active along the route of flight. The crew determined once airborne that the weather radar was malfunctioning. Upon arrival in Rochester the crew did

Adverse Wx ATC Airport Condition

Gnd and Air Traf

Airline Ops Pressure

A/C Mtce + MEL

0

5

10

15

20

25

30

35

THREATS

THREATS


20

not enter the snag into the journey logbook or take any other action (the aircraft should have been grounded in Rochester). They called for boarding as quickly as possible and departed on the return flight back to Toronto. In this situation the crew violated the Canadian Aviation Regulations (CAR 704.64) by knowingly operating an aircraft with the weather radar inoperative with forecast thunderstorms along the route of flight.

During a YYZ-DAY flight the crew failed to properly check the NOTAMs. The Dayton International Airport was closed at the flight’s scheduled arrival time due to an air show. The crew only discovered this on the descent into Dayton when ATC pointed it out. The flight had to hold for more than a half an hour. The crew clearly failed to do the proper flight planning prior to departure. It should also be pointed out that the Flight Following office also missed this NOTAM. At the time of the LOSA Air Georgian used a Type C pilot self-dispatch system thus making the Captain fully responsible for all flight planning. Having said that the Company should have been aware of the temporary airport closure.

On a BDL-YYZ flight the crew activated the wrong flight plan in the GPS (BDL-YUL instead of BDL-YYZ). They did not properly check the loaded flight plan as per the ‘Before Take-off’ checklist and only discovered the error when ATC intervened once en route.

Prior to starting the approach during a YHZ-YFC flight the crew briefed a Pilot Monitored Approach (PMA). As per the SOPs the First Officer is to fly the aircraft during the approach and, if the runway is seen prior to minimums, the Captain takes over control to land the aircraft. On the flight in question the First Officer flew the aircraft all the way to landing. The failure to properly follow the PMA procedure increased the risk of confusion over which pilot had control of the aircraft at a critical phase of flight at approach minimums.

On a YHZ-YFC flight the aircraft assigned to the flight had a deferred pressurization system that was properly logged. The crew was aware of the deferral yet they still opted to fly at a cruising altitude of 12000 feet. This is a relatively short flight (forty-five minutes) and the increased fuel consumption at an altitude below 10000 feet was not a concern (nor were there any other factors that made 12000 feet more desirable than an altitude below 10000 feet). In addition, the crew did not start a timer once they climbed above 10000 feet to comply with the CAR 605.32 allowing no more than thirty minutes unpressurized at an altitude above 10000 feet (to a maximum of 13000 feet). No briefing was made regarding the possible adverse consequences of flying above 10000 feet in an unpressurized aircraft. The decision to operate the flight above 10000 feet needlessly increased the crew’s susceptibility to hypoxia and may have violated the CARs.

On a MHT-YYZ flight the aircraft pressurization system malfunctioned and the aircraft did not pressurize during the climb. The crew caught the fault well before they climbed above 10000 feet (and before the ‘Cabin Alt High’ annunciator illuminated) and they briefed a lower altitude for the flight back to Toronto once they had determined that the aircraft had sufficient fuel on-board for the higher burn rate. Unfortunately the crew did not enter the snag into the journey logbook and, upon arrival in Toronto, the aircraft was left on the gate with an undocumented open snag (this is a violation of 3.15 of the Company Operations Manual as well as a number of CARs).


21

A flight departed from YZR for YYZ and the crew opted to take off with a tailwind (Sarnia is an uncontrolled airport). The crew did not refer to the runway analysis chart for the tailwind departure as per SOPs. This despite the fact that the runway is relatively short with a TORA of 5100 feet. There is a significant weight penalty per knot of tailwind for this runway.

During the cruise climb portion of a YYZ-MHT flight the observer noted that both crew members were reading magazines. The aircraft was equipped with an autopilot and, during the climb, the vertical speed decreased to 300 FPM without either crewmember noticing. Although a 300 FPM vertical speed is not unsafe it is operationally undesirable and should have been observed and corrected by the crew. Also, had it been the speed that had been decreasing (if, for example, the crew had selected the incorrect climb mode) without being noticed by the crew, the situation would have quickly become dangerous. In addition, it should go without saying (and it is a violation of Company policy) that it is unacceptable for both crewmembers to be engaged in non-operational activities while operating a flight.

A number of serious issues were noted that needlessly increased risk and which took place repeatedly during the series of LOSA observation flight such as:

Excessive taxi speeds. On a surprising number of flights the observers noted high taxi speeds (above the twenty knot maximum allowed by the BE02 SOPs). Air Georgian has very clear taxi SOPs and the hazards of taxiing too fast are pointed out in Company literature. All Air Georgian flight crews are made aware that rushing (in any form) is never acceptable and yet high taxi speeds seem to be relatively common.

From the data returned on the LOSA forms a large number of crews ignored the sterile cockpit SOP; casual conversation during critical phases of flight seems to be endemic. Since the findings from the February 2009 Colgan Air crash in Buffalo began to emerge Air Georgian has regularly publicized the hazards of ignoring the sterile cockpit SOP to flight crews. Apparently these efforts have yet to yield concrete results. Flight crews often discussed non-operational topics or engaged in non-operational activities at inappropriate times and the result during the observed flights included repeatedly missing radio calls from ATC, forgetting checklists and minor losses of situational awareness. As the Colgan Air crash demonstrated, ignoring the sterile cockpit SOP can lead to far worse consequences as well.

There were a number of cases of crewmembers doing their checklists from memory. At Air Georgian we make it very clear that checklists must be properly read by the Pilot Monitoring; doing a checklist from memory is unacceptable as it regularly leads to errors.

On a small number of flights the attitude of the Captain was deplorably lax. There are clearly a small minority of pilots who enjoy flaunting rules and procedures and who make a show of doing so. On flights operated by these Captains the level of risk is raised unnecessarily from start-up to shut-down.


22

The LOSA data has uncovered less significant, but still notable, crew related errors. All of these findings were noted on numerous flights and point to items that flight crews must pay more attention to and that the Company must make more of an effort to stress during training.

Flight crews often did not check their runway analysis charts when required to (Air Georgian uses APG charts). During training and line indoctrination Air Georgian pilots are instructed to refer to the APG charts for each take-off and landing. While it is certainly possible that crews had indeed checked the APG charts before the LOSA in-flight observer joined them it is extremely unlikely that this occurred in all, or even the majority, of cases where the observer noted that the charts had not been checked.

In a significant minority of the observed flights the briefings provided between crew members were lacking. Important information was often left out of the briefings and, in a few cases, required briefings were not made at all.

The KLN-90B GPS is the primary navigation system on all Air Georgian BE02s. The BE02 SOPs require that the GPS be backed up by traditional ground based navigation aids though this requirement was ignored on the vast majority of observed flights.

The BE02 SOPs require that crews brief one emergency or abnormal situation as a refresher prior to, or during, the first flight of each pairing. This SOP was regularly ignored during the observed flights.

The TCAS mode is often not set properly during flight (note that this also involves a check list flaw).

Crews often started and shut down aircraft with the EFIS power and/or the Avionics Master on despite repeated direction from management not to do so as this procedure may cause damage to the avionics.

The PF is required to make a 1000 foot call prior to level off in climbs and descents while the PM is required to make a 100 foot call prior to level off in climbs and descents; these calls are often missed.

A number of floated landings were observed due to the PF landing with power on instead of idle power (as per the SOPs).

The PM was often observed performing non-essential duties (such as filling out the journey logbook, making Company radio calls, or other non-essential paperwork) at inappropriate times when his/her attention should have been focused on the operation of the aircraft (i.e. during the taxi and while in busy terminal airspace). While these are more examples of violations of the sterile cockpit SOP, these tasks are at least operational in nature.

There were many cases where the PF executed duties that are assigned to the PM by the SOPs. This can cause confusion between crew members and distracts the PF from actually flying or taxiing the aircraft. There is almost never any need for the PF to execute PM functions during a flight. This took place most often when the Captain was the PF.

Crews often failed to play the automated turbulence briefing prior to entering turbulent air. Passengers were thus not given any warning upon encountering turbulence.


23

The one passenger incident that occurred during the observed flights took place in Rochester. The flight had arrived at the gate and a ‘senior Air Canada employee’ insisted on being allowed off of the aircraft immediately. The BE02 SOPs require that the air stair door not be opened until the ground crew is prepared. However, the crew seems to have been intimidated into opening the door and allowing the passenger off of the aircraft at an inappropriate time. The proper procedure would have been to instruct the passenger to remain seated until the door was opened at the appropriate time. In addition, the crew should have contacted management as soon as practicable to provide information about the totally unacceptable behavior exhibited by the non-revenue passenger.

The above chart shows the main error types and the percentage of flights where these were encountered (some flights had multiple errors).

It should be stressed that the vast majority of errors noted were very minor. Perfection is impossible regardless of who is in the cockpit; the point of the above information is to make pilots aware of the types of errors that occur so that individuals and the Company as a whole can work to reduce the frequency of mistakes.

0

10

20

30

40

50

60

ERRORS

ERRORS


24

3.3 Undesired Aircraft States Almost all of the undesired aircraft states that were noted by the in-flight observers during the LOSA were related to crew errors, as one would expect. The undesired aircraft states that were encountered were all relatively minor and most were small deviations that were corrected almost immediately. However, the data returned is quite valuable as it will help the Company to tailor training so that times and locations where increased vigilance is required can be pointed out to pilots.

The UAS that gives the most concern involves unstable approaches. The Air Georgian BE02 SOPs define a stabilized approach as follows:

“The aircraft shall be stabilized on the approach by 1000 feet AGL in IMC and by 500 feet AGL in VMC.

The criteria for a stabilized approach are as follows:

(i) In the landing configuration with the Before Landing Checklist complete

(ii) Indicated airspeed within plus 10 knots to minus 5 knots of target airspeed. Approach speed should be maintained until the missed approach point, then reduced to Vref following the decision to land.

(iii) When conducting a single engine non-precision approach, flap 35 should be selected following the decision to leave MDA for landing.

If the aircraft is not stabilized on the approach by 1000 feet AGL in IMC, or by 500 feet AGL in VMC, or if it becomes destabilized below these altitudes, the crew shall immediately execute a go-around.”5

There were three instances of unstable approaches observed and, though relatively minor occurrences, the Company must do a better job at emphasizing the danger of not having the aircraft properly stabilized for landing below the altitudes set out in the SOPs. Industry-wide the majority of landing accidents have followed unstable approaches and an occurrence rate of almost 3% is not acceptable.

5 Air Georgian Limited, Beech 1900D Standard Operating Procedures. (Toronto: Air Georgian Limited, 2009), 2-68.


25

The above chart shows the Undesired Aircraft States that were encountered along with the percentage of flights that these were observed on.

3.4 Crew Performance Marker Worksheet In the Crew Performance Marker Worksheet the observer was able to sum up the flight as a whole. The flight is divided into phases and planning, execution, review and overall performance markers are assessed by assigning a score of one to four for each item (1 = Poor, 2 = Marginal, 3 = Good, 4 = Outstanding). The reasoning for each assigned score can be found in the narrative section of the form. Far more 4s were assigned by the observers than 1s and the narratives seem to justify this for the most part though there are instances where, based on the narrative, a mark of 1 would likely have been more appropriate. Still, the low number of crews assigned a ‘poor’ rating is positive and is indicative of the quality of work performed by most Air Georgian pilots. Still, the Company must work to reduce the number of ‘poor’ ratings, especially since the category with the highest number is ‘Leadership’ (in the Overall Performance Markers section) with a total of five (4.5% of observed flights). A ‘poor’ leadership rating was generally assigned where Captains utterly failed to establish a professional atmosphere in the cockpit.

0

1

2

3

4

5

6

7

UNDESIRED AIRCRAFT STATES

UNDESIRED AIRCRAFT STATES


26

This chart shows the number of 1s (Poor) assigned on the Crew Performance Marker Worksheet (note that unlike the previous charts these are total numbers rather than percentages).

3.5 The LOSA Safety Survey

The LOSA Safety Survey was intended to compliment the audit flights by allowing Air Georgian pilots to verbalize their impressions of the Company safety culture. The ten questions included in the survey asked pilots about their flying backgrounds, their views on the major safety hazards faced in our operation, as well as their impressions of how the Company approaches safety and the priority placed on it. The survey was not meant to be scientific; rather, the intention was to gain a feel for the general impression that the pilot group has regarding the approach that the Company takes toward safety. Had individuals expressed serious safety concerns or had there been a general frustration among pilots at the way the Company prioritizes safety there would certainly have been grounds for worry. Fortunately, in general, the survey respondents were satisfied with the Company’s approach toward safety and a number were quite complimentary. Of the twenty-five surveys returned sixteen pilots had no significant safety concerns to report, seven had concerns that had more to do with union issues than safety. Only two of the respondents raised what could be considered to be significant safety concerns, one involving a maintenance issue and one involving an internal communication failure that resulted in an aircraft going flying with an expired deferred maintenance item. Also, only two of the twenty-five respondents felt that there would be negative consequences from management if they ever refused to operate a flight because of a safety concern.

0

1

2

3

4

5

6

Pre‐Dep/Taxi

Takeoff/Climb

Descent/App/Land

Overall


27

The main issue raised by the respondents had to do with fatigue; many feel that Air Georgian’s operations are fatiguing and this is obviously a safety concern. While it is possible that a pilot may become fatigued while at work the Company has addressed this with a comprehensive fatigue policy. This policy allows pilots to call in fatigued (for any reason) if necessary or, if already at work, to remove themselves from duty if required with no disciplinary consequences. The Company also has a standing offer of a paid hotel room for any pilot (or other employee) who feels too tired to drive home after a duty period. In addition, Air Georgian pilots are guaranteed eleven days off per month and are not allowed to be scheduled with less than twelve hours off between pairings (that is, from the time they leave the airport until they have to arrive at work the next day) and eleven hours off while on a layovers (this is a bare minimum; most layovers are considerably longer). Given the above, and considering the available fatigue research, Company pilots are clearly provided with enough time free from duty to avoid becoming fatigued due to the schedule. It should also be mentioned that, because of the union scheduling rules, the distribution of work is heavily slanted in favour of more senior pilots. If it is an issue for pilots that some of them work much harder than others at Air Georgian the union scheduling rules that give some pilots twenty-five hours of flying per month while others do 100 hours would have to be changed.


28

4. Recommendations

Now that some of the threats, errors and undesired aircraft states that Air Georgian flight crews encounter on a regular basis have been identified and quantified it is up to the Company as a whole to use this information to make improvements that will decrease risk. The following recommendations should, if followed, allow the Company to mitigate threats and substantially reduce errors and undesired aircraft states.

1. Air Georgian must devise a better method of informing flight crews of deferred maintenance items on their assigned aircraft. By informing flight crews in advance of DMIs there will be much less chance of errors (involving the maintenance status of the aircraft) occurring, the most important one being the operation of an aircraft in conditions that are prohibited by the deferral procedure.

2. As much as is operationally possible the Company should not pair two Captains together to operate flights. As this is not always possible the Company should make Captains aware of the higher number of errors that occur when they fly with each other rather than with a First Officer as well as the reasons for this.

3. The Company should make more of an effort in making pilots aware of the dangers involved in complacency. This can be done via the monthly newsletter, in ground school, during line checks and by managers when they fly with line pilots.

4. The leadership of the Flight Operations branch of the Company must make a major, sustained effort to encourage and increase adherence to SOPs. While SOP compliance is not bad within the Company the LOSA has made it clear that it can and should be much better. The following measures should be adopted:

i. The Training Department should develop and use more scenario based simulator training that incorporates ‘normal’ SOP use rather than simply throwing one emergency after another at the candidates;

ii. Management pilots should fly as often as possible and lead by example by strictly following the SOPs;

iii. The very real dangers of non-compliance with SOPs must regularly be advertised to Company pilots via the monthly newsletter, during all training (ground school and simulator), and in person by Managers and Line Indoctrination Captains;

iv. The small number of pilots who do not feel that SOPs apply to them should be identified when possible. These pilots should be counseled and, in extreme cases where adequate improvement is not shown, should have their employment terminated if they are unable or unwilling to comply with SOPs;

v. It should be made clear to all Company pilots that anything less than full compliance with the SOPs is not acceptable and is unprofessional; it is not something that should be tolerated within the Company.


29

5. Unstable approaches (as defined by the SOPs) must be eliminated completely from Company operations. It must be emphasized to all Company pilots during training, line indoctrination and line checks that unstable approaches cannot be continued; the only acceptable action is to discontinue an unstable approach.

6. Too many Company pilots operate flights without referencing the runway performance data. Crews become complacent because almost all of the runways that Air Georgian operates from are more than adequate for the BE02 at maximum weight under all but the most extreme temperature conditions. However, not all of the runways are usable at maximum takeoff weight without a special departure procedure in the event of an engine failure; nor are all runways adequate under tailwind or icing conditions. It is crucial that the runway performance data charts are referenced prior to flight. However, the Company can facilitate this process by highlighting the performance limiting runways (and conditions) used by the scheduled service for pilots. This would be a simple matter of creating an information sheet that could be placed into the route manual.

7. The Company must place more emphasis on the proper use of aircraft automation during training, line indoctrination and line checks (automation includes the passenger briefing system).

8. It is clear that the Company has to enhance the training that we provide regarding the proper use of the aircraft Minimum Equipment List and the entering of snags in the aircraft Journey Log Book.

9. High taxi speeds (outside of SOP limits) were a recurring issue during the observed flights (11% of the flights). Risk is substantially increased by taxiing too fast and the dangers of doing so must be advertised to flight crew more emphatically. Company management must also be on the lookout for fast taxiing and the Captains responsible should be counseled.

10. The entire pilot training program for both initial and recurrent candidates requires an extensive review. The training program was first developed over ten years ago and components have been added in a haphazard fashion over time. The training program does not take advantage of many of the latest technologies available and some of these should be incorporated into a revamped program. The type of threats, errors and undesired aircraft states that have been noted in this report should be emphasized in the training program, during line indoctrination and on line checks. The Chief Instructor, working closely with the Chief Pilot, should conduct this review.

11. Data from check rides, line indoctrination, line checks and training should be recorded and scrutinized so that deficiencies can be identified and corrected. These data can be used to set Company benchmarks so that progress can be measured over extended periods.

12. While there is always more that can be done by any organization to enhance safety much of it will be pointless if the line employees who are actually doing the work do not ‘buy in’. For example, the Company can go on about SOP compliance on a daily basis but if pilots are not willing to make the effort to do the job properly very little will change. It has to be made clear to pilots that professionalism is essential at all times in this line of work. Safety has to involve all of the stakeholders making an effort to continually improve. Pilots must be clear that their own


30

safety depends on always maintaining a high level of professionalism; this includes SOP compliance, regular review of emergency procedures and good aircraft systems knowledge.

13. The Company should introduce formal Threat and Error Management training into the pilot initial and recurrent curricula.

14. Management must review the BE02 checklists and correct any deficiencies (such as the lack of a reminder to change the TCAS mode at the top of the climb and the top of descent).

15. The Flight Following office should become more proactive in the way that flights are handled. While aircraft dispatched under a Type C system do not require licensed dispatchers there is no reason why Company Flight Followers cannot play a more involved role in the way that flights are followed. This includes maintaining a high level of awareness throughout each shift of the maintenance status of aircraft, the weather conditions system-wide, NOTAMs that may affect operations, problematic airport conditions and any other essential operational information. Attaining this level of awareness among Flight Followers will require close supervision from management (at least initially) and a review of the training provided.


31

Conclusion

The LOSA was a large undertaking for a regional operator the size of Air Georgian. A substantial amount of resources were devoted to the project. However, the process was well worth the monetary and time commitment involved and a follow-up LOSA will be conducted in the next year and a half to two years.

To a large degree the results of the LOSA are quite reassuring; the data returned shows that our operating environment and operation are safe with low levels of risk. Our flight crews are generally dedicated professionals who do not take undue risks and who place safety ahead of all other considerations. However, as noted above, the data also shows that there is work to be done in many areas. The Company must make more of an effort to enhance our pilot training by putting to better use the knowledge gained over the fifteen years that Air Georgian has operated the BE02. We should also make more use of some of the cutting edge training technologies that are available today. Company communication to the pilot group should also be improved. Pilots must also recognize that maintaining a high level of professionalism is every bit as much their responsibility as it is the Company’s and they must strive every day to operate at the highest level possible. In short, continuous improvement has to be the goal for both management and pilots so that risk levels can be lowered to the bare minimum. Some of the data and recommendations that have come out of the LOSA will hardly be surprising to experienced aviation professionals; the value is derived from the fact that we now have verified, quantified information rather than assumptions. The LOSA has provided a very clear picture of the operation at the time that the data was gathered.

The findings provided by the LOSA will be used as benchmarks that the results of subsequent LOSAs will be measured against. When the next LOSA is conducted we will be able to determine the effectiveness of the recommendations included in this report (and/or the effectiveness of our execution). None of the issues raised in this report are insurmountable; on the contrary, given the necessary effort and some time, every concern that has been brought to light by the LOSA is correctable to a large degree. No operation is or can be perfect but a solid safety culture and a commitment to continuous improvement will result in a sustained reduction in the level of risk inherent in flight operations.


32

APPENDIX A

Below is a copy of the LOSA form; one form was filled out for each observed flight.

Observer Information

Observer ID (Employee number) Observation Number

Crew Observation Number (e.g., “1 of 2” indicates segment one for a crew that you observed across two segments

Of

Flight Demographics

City Pairs (e.g., YYZ-ALB) A/C Type BE02 Pilot Flying (Check one) CA FO

Time from Pushback to Gate Arrival (Hours:Minutes)

Local Arrival Time (Use 24 hour time)

Late Departure? (Yes or No)

Predeparture/Taxi

Narrative Your narrative should provide a context. What did the crew do well? What did the crew do poorly? How did the crew manage threats, crew errors, and significant events? Also, be sure to justify your behavioral ratings.


33

Takeoff/Climb

Narrative Your narrative should provide a context. What did the crew do well? What did the crew do poorly? How did the crew manage threats, crew errors, and significant events? Also, be sure to justify your behavioral ratings.


34

Cruise

Narrative Your narrative should provide a context. What did the crew do well? What did the crew do poorly?

Descent/Approach/Land/Taxi

Narrative Your narrative should provide a context. What did the crew do well? What did the crew do poorly?


35

Overall Flight

Narrative This narrative should include your overall impressions of the crew.

Threat Management Worksheet

Th

reat

ID

Threat Description Threat Management

Describe the threat Threat Type

Phase of Flight

1 Predepart/Taxi 2 Takeoff/Climb 3 Cruise 4 Des/App/Land 5 Taxi-in

Linked to flight crew

error?

(Yes/No)

How did the crew manage or mismanage the threat?

T1

T2


36

T3

T4

T5

T6

Threat Codes

Environmental Threats Airline Threats 100 Adverse Weather 101 ATC 102 Terrain

103 Airport Conditions 104 Heavy Traffic (air or ground) 199 Other Environmental Threats

200 Airline Operational Pressure 201 Cabin 202 A/C Malfunctions/MEL Items 203 Ground Maintenance

204 Ground/Ramp 205 Dispatch/Paperwork 206 Manuals/Charts 299 Other Airline Threats

Error Management Worksheet

Err

or

ID

Error Description Error Response/Outcome Error

Management

Describe the crew error

Phase of Flight

1 Predepart/Taxi 2 Takeoff/Climb 3 Cruise 4 Des/App/Land 5 Taxi-in

Linked to

Threat?

(If Yes, enter the

Threat ID)

Error Type

Crew Error Response

1 Detected 2 No response

Error Outcome

1 Inconsequential 2 Undesired state 3 Additional error

How did the crew manage or

mismanage the error?

E1


37

E2

E3

E4

E5

E6

Error Type Codes

Aircraft Handling Procedural Communication 300 Manual Flying 301 Flight Control 302 Automation 303 Ground Handling 304 Systems/Instruments/Radios 399 Other Aircraft Handling

400 SOP Cross-Verification 401 Checklist 402 Callout 403 Briefing 404 Documentation 499 Other Procedural

500 Crew to External Communication 501 Crew to Crew Communication 599 Other Communication

Undesired Aircraft State (UAS) Management Worksheet

UA

S ID

UAS Description UAS Response/Outcome UAS Management

Linking Error?

(Enter the Error ID)

Undesired aircraft state description

UAS Code

Crew UAS Response

1 Detected 2 No response

UAS Outcome

1 Inconsequential 2 Additional Error

How did the crew manage or mismanage the

undesired aircraft state?

UAS1


38

UAS2

UAS3

Undesired Aircraft State Type Codes

Configuration States Ground States Aircraft Handling States-

All Phases Approach/Landing States

1 Incorrect A/C configuration---

flight controls, brakes, thrust reversers, landing gear

2 Incorrect A/C configuration--- systems (fuel, electrical, hydraulics, pneumatics, air-conditioning, pressurization, instrumentation

3 Incorrect A/C configuration--- automation

4 Incorrect A/C configuration--- engines

20 Proceeding toward wrong runway

21 Runway incursion

22 Proceeding toward wrong taxiway/ramp

23 Taxiway/ramp incursion

24 Wrong gate

25 Wrong hold spot 26 Abrupt aircraft control—taxi

40 Vertical deviation

41 Lateral deviation

42 Unnecessary WX penetration

43 Unauthorized airspace

penetration

44 Speed too high

45 Speed too low

46 Abrupt aircraft control---(attitude)

47 Excessive banking

48 Operation outside aircraft

limitations

80 Crew induced deviation

above G/S or FMS path

81 Crew induced deviation below G/S or FMS path

82 Unstable approach

83 Continued landing---unstable

approach

84 Firm landing

85 Floated landing

86 Landing off C/L

87 Long landing outside TDZ

88 Landing short of TDZ

99 Other Undesired States

Crew Performance Marker Worksheet

1 2 3 4 Poor

Observed performance had

safety implications

Marginal

Observed performance was adequate but needs improvement

Good

Observed performance was effective

Outstanding

Observed performance was truly noteworthy


39

Phase of Flight Ratings

Planning Performance Markers Predeparture/

Taxi Takeoff/ Climb

Descent/Approach/ Land/Taxi

SOP BRIEFING The required briefing was interactive and operationally thorough.

PLANS STATED

Operational plans and decisions were communicated and acknowledged.

CONTINGENCY MANAGEMENT

Crew members developed effective strategies to manage threats to safety.

Execution Performance Markers

MONITOR/CROSS-CHECK

Crew members actively monitored and cross-checked systems and other crew members.

WORKLOAD MANAGEMENT

Operational tasks were prioritized and properly managed to handle primary flight duties.

VIGILANCE Crew members remained alert to the environment and position of the aircraft.

AUTOMATION MANAGEMENT

Automation was properly managed to balance situational and/or workload requirements.

TAXIWAY/RUNWAY MANAGEMENT

Crew members used caution and kept watch outside when navigating taxiways and runways.

Review/Modify Performance Markers

EVALUATION OF PLANS Existing plans were reviewed and modified when necessary.

INQUIRY

Crew members not afraid to ask questions to investigate and/or clarify current plans of action.

Overall Performance Markers Ratings

COMMUNICATION ENVIRONMENT

Environment for open communication was established and maintained.

LEADERSHIP Captain showed leadership and coordinated flight deck activities.


40

APPENDIX B

This memo was issued to managers only in order to provide a brief description of the LOSA.

Line Operations Safety Audit

We will be carrying out a Line Operations Safety Audit (LOSA) at Air Georgian over the summer of 2010. LOSA is recognized worldwide as an effective diagnostic tool that enables air operators to credibly measure their safety performance. By conducting a LOSA those responsible for operational safety will be provided with enough data to identify and thus be able to correct weaknesses that would otherwise be difficult to pinpoint.

As this will be the first LOSA carried out at Air Georgian the scope will be quite wide; all aspects of the flight operation will be audited. Subsequent LOSAs can be narrowed in scope to concentrate on issues identified by the 2010 audit. The time frame for the next LOSA would be two to three years after the report for the 2010 audit is issued.

The LOSA will be implemented by the LOSA steering committee which will consist of the Operations Manager, the System Chief Pilot, the Chief Instructor, the Corporate Safety Officer and a representative from the employee association. In general terms the audit will be executed in six stages as follows:

1. Planning – After the training course is completed on June 8th the steering committee will finalize the audit process and procedures in one or more meetings.

2. Audit flights – The actual audit flights will be carried out by six auditors. It is envisaged that each auditor will audit two turns per day over a five day period. The audit flights will be spread out through the month of July so that only two auditors are active at any one time. After their first day of audit flights the auditors will meet with at least two members of the steering committee to go over the process and clarify any points as necessary. The auditors will be given specific audit days on their schedules but will pick the flights to be audited themselves so that management is not in any way involved in selecting the crews of the audit flights.

3. Compile data – Once the entire series of audit flights have been completed the data must be compiled into a database. This will be done by the auditors themselves and each will be given an office day in early August or late July for this task. The data will be totally de-personalized so that there is no way to identify which crew members operated any of the flights. The format of the database is to be announced.

4. Review data – In August the steering committee and the auditors will meet to review the data captured during the audit and come up with a list of findings.


41

5. Issue report – Once the findings have been identified the Operations Manager, with input from all involved, will write up the audit report. This document will be detailed and will include not only the findings but the responses as well.

6. Implement fixes – All findings must be addressed and corrected or the audit will have been a waste of time and resources. Correcting any findings will require the full buy-in of managers at all levels.

Please bear in mind that the time-line will be as flexible as it needs to be. The LOSA is a priority project but if our crewing situation becomes too tight it may well get pushed back into the fall.


42

APPENDIX C

Below is a copy of the LOSA Safety Survey.

Air Georgian Limited LOSA Safety Survey

1. What are your total hours? Under 2000 2000 – 4000 4000 – 6000 6000+

2. Before joining Air Georgian, what type of operation did you work for (i.e. air taxi, 704 commuter operation, flight training unit etc…)?

3. What do you feel are the top issues that adversely affect safety in our operation (name up to 3 issues)?

4. Have you ever encountered a significant safety deficiency at Air Georgian? If yes, please elaborate.

5. How seriously do you think that Air Georgian management takes safety on a day to day basis? 6. What could Air Georgian management do better when it comes to enhancing safety? 7. Have you ever felt that an ATC clearance or instruction that you have been issued has

unnecessarily increased risk or has been unsafe? If yes, please elaborate and state how you dealt with the situation.

8. While working at Air Georgian, have you ever felt that your safety has been put at risk due to an encounter with adverse weather?

9. How do you think you would be treated by Air Georgian management if you ever refused to carry out a flight due to a safety concern?

10. Do you have any comments or concerns not addressed by the above questions?


43

APPENDIX D

These instructions should be issued to the in-flight observers in future LOSAs.

LOSA Auditor Checklist

LOSA Auditors should plan to complete one flight in the morning and one flight in the afternoon. It is the Auditor's responsibility to plan sufficient time between flights for customs, LOSA Crew briefings, and meals.

Equipment: Box type splitter- This splitter has a 9 volt battery inside the control box. (There is no external indication of this.) Its function is to power the Auditor's microphone. This selection should be turned off for our LOSA purposes as the Auditor is not to be talking with the flight crew. If the battery is dead/nearly dead and the MIC is turned on, gross amounts of feedback will be heard in the Auditor's headset, preventing effective observation. ALWAYS LEAVE THE MICROPHONE TURNED OFF.

All splitters are to be plugged into the left seat jacks. The captain will then plug into the splitter box.

The day before your LOSA flights:

-Call Crew Sched to plan your LOSA flights. It is imperative that you do so early as there may be more than one LOSA auditor operating when you are. This means you may not have access to the exact flights you want.

-Copy down the flight numbers for your planned flights and request that the Crew Scheduler list you for these flights.

On arrival at the North End:

-Get your flight locator numbers from Crew Sched.

-Check in for your first flight on-line if possible.

-If it is your first LOSA flight, get the splitter from the office.

-Print a weather package/NOTAMS for your flights.

On arrival at the Departure Gate:

-Arrive at the gate 30 minutes prior to departure so that you can contact the crew.

-Supply the crew with the LOSA briefing and give the captain the headset splitter and advise him/her of its operation.


44

-Advise the sales agent of the LOSA observation, and confirm that you are to be seated in seat 1A or 1B.

On arrival at the aircraft:

-Check the logbook for deferred items that might affect the flight/observation.

On departure from the airport after your LOSA observations:

-Confer with Crew Sched and plan out the following days LOSA flights.

-Copy down flight numbers for your planned flights, and request that the Crew Scheduler list you for these flights.

Preparation of LOSA paperwork:

-Complete full LOSA write-ups prior to your data entry day. Make sure it is checked and complete so that the only task required at the office is transferring it onto the company database.


45

APPENDIX E

LOSA Lessons Learned

This was our first attempt at conducting a LOSA at Air Georgian; for the most part we are very pleased with how it went though we did learn a number of lessons for next time.

1. The Crew Performance Marker Worksheet section of the LOSA form should include an area to mark overall SOP compliance during the flight;

2. Before the series of observation flights begin the LOSA Steering Committee should ensure that the headset splitter boxes work with all of the Company aircraft involved in the audit. Unfortunately we did have some complications as one of the headset splitters was not compatible with one or two of the aircraft (the plug fit but it simply did not work);

3. In the future the observers should not be scheduled to do seven days of in-flight observation in a row. All of the observers complained that this schedule was too demanding and that at least one day off should be inserted. This is understandable as the observers had long days at the airport and then they had to type their notes in the evenings;

4. Observer training needs to more heavily emphasize correct and standardized form marking. The observers all made errors in the way that they categorized the various threats, errors and undesired aircraft states. All of these were subsequently corrected but this required quite a bit of additional work and could have been avoided with more thorough training;

5. Crew Schedulers and Flight Followers should have been briefed about the LOSA in person by management. Instead we simply issued them a memo and this was not sufficient. There were quite a few misunderstandings regarding the job being done by the observers. Better communication from management would have reduced or eliminated the day of operations confusion that occurred on a number of occasions;

6. For the next LOSA a much higher number of first turns of the day should be observed. These would include both early morning departures and afternoon departures depending on the flight crew’s start time. The observer will thus be able to observe the crew as they flight plan so that they can note how thoroughly it is done.

Date post:	07-Nov-2014
Category:	Documents
Upload:	bockner
View:	807 times
Download:	10 times

Losa Report

Documents