RATERS:SELECTING AND TRAINING FOR BENCHMARKING AND PROFICIENCY EVALUATION PURPOSES
•ICAO WORKSHOP ON LANGUAGE PROFICIECY •Baku, Azerbaijan, December 7-9, 2005
•Evgenia Povaliy, Aviation English Instructor •Sergey Melnichenko, Deputy Director•Moscow ATC Training Center
CASE STUDYCASE STUDY
Accidents where language was a Accidents where language was a causal or contributing factorcausal or contributing factor
Phraseologies only are not Phraseologies only are not enoughenough
Plain language is needed in non-Plain language is needed in non-routine and urgent situationsroutine and urgent situations
English as an Instrument English as an Instrument of Communicationsof Communications
70 per cent of 28,000 safety reports 70 per cent of 28,000 safety reports were connected with communicationswere connected with communications
70 per cent of those who communicate 70 per cent of those who communicate are non-native English speakersare non-native English speakers
No other existing live or artificial No other existing live or artificial language is either better or worse than language is either better or worse than EnglishEnglish
English has become a “lingua franca”English has become a “lingua franca”
Manual on the implementation Manual on the implementation of ICAO Language Proficiency of ICAO Language Proficiency
Requirements (Doc 9835)Requirements (Doc 9835)
Licensing authorities should require Licensing authorities should require specialized evaluation of those who do specialized evaluation of those who do not demonstrate Level 6 proficiencynot demonstrate Level 6 proficiency
Phraseologies-only testing is not Phraseologies-only testing is not appropriateappropriate
Demonstration of actual speaking and Demonstration of actual speaking and listening ability is requiredlistening ability is required
Until 5 March 2008 States may continue Until 5 March 2008 States may continue to use procedures they currently have to use procedures they currently have in place to assess English proficiency of in place to assess English proficiency of flight crews and controllersflight crews and controllers
Requirements for Requirements for ratersraters
Higher education in LinguisticsHigher education in Linguistics Level 6 proficiency according to the Level 6 proficiency according to the
ICAO ScaleICAO Scale Highly developed moral standardsHighly developed moral standards Aviation background Aviation background Relevant training, preferably in an Relevant training, preferably in an
English-speaking countryEnglish-speaking country
THE SCALETHE SCALE
Who has seen it?Who has seen it? Who is happy with it?Who is happy with it? Who has used it for Who has used it for
benchmarking evaluation?benchmarking evaluation? Who has been taught to use Who has been taught to use
the Scale?the Scale?
TESTSTESTS
EntryEntry PlacementPlacement DiagnosisDiagnosis ProgressProgress ProficiencyProficiency
ENTRY TESTENTRY TEST
Enables Airline Operators and Enables Airline Operators and ANS providers to avoid new ANS providers to avoid new personnel that is likely to personnel that is likely to have problems in 2008 have problems in 2008
because of poor knowledge of because of poor knowledge of English at presentEnglish at present
PLACEMENT TESTPLACEMENT TEST
Enables Airline Operators and Enables Airline Operators and ANS providers to map out the ANS providers to map out the
level of existing personnellevel of existing personnel
DIAGNOSIS TESTDIAGNOSIS TEST
Enables linguists to identify Enables linguists to identify the needs of individuals the needs of individuals (problem areas, course (problem areas, course
duration and its content)duration and its content)
PROGRESS TESTPROGRESS TEST
Enables Airline Operators, Enables Airline Operators, ANS providers and linguists to ANS providers and linguists to
mark progress/no mark progress/no progress/regress progress/regress and to use and to use
other training path when other training path when neededneeded
PROFICIENCY TESTPROFICIENCY TEST
Shows the actual level of Shows the actual level of English language proficiency English language proficiency
according to ICAO Scaleaccording to ICAO Scale
Available TestsAvailable Tests
ChineseChinese KoreanKorean American (Oral Proficiency Interview)American (Oral Proficiency Interview) British (Mayflower College)British (Mayflower College) Russian (Moscow ATC Center)Russian (Moscow ATC Center) Eurocontrol (ELPAC – July 2007)Eurocontrol (ELPAC – July 2007)
DrawbacksDrawbacks
Computer – no interactionComputer – no interaction Oral Proficiency – time and Oral Proficiency – time and
manpower-consumingmanpower-consuming THERE IS NO IDEAL TESTING TODAYTHERE IS NO IDEAL TESTING TODAY
BUT…BUT… ““Any test at all” is not better than Any test at all” is not better than
“no test” (Doc 9835)“no test” (Doc 9835)
Steps to Make Steps to Make
Introduce ICAO qualification Introduce ICAO qualification requirements into national legislationrequirements into national legislation
Select (through independent linguistic Select (through independent linguistic institutions) and train ratersinstitutions) and train raters
Evaluate current level of pilots and Evaluate current level of pilots and controllerscontrollers
Ensure they are adequately taught Ensure they are adequately taught General and Aviation EnglishGeneral and Aviation English
Evaluate proficiency level of pilots and Evaluate proficiency level of pilots and controllers in 2008controllers in 2008
LAST BUT NOT LEASTLAST BUT NOT LEAST
Code of ethicsCode of ethics
Ethical PrinciplesEthical Principles (ILTA Code of Ethics)(ILTA Code of Ethics)
Raters shall have respect for humanity and dignity of each Raters shall have respect for humanity and dignity of each of their test takersof their test takers
Raters shall have all information in confidence and use Raters shall have all information in confidence and use professional judgment in sharing itprofessional judgment in sharing it
Raters should adhere to all relevant ethical principles Raters should adhere to all relevant ethical principles embodied in national and international guidelines when embodied in national and international guidelines when undertaking any trial, experiment or other research activityundertaking any trial, experiment or other research activity
Raters shall not allow the misuse of their professional Raters shall not allow the misuse of their professional knowledge or skillsknowledge or skills
Raters shall continue to develop their professional Raters shall continue to develop their professional knowledge, sharing it with colleaguesknowledge, sharing it with colleagues
Raters shall share the responsibility of upholding the Raters shall share the responsibility of upholding the integrity of language testing professionintegrity of language testing profession
Raters shall strive to improve the quality of language Raters shall strive to improve the quality of language testing, assessment and teaching servicestesting, assessment and teaching services
Raters shall be mindful of their obligations to the societyRaters shall be mindful of their obligations to the society Raters shall regularly consider the potential effects on all Raters shall regularly consider the potential effects on all
stakeholders, reserving the right to withhold their stakeholders, reserving the right to withhold their professional services on the grounds of conscience professional services on the grounds of conscience
THANK YOU!THANK YOU!
Evgenia Povaliy, Aviation English InstructorEvgenia Povaliy, Aviation English Instructor [email protected]@atcm.ru Sergey Melnichenko, Deputy Director Sergey Melnichenko, Deputy Director [email protected] www.aviaenglish.ru Moscow ATC Training CenterMoscow ATC Training Center