Background #1ELICOS offered in Australia since mid-1980s
ELICOS delivered to student body 100% international
Offered at 240 institutions - private “standalone, VET, TAFE, secondary and university.
2005: 101,000 students whose visas were 49% student, 37% visitor, 14% working holiday and other.
65% of student visa holders in 2002 continued to further study.
Background #2Status and diversity of Australian ELICOS industry
Rationale for a national language levels framework
Nature of consultancy
Aims of the studyidentify factors the Australian ELT/ELICOS industry would need to consider in determining whether to make use of a language levels framework e.g. CEFR
assess benefits of adopting such a framework by the sector
consider the implications for providers and students of adopting such a system
MethodologyStep1 Background research2 Design of data collection instruments3 Review of methodology4 Distribution of draft interim report5 Collection of survey data6 Follow-up interviews7 Preparation of draft final report8 Circulation of draft final report9 Submission of Final (edited) report
What is a language levels framework?
A language levels framework is a series of language descriptions arranged sequentially to indicate an expected order of language progression or development over time.
A framework is NOT a test.
Purposes of language level frameworks
1. Establish a framework of reference which can describe achievement in a way that scores from test items cannot.
2. Provide coherent internal links within one system between pre-course or entry testing, syllabus planning, materials organisation, progress, exit assessment and certification using a common yardstick.
3. Enable comparison between systems or populations using a common set of standards.
How frameworks differ from one another
Target language/language variety
Learner groups
Purpose
Audience
Views of language proficiency/ language learning
Aids to implementation
Examples of currently used frameworks
ACTFL proficiency guidelinesALSPR/ ILSPRNLLIA ESL BandscalesCanadian Language BenchmarksCertificates of Spoken and Written English (CSWE)National Reporting System (NRS)Common European Framework of Reference (CEFR)
Common European Framework of Reference (CEFR) #1
Six sequential levels:
A B CBasic user Independent user Proficient user
A1 A2 B1 B2 C1 C2Breakthrough Waystage Threshold Vantage Effective Mastery
OperationalProficiency
Descriptors framed as “can do” statementsE.g. “Can summarise information from different spoken
and written sources”.
Common European Framework of Reference (CEFR) #2Accompanying tools:
Dialang
European language portfolio
User’s manual
Common European Framework of Reference (CEFR) #3CEFR has been used
to develop curricula e.g. English Profile Project -British Council
as a basis for test development e.g. Dutch CEFR construct project
to link tests to framework documents e.g. ETS mapping project
Range of expert viewsThose who support CEFR’s aim to create a
common language for describing learners progress and unifying curriculum and assessment
VERSUSThose who see the institutionalisation of
CEFR as politically motivated and as having unwarranted global influence.
Criteria for effective implementation of a frameworkNeed for the framework to:
be clear in purpose for its usershave a sound theoretical basespan the ability range of learners in the ELICOS sectormeaningfully describe learner development be taken up seriously by relevant stakeholders be highly practical be accompanied by appropriate materialsbe underpinned by standardised tests
Focus groupsHeld in Sydney and Melbourne, October 2006
Participants from a range of centres:Endorsed proposed methodology and timeline.Made suggestions for refinement of a) sampling methods and b) content and format of survey.
Survey of Directors of StudiesInitial sample of 46 ELICOS providers
proportionally representative in terms ofNEAS and non-NEAS accreditedthe various Australian statesprivate (stand alone), VET private,
TAFE, secondary and university.
Survey design
First part designed to elicit information from DoS about relevant features of their institution and their views about assessment needs and challenges.
Second part canvassed DoS’ knowledge, experience and views of different language frameworks as well as their attitudes regarding feasibility of implementing such a system within the sector.
Survey respondentsCompleted by 36 institutions across Australia:
34 NEAS 12 Private (stand alone) 6 university7 Private VET4 TAFE4 secondary schools1other
2 Non-NEAS
Survey results - Part 1Respondent profile (students, staff, type of operation)Type, level and duration of coursesSyllabi and course materialsAssessment practicesPathway arrangementsStaff with examiner credentialsAssessment problems and needs
Respondent profileStudents
Average enrolments for 2006:25.6%: < 10037.2%: 100-30037.2: > 300
StaffTotal number from 3.3 -120 with average of 28 across all providersAlmost 50% employed on contractMost had either CELTA or Grad Dip Ed
Type, level and duration of courses
Wide variety of courses and levels listed
General and Academic English as well as IELTS Preparation the most common.
Also large number of specific purpose courses e.g. linked to workplace needs, business travel and tourism.
Duration: range from 2 - 50 weeks. Most of limited duration, no longer than 12 weeks.
Syllabi and course materialsMajority of centres relied on syllabi produced in-house or mixture of in-house and textbook based.
13 centres based their syllabi exclusively on textbooks.
Only 7 centres relied solely on their own teaching materials.
Assessment practicesFor placement:Descriptors such as “Elementary”, “Intermediate” are most commonMajority of centres use placement testsFor promotion to higher levels:Most centres use in-house “off the shelf” assessment tools For certification:Most centres use continuous assessmentCertificate usually only awarded when student completes chosen programInformation about level of achievement on back of certificates varies considerably.
Pathway arrangementsMajority of providers had “direct entry”pathways with other institutions, mainly universities (67%).
Large majority prepared students for IELTS (83%). Minority of centres had other test preparation courses e.g. Cambridge exams, especially FCE, TOEIC, TOEFL.
Staff with examiner credentialsOn average, 20% of staff in ELICOS centreswere accredited IELTS examiners (although this varied widely across providers)
There were also accredited examiners of the Cambridge, TOEIC and Trinity College examinations amongst the providers surveyed.
Assessment problems and needsProblems (in order):Teachers apply standards inconsistentlyTeachers lack understanding about assessmentAssessments/tests are not sufficiently related to the
curriculumTeachers are reluctant to use explicit criteriaAssessments/tests are not appropriate for students’
proficiency level
Needs (in order):End-of-level testsDiagnostic testsSelf-assessment toolsPlacement proceduresExit tests
Survey results - Part 2English language frameworksViews on desirability of a frameworkType of framework favouredReasons for choice of frameworkEducation about frameworksNature of preferred frameworkNumber of levelsUse of a frameworkAcceptability to studentsDisadvantages of a framework
English language frameworksMost familiar frameworks (in order):ISLPR, CSWE, CEFR, NRS
Most commonly used frameworks (in order): ISLPR, CSWE, NRS and CEFR
Views on desirability of a framework
76% of respondents (30 of 36) were positive in principle about the idea of a common national framework
However, many respondents gave a mixture of positive and negative views when asked to explain their initial responses.
Positive views: transparency, portability, objectivity, professionalism, quality and credibility.
Negative views: inflexibility, homogenization, impracticality and redundancy.
Transparency and portability“It would of course help every institution
understand the English language skills of any student who wishes to enrol in either an ELICOS program or a university/TAFE program.”
“I believe that a national framework would make ELICOS results much more useful for students in future study & employment.”
Objectivity“Currently it is difficult for education providers to
make accurate judgements about a student's proficiency based on a certificate from an ELICOS centres; one's opinion of the centre or knowledge of its reputation is factored into the judgement of the value of the certificate.”
“It would be great to have a degree of standardisation between schools and set tests for teachers to use. It is very hard to design valid and reliable tests with little time available.”
Professionalism“A national framework would hopefully
educate staff in assessment.”
“[It would] aid in the development of curricula and syllabi.”
Quality and credibility“To ensure quality across the board, which will in
turn improve the industry”
“It is time to develop a true international test and reporting system, which has global integrity and recognises the modern purpose for language study: international information technology, global employment, travel and communication.”
Inflexibility and homogenisation“(A common framework) might lead to a lack of
flexibility in future programs as every ELICOS centre would need to teach to the same objectives or thereabouts, leading to a certain sameness in English progammes, that may harm future trade.”
“Niche markets developed by institutions may be harmed if the focus of the students narrows to finding the ELICOS program that other students have found easy to get the levels they need. “
“Should not be imposed. One-size-fits-all approach would not work. Australian context/market too small.”
Impracticality
“Too often the processes involved to meet criteria with training, ongoing PD, recording and reporting are too time consuming or inadequate.”
“There is a great deal of work involved for all centres in aligning current assessment andmarking criteria to a national standard. This would be at considerable cost and many centres would be unwilling to bear this.”
Redundancy“…just yet another reporting mechanism to
add to a full basket.”
“…there are already external working alternatives (IELTS and other tests, ESL Scales, etc) and what are these scales used for ? Where it counts (ie. in dealing with other institutions) most colleges have direct entry agreements already.”
Type of framework favouredThree most popular (in order of preference):
1. CEFR
2. ISLPR
3. NLLIA bandscales
Reasons for choice of frameworkCEFR:International currencyWell developed Specifies both proficiency levels and types of language study (e.g.social, academic, business)
ISLPR:Already in use
NLLIA bandscales:School-based and thus suitable for younger learners
Education about frameworksFour most popular options (in order):
In-house meeting or professional development sessions
Published information forwarded to each ELICOS centre
Externally run training sessions by industry
Government led training sessions
Nature of preferred framework and number of levelsNature of framework (in order):General EnglishAcademic EnglishWorkplace English
Number of levels:6 or more
Use of a frameworkMost important:
PlacementCertificationExit assessment
Also:End of level assessment Curriculum planning Self assessment
Acceptability to students“They will accept it only if it becomes the new
gatekeeper, they will welcome it if it can be shown to have status outside Australia (e.g. through its equivalence to European Reporting standards and standardised international tests).”
“I think they would believe it gives authority to their assessment. They would like it to provide a useful and recognised certificate for future work or study.”
Disadvantages of a framework
Two additional concerns:Two additional concerns:
Costs and resources e.g. Costs and resources e.g. need for an need for an independent moderating authorityindependent moderating authority
Lack of staff expertise in using standards Lack of staff expertise in using standards appropriately.appropriately.
Interviews7 of the questionnaire respondents and another DoS from a non-NEAS centre were interviewed.
Interviewees reinforced and extended the range of views expressed in the surveys.
Summary of key findingsMajor issues:
Variability of ELICOS providersAssessment a problem areaAttractiveness of a national frameworkProposed uses vary widelyMixed understandings about frameworksType of framework favouredProblems of implementationConcerns about homogenizing effect of a framework
ConclusionsMandating adoption of national framework
not feasible for ELICOS sector
However, could link existing frameworks, tests and curricula loosely to CEFR
RecommendationsIn-service and action research program allowing providers to familiarise themselves with CEFR
Further research into the creation of a national EAP certificate for “direct entry” into further education courses.
Development of an inventory of existing test materials already mapped onto CEFR and identifying gaps where new tests might be needed
ELICOS language levels ELICOS language levels feasibility study feasibility study
Final report Final report
Prepared for the Department of Prepared for the Department of Education, Science and TrainingEducation, Science and Training
byby Cathie Elder & Kieran OCathie Elder & Kieran O’’LoughlinLoughlin
April 2007April 2007