Date post: | 23-Dec-2015 |
Category: |
Documents |
Upload: | carmel-banks |
View: | 216 times |
Download: | 0 times |
Peggy Garza Associate BILC Secretary
For Testing Programs
BILC Standardization Initiatives and
Conference Objectives
• Standardization Efforts
- Language Testing Seminars - Tailored Workshops - Toolbox for Language Testers- STANAG 6001 Testing Conferences
• Conference Topics and Goals
Outline
• Language Testing Seminar (LTS)
• Advanced Language Testing Seminar (ALTS)
• Language Standards and Assessment Seminar (LSAS)
BILC Testing Seminars
LANGUAGE TESTING SEMINAR(LTS)
Language Testing Seminar (LTS)
• Developed and conducted by BILC testing experts
• Background: A BILC initiative aimed at promoting a common interpretation of STANAG 6001 and consistency in language proficiency testing
• Specific objective: To give NATO/PfP language teaching professionals the opportunity to practice developing test items based on STANAG 6001
Language Testing Seminar (LTS)
• Held since Autumn 2000 at the Marshall Center, Garmisch, Germany
• Two-week graduate-level seminar
• Offered twice times a year (February and September)
• More than 350 graduates from NATO/PfP/MD nations
ADVANCED LANGUAGE TESTING SEMINAR
(ALTS)
Advanced Language Testing Seminar (ALTS)
• Need for continuing professional development for test developers identified
• Stakeholders’ meeting in September 2008
• Result: Advanced Language Testing Seminar (ALTS)
• Validated in 2009
• Target audience: LTS graduates who continue to work on STANAG 6001 testing- STANAG 6001 Edition 4 standardization
sessions- Focus on Level 3 for practical exercises- Opportunity to exchange best practices
• Modular approach to seminar
Advanced Language Testing Seminar (ALTS)
• Pre-seminar assignments- Approximately 40 hours
• Three-week seminar- Module 1: Receptive Skills: Listening and Reading (1 week)- Module 2: Productive Skills: Speaking and Writing (1 week)- Module 3: Test Analysis (3 days)- Module 4: Managerial Issues (2 days)
Advanced Language Testing Seminar (ALTS)
NEW
LANGUAGE STANDARDS AND ASSESSMENT SEMINAR
(LSAS)
Language Standards and Assessment Seminar (LSAS)
3-11 July 2013• For language teachers and program managers• Objectives– STANAG 6001 familiarization– Guidelines for aligning instruction to standards– Test and measurement principles– Classroom assessment and practical exercises
Testing Seminar Schedule in 2014
• Language Testing Seminar (LTS)‒ 10-21 February‒ 15-26 September
• Advanced Language Testing Seminar (ALTS)‒ 2- 20 June
• Language Standards and Assessment Seminar (LSAS)‒ 23 Apr – 1 May ‒ 3-11 December
• Recommended by ALTS participants
– Samples of listening and reading items at levels 1-3
– Samples of speaking and writing tests at levels 1-3, including plus levels
– Frequently Asked Questions (FAQs)
– Self-Assessment Outline for STANAG 6001 Testing Programs
STANAG 6001 Toolbox for Testers
BILC Website
www.natobilc.org
Tailored Workshops for National Testing Programs
• Ukraine‒ November 2011, 2012
• Moldova‒ May 2012
• Russia‒ June 2013
Tailored Workshops
• Zagreb, 2009– Focus on standardization, moderation, and validation
• Sarajevo, 2010– Dates: 31 August – 2 September
– Focus on testing speaking
• Stockholm, 2011– Dates: 6-8 September
– Focus on testing writing
• Copenhagen, 2012– Dates: 3-6 September, 2012
– Focus on testing listening
STANAG 6001 Testing Conferences
2013 STANAG 6001 Testing Conference
• Some of your questions:– Our stakeholders have questions. Is there advice for
how best to communicate with the stakeholders?– Our stakeholders are asking us to test more and more
people. What should we tell them?– We have developed items. What’s next?– How do we know if our tests are good?– We need to hire new testers. Is there a position
description for testers? How can we recruit for testers?
– How can we keep professionally current ?
2013 STANAG 6001 Testing Conference• Conference format– Presentations– Guest speakers from NATO and SHAPE– Panel discussions
o Resolving challenges
– Study Groups meetings and reportso Making a validity argumento Washbacko Action planso Renorming and standardizing testerso Piloting items and prompts
Specific Conference Goals• Review and clarify the entire test construction process• Identify action plans for test development and
administration • Provide ideas for selecting the appropriate testing
techniques• Discuss piloting and pretesting • Learn more about determining test validity• Identify practicality issues and discuss lessons learned• Discuss best practices in renorming and standardizing
testers• Learn about innovations in testing in the BILC community
– Computer-delivered testing– Proficiency and specialist testing– Testing at Level 4
Test Life Cycle Diagram
Communicating with the Stakeholders
• Identify the stakeholders• Coordinate with the stakeholders during the needs
analysis and planning stages– Stakeholders approve personnel and resources
• Close the loop with the stakeholders during the Test Life Cycle– Action plans– Technical reports– Updates– Washback
Test Construction
Needs Analysis
Planning
Design
Development
Administration
Monitoring
Test Specifications
Pre-testing / AnalysisStandard SettingEvaluation/review
Evaluation
Revision
Stage 2
Stage 3
Stage 4
Stage 1
Stage 5
Stage 6
Development Stage Part 1
Text selection
Item writing
Item review I (moderating)
Piloting on a small scale
Item review II (editing)
Assembly of trial test
• Retain
• Revise
• Reject More items needed?
Development StagePart 2
Pre-testing/trialling on a larger scale
Statistical Analyses
Standard Setting
Item banking
Assembly of final version of test
Item revision• Retain
• Revise
• Reject
2013 STANAG 6001 Testing Conference
• Expectations– Active participation and interaction– Frank and open discussions– Provide answers to the difficult questions– Create a forum for future networking and
cooperation• BILC appreciates your participation and we look
forward to a very productive conference!