This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Maximizing the Utility of English Assessment ScoresNew Directions in English Language Assessment Conference: Mexico City
March 21-22, 2019
Jason Turner
Sr. Vice-President, Global Services
MetaMetrics
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Overview
• Linking Aptis with the Lexile Framework for
Reading• Aptis Teens Advanced, Aptis General, Aptis Advanced
• CEFR
• Aligning Readers and Texts• Graded Readers
• S. Korean Textbooks, U.K. and U.S. texts
• U.S. text level expectations
2
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Why Link?
• Provides supplementary information to extend the utility
of the test score and CEFR information
• Provides an additional layer of feedback, connecting the
test to real-world reading
• Provides a way of benchmarking real-world reading
demands and a way to evaluate the gap between reading
ability and reading demand of texts
3
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
What is the Lexile Framework?
4
The most widely used standard
for matching readers with with
texts, and connecting learners
to level-appropriate resources.
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
The Lexile Scale & Concept
Student Measures
Advanced Readers
Beginning Readers
More Complex Text
Less Complex Text
Book Measures
200L
680L
940L
1190L
1410L
1050L
230L
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
The Reader & Text Relationship
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved. 7
Obtained
through
measuring text
with the
Lexile Analyzer
Software
Evaluates Text
Characteristics
Obtained
through
Tests of
English
Reading
Ability
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved. 8
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Aptis
Score Reports
• Scaled scores for each component assessed (e.g.
Reading, Writing) on a 0-50 score scale
• CEFR levels reported for each skill (e.g. Listening,
Reading)
• Overall CEFR level when all 4 skills taken
9
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Aptis Reading Test – Example Tasks
10
Skill Focus CEFR
Level
Task Description Cognitive Processes
Inter-sentence
cohesion
A2 Reorder jumbled sentences
to form a cohesive text
Careful reading, global,
inferencing, building a mental
model
Reading short
paragraphs-main idea
B1 Matching description of
person with main idea of
paragraph
Careful reading, global
Reading longer text,
integrating related
propositions
B2 Matching headings to
paragraphs within a longer
text
Expeditious reading, global
Careful reading, global
Reading across
extended sections of
related text
C1 Select text to complete gaps
in texts within two related
selections
Careful reading, global
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Aptis-Lexile Linking Studies
• Lexile linking items administered at the same time as
Aptis reading items
• Aptis General and Advanced – October - November 2015
• Aptis Teens Advanced – November 2017 - April 2018
• Lexile linking items, N = 25 ranging in difficulty to align
with the Aptis test being linked.
11
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Sample Lexile Item970L
12
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Aptis-Lexile Linking Studies
Samples
• Aptis General and Advanced
• 563 examinees from 8 countries
• Austria, Columbia, Great Britain, Greece, Malaysia,
Romania, Turkey, and Vietnam
• Aptis Teens Advanced
• 1,662 examinees from China
13
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Aptis-Lexile Linking Function
Linking function determined by Rasch
(Winsteps) analysis and then smoothing at
the ends of the distribution (when necessary)
14
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Aptis General Linking Function
15
0
200
400
600
800
1000
1200
1400
1600
1800
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Aptis Advanced Linking Function
16
0
200
400
600
800
1000
1200
1400
1600
1800
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Aptis Teens Advanced Linking Function
17
0
200
400
600
800
1000
1200
1400
1600
1800
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50
CEFR
B1
805L
985L
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Aptis CEFR Levels on the Lexile Scale
18
0
200
400
600
800
1000
1200
1400
1600
1800
2000
A1 A2 B1 B2 C1 C2
Lexile
Measure
s
Advanced
Teens Advanced
General
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Maximizing Utility
19
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Provide Targeted Practice
A Placement Test places your
customer on the Lexile scale
Your Student
Your Content
Your measured content provides targeted practice for your
customers
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
What Else Do Lexile Measures Tell Us?
Lexile Measures allow a customer to compare their current ability with real
world English reading demands and opportunities.
Reading Demand Examples in the World
Your customer
Cu
sto
me
r Le
xile
Me
asu
re
Reading Demand Examples
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Lexile Reading Stories:
What does reading demand look like around the world?
Daily Reading
Reading Demand Examples in the World
College Entrance
Career and Workplace Readiness
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
• Benchmarking
• Commonly read materials – Lexile map
• Graded readers
• Textbooks and exam materials
• International newspapers
• U.S. grade level text expectations
23
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
The Lexile Map
24
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
The Lexile Map
25
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Graded Readers
26
• Internationally prominent graded reader publishers
• Four series
• 422 graded readers across 7 levels
• 319 had a single CEFR designation
• Research Questions
• Is there a systematic increase in reading challenge across
graded reader levels and CEFR levels?
• Is there variation across publishers in the reading challenge of
books designated at similar levels?
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Graded Readers
27
0
200
400
600
800
1000
1200
1400
A C D A B C A B C D A B C D A B
Lexile
Measure
Series and CEFR Level
A1 A2 B2B1 C1
Series and CEFR Level
A1 A2 B2B1 C1
Series and CEFR Level
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Graded Readers – Aptis Teens CEFR Levels
28
0
200
400
600
800
1000
1200
1400
A C D A B C A B C D A B C D A B
Lexile
Measure
Series and CEFR Level
A1 A2 B2B1 C1
Series and CEFR Level
A1 A2 B2B1 C1
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Textbooks and Exam Materials
• English Medium Contexts in China – Research Study
• 10 textbooks were analyzed for text complexity
• Biology, chemistry
• English language, World modern history
• Maths core and extended, Pure Maths 1, Pure Maths 2 & 3
• Mechanics
• 18iGSCE exam papers taken at English medium schools
• Research Questions
• Does the complexity in the texts match the exam?
• Is the complexity comparable to the reading material in other
English-speaking and non-English speaking countries?
29
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved. 30
Comparing with Aptis Teens Examinee Scores
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved. 31
Comparing with Aptis Teens Examinee Scores
Aptis Teens Advanced
Mean B2 (1033L)
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved. 32
English Textbooks used in South Korea
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved. 33
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved. 34
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved. 35
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Lexile Text Measures by US K-12 Grade
Levels
36
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Lexile Text Measures and US K-12 Text
Levels
37
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
Summary
Linking an existing assessment to the Lexile Framework
extends the utility of the test scores
- Matching readers to text
- Benchmarking against existing text sets to anticipate
need for scaffolding
- Understanding reading demands of international
reading materials
38
This presentation constitutes the confidential information of MetaMetrics, Inc.
Copyright © 2018 MetaMetrics, Inc. All rights reserved.
THANK YOU
39