MCAT UpdateJudith ByrneDirector, MCAT Administration & Reporting

Jen PageDirector, MCAT Preparation Products

Scott OpplerDirector, MCAT Development & Psychometrics

November 5, 2011

MCAT UpdateMCAT Update• Test Administration 2011 Review 2012 Preview

• Resources for Examinees• Test Scores and Research 2011 Test Scores Using MCAT Data in Student Selection

• Upcoming MR5 Sessions

2

Test Administration:2011 in Review

3

We saw overall increasesWe saw overall increases

2009 2010 20112009 2010 2011

Scores Released 79,344 82,033 85,754

Unique Examinees 68,823 70,941 75,313

1 Attempt 58 768 60 297 64 8691 Attempt 58,768 60,297 64,869

2 Attempts 9,686 10,225 10,018

3 Attempts 369 419 426

4

Number of unique examinees increased by 6% in 2011 increased by 6% in 2011

78,000

80,000

75,313

74,000

76,000

78,000

68,823

70,941

70,000

72,000

66,000

68,000

62,000

64,000

2009 2010 2011

5

Number of scores increased by 5%

85,75486,000

88,000

82 033

84,000

79,344

82,033

80,000

82,000

79,344

78,000

76,0002009 2010 2011

Number of voids increased by 5% in 2011

4,8975,120

5 000

6,000

4,2184,000

5,000

2,000

3,000

0

1,000

2009 2010 2011

7

14% of examinees retested within 2011

64,869

60 000

70,000

50,000

60,000

30,000

40,000

10,01810,000

20,000

4260

1 Attempt 2 Attempts 3 Attempts

Shaking things up

• 28 administrations in 2011 (Jan – Sept)th• PLUS one additional exam on October 7th

to accommodate earthquake examinees from August 23rdfrom August 23rd

• 112 examinees tested on October 7th

• Scores for October 7th released on October 27thOctober 27

Numbers of 2011 examinees varied by race/ethnicityrace/ethnicity

34,02535000

40000

25000

30000

17,760

15000

20000

897

6,654 6,4465000

10000

8970

American Indian and Alaskan Native

Asian Black or African American

Hispanic or Latino White

10

Numbers of examinees in each group increased, some more than othersincreased, some more than others

Percent Group 2010 2011 Change

American Indian and Alaskan Native 857 897 5%

Asian 16,574 17,760 7%

Black or African American 6,185 6,654 8%

Hispanic or Latino 5,632 6,446 14%

White 32,640 34,025 4%

11

35% increase in accommodations applications approved in 2011applications approved in 2011

399400

450

292 295300

350

400

200

250

50

100

150

02009 2010 2011

12

Still a very small percentage of examinees tested with accommodationsexaminees tested with accommodations• < .5% testers with accommodations• Most commonly provided accommodations include:

Extra testing timeTime and a quarter• Time-and-a-quarter

• Time-and-a-half• Double-time

E t B k• Extra Breaks

Separate testing rooms• Specifically for disabilitySpecifically for disability • To allow other accommodations, such as food and

drink

Additional accommodations offered• Food and drink, medication, other personal medical items

• Zoom Text or other magnification• Adjustable workstations and other seating adaptations

• Extra breaks• Paper exams• Readers and scribes• Readers and scribes

FAP approvals have gone up by approximately 8% in 2011pp y

6 6967 000

8,000

4,827

6,2036,696

5,000

6,000

7,000

3,000

4,000

0

1,000

2,000

02009 2010 2011

15

Benefits to FAP Recipients in 2011• Reduced MCAT registration fee of $85 (from $235))• $500 to help defray costs of re-evaluations for those seeking accommodationsg• Free copy of The Official Guide to the MCAT ®

Exam • Free copy of MSAR ®

•14 free AMCAS applications•14 free AMCAS applications

Test-Day Survey Results

84% 84% 86%

80%

90%

100%

60%

70%

80%

40%

50% Satisfied

Dissatisfied

10%

20%

30%

3% 3% 4%0%

10%

2009 2010 2011

Test Administration:2012 Preview

18

No major changes for 2012• Planning for analogous volume increases

S&R d O t b 12th f th 2012 t ti• S&R opened October 12th for the 2012 testing season

O 4k i t t i d• Over 4k registrants on opening day• Same distribution of test dates• Base registration fee has increased by $5• No major policy changes, but have updated webNo major policy changes, but have updated web and print material to communicate better

19

New AdditionsNew roles, but not new faces:Jeremy Manus, M.A.Supervisor, MCAT Examinee Servicesp(202)862-6095

Oscar WhitemanSupervisor, Accommodated Services(202)862-6004

New to AAMC:Kathryn Bugbee, Ph.D.Director Accommodated TestingDirector, Accommodated Testing(202)862-6073

Resources for ExamineesUpdates to existing resources &Updates to existing resources & new resources planned

What do we know about student preparation? p pHow did you prepare for the examination you took today (select all that apply)?took today (select all that apply)?

• Used an MCAT preparation book 77%

• Took one or more MCAT practice tests online 69%

• Took an MCAT preparation course provided by a private company 44%company 44%

• Took one or more paper-based MCAT practice tests 25%

T k MCAT ti id d b (• Took an MCAT preparation course provided by your (or another) university 8%

• No Response 3%• No Response 3%

Preparing for the MCATResources from AAMC:• The Official Guide to the MCAT® Exam• MCAT Practice Tests• New Resources

The Official Guide to the MCAT® Exam

Updated content includes:

• Current data on MCAT scores & GPA• Updated info on changes in MCAT

scores after retestingscores after retesting• 2 new writing samples, with sample

essays y

Buy online for $30 at: www.aamc.org/officialmcatguide

Official Guide: Two SectionsNarrative: 7 chapters Practice: new questions

with detailed analysis

Part I: Narrative Overview of the MCAT Exam MCAT Fundamentals Test-Day Process and Test Day Process and

Procedures

All About MCAT Scores

Your MCAT Score as Part of the Admissions Decision

To Retake or Not Retake…THAT is the Question

Th St t M ti The Strategy Mystique: Debunking the Myths

Practice Effectively Practice Effectively

Part II: Practice

Content and Skill Content and Skill ClassificationsClassificationsClassificationsClassifications

TipsTips

Diffi lt L lDiffi lt L lDifficulty Level:Difficulty Level:Medium (45%)Medium (45%)Explanation of correct Explanation of correct

& incorrect answers& incorrect answers

The Official Guide to the MCAT® Exam

Great for getting familiar with the exam!

• 140 previously administered items, none found on practice tests

• Overview of “everything MCAT”• Overview of everything MCAT

Available online for $30 at: www.aamc.org/officialmcatguideg g

*free to FAP recipients*free to FAP recipients

MCAT Practice Tests• Updated look, same tests• Free practice test

available to allavailable to all• 7 additional tests for sale• $35 each$35 each

www.e-mcat.com

MCAT Practice Tests• Simulates actual testing

environmentwww.ewww.e--mcat.commcat.com

• Real MCAT questionsC t i bl• Customizable practice modesS l ti b t t• Solutions by test developers

• Detailed score• Detailed score reports

Detailed Score Reports

What’s next? ®The Official MCAT® Self-Assessment Package

•Coming in Spring 2012•3 Self-Assessment Tests

• Physical Sciences Bi l i l S i• Biological Sciences

• Verbal Reasoning R lt i di id l f t th•Results compare individual performance to the average performance on the same items

•Provides guidance on where to focus test•Provides guidance on where to focus test preparation efforts

•Help students use practice data to guide studyHelp students use practice data to guide study

Data to Support Individualize Prep®The Official MCAT® Self-Assessment Package

•Detailed analysis by:• Exam section • Content category within a section• Self rated confidence pre & post test• Self-rated confidence pre & post test• Items self-identified as “guessed”• Cognitive skills testedg• Item difficulty and type (discrete or passage-based)

Practice Test Data to Track Progressg•Baseline test, before preparation begins•Follow-up tests, to track improvementsFollow up tests, to track improvements

New AdditionsCreating more resources

Abby Thomsen, M.A.Abby Thomsen, M.A.Instructional Designer, MCAT Preparation Productsathomsen@aamc.org(202) 828-0664 ( )

Test Scores and Research

Test Scores and Research

• Comparison of 2010 and 2011 test scores“Using MCAT Data in Medical Student• “Using MCAT Data in Medical Student Selection”

• Looking to the future: MR5 and the new MCAT• Looking to the future: MR5 and the new MCAT

T S d R hTest Scores and ResearchComparison of 2010 and 2011Comparison of 2010 and 2011 scores

How did examinees score on the MCAT in 2011?• Results in 2011 were very similar to thoseResults in 2011 were very similar to those

reported in 2010 Total scores Section scores

38

2010 vs. 2011 Total scores

7

5

6

ge

2010 (Mean=25.0)2011 (Mean=25.1)

2

3

4

Perc

enta

g

0

1

3 6 9 12 15 18 21 24 27 30 33 36 39 42 45

Total Score

39

2010 vs. 2011 MC Section Scores2010 vs. 2011 MC Section Scores

MCAT Section 2010 2011MCAT Section 2010 2011

Physical Sciences 8 3 8 4Physical Sciences 8.3 8.4

Biological Sciences 8.8 8.8g

Verbal Reasoning 7.9 7.9

2010 vs. 2011 Writing Sample Scores2010 vs. 2011 Writing Sample Scores

Percentile 2010 2011Percentile 2010 2011

25th percentile M M25 percentile M M

50th percentile O Op

75th percentile Q Q

Test Scores and Research“Using MCAT Data in MedicalUsing MCAT Data in Medical Student Selection”

Using MCAT Data in Medical Student SelectionSelection• Expanded from version of document sent to

admissions offices last yearadmissions offices last year• Purpose is to provide admissions committees

with information relevant to the use of MCATwith information relevant to the use of MCAT scores and other applicant information in the selection of this year’s medical school

liapplicants• Please take one with you (and keep it handy)!• We want to hear your feedback…

Questions addressed:Questions addressed:• What does the MCAT exam measure?• How is the MCAT exam scored?• How well do examinees score on the MCAT

?exam?• How accurate are examinees’ MCAT scores

and how should they be interpreted?and how should they be interpreted?• Why are some MCAT scores marked as non-

standard?standard?

44

Questions addressed (cont’d):Questions addressed (cont d):• How do examinees’ scores change when they

retake the MCAT exam and how do admissionsretake the MCAT exam and how do admissions officers use scores for applicants who test more than once? [EXPANDED][ ]

• How do admissions officers use MCAT scores and other application data in the holistic review

f li t ’ lifi ti ? [EXPANDED]of applicants’ qualifications? [EXPANDED]• How well do UGPAs and MCAT scores predict

students’ performance in medical school?students performance in medical school? [EXPANDED]

45

How do examinees’ scores change when they retake the MCAT exam?• MCAT examinees can test up to three times inMCAT examinees can test up to three times in

one calendar year, with no lifetime limit• In 2010, approximately 47% of applicants , pp y pp

reported scores from multiple test administrations in their applicationsO th l th i iti l MCAT• On average, the lower the initial MCAT score, the greater the amount of score improvement

• However the degree (and direction) of score• However, the degree (and direction) of score change varies considerably across individuals

46

Changes in MCAT Total Scores for First-time Examinees Retesting in Same Year (2008-2010)

47

How well do UGPAs and MCAT scores predict students’ scores predict students performance in medical school?

I thi ’ d t h id d d d• In this year’s document, we have provided an expanded set of “grids” depicting the relationships between UGPA, MCAT Total scores, and a variety of medical education

t i bl i l dioutcome variables, including: 4- and 5-year graduation rates

Withd i b i di i d f d i Withdrawing or being dismissed for academic reasons

First attempt and eventual pass rates for Step 1,First attempt and eventual pass rates for Step 1, Step 2-CK, and Step 2-CS exams (see Appendix)

48

Percentage of 2003-2005 Students who Graduated from Medical School in 4 Years, by MCAT and UGPA Ranges

6

7

4

5Series 1Series 2

3

4 Series 2Series 3Series 4Series 5

1

2Series 5Series 6

0Category 1 Category 2 Category 3 Category 4

Percentage of 2003-2005 Students who Graduated from Medical School in 5 Years, by MCAT and UGPA Ranges

6

7

4

5Series 1Series 2

3

4 Series 2Series 3Series 4Series 5

1

2Series 5Series 6

0Category 1 Category 2 Category 3 Category 4

Percentage of 2003-2005 Students Passing Step 1 Exam on the First Attempt, by MCAT and UGPA Ranges

6

7

4

5Series 1Series 2

3

4 Series 2Series 3Series 4Series 5

1

2Series 5Series 6

0Category 1 Category 2 Category 3 Category 4

Percentage of 2003-2005 Students Eventually Passing Step 1 Exam, by MCAT and UGPA Ranges

6

7

4

5Series 1Series 2

3

4 Series 2Series 3Series 4Series 5

1

2Series 5Series 6

0Category 1 Category 2 Category 3 Category 4

Test Scores and ResearchLooking to the Future: MR5 and theLooking to the Future: MR5 and the New MCAT

The 5th Comprehensive Review of the MCAT• After 3 years of dedicated effort, the MR5After 3 years of dedicated effort, the MR5

committee has made its final recommendations regarding changes to the MCAT exam

• Among these recommendations are: The addition of a new section that will include

content from psychology, sociology, and biology The addition of content typically taught in first

semester introductory biochemistry coursessemester introductory biochemistry courses The elimination of the Writing Sample section

54

The 5th Comprehensive Review of the MCAT• The committee has recommended that the newThe committee has recommended that the new

MCAT be introduced in the Spring of 2015

• To support the development of new testTo support the development of new test content, the AAMC plans to begin administering an optional un-scored section at the end of the current exam beginning in January 2013current exam, beginning in January 2013

• To prevent an increase in the length of the testing day the decision has been made to stoptesting day, the decision has been made to stop administering the Writing Sample at that time

55

The 5th Comprehensive Review of the MCAT• For more information about theFor more information about the

recommendations of the MR5 committee, please join us at the following sessions: MR5 Update (Today,12:00 – 1:00 p.m., Convention

Center, Rooms 406/407)AAMC Focus Session: “Rethinking Admissions: AAMC Focus Session: “Rethinking Admissions: Gateway to the Profession of Medicine” (Tuesday, 8:00 – 9:30 a.m., Convention Center, Rooms 205/207)205/207)

56

And last but not least…..

• Please join us in welcoming the most recent addition to the MCAT team:addition to the MCAT team: Dr. Cindy Searcy, Director of MCAT

Research Phone number: (202) 862-6105

57

Questions?

58

©