The NCLEX® Program ReportsBrian D. Bontempo, Ph.D.Daniel John WilsonMountain Measurement, Inc.

Agenda

History of the NCLEX® Program Reports Where do the NCLEX® Program Reports

data come from? How do you use the NCLEX® Program

Reports? NCLEX® Program Reports walk through Future Directions

History of the NCLEX® Program Reports

CTB-McGraw-Hill Chauncey Group International Pearson Educational Measurement

Maximum Inclusion, March 2000 Mountain Measurement

Moved to PDF in Fall 2005 Released the new Annual NCLEX Program

Reports in 2010

Where do the NCLEX® Program Reports data come from?

5-Digit Educational Program Code

What is a 5-digit educational program code? A 5 digit educational program code is a unique

identifier assigned to all nursing programs in NCSBN jurisdictions.

How is an edprog code used? Students enter this code on their NCLEX application. Pearson VUE attaches this code to the test results

data. Mountain Measurement analyzes the test results

data using these codes. Nursing Programs use this code to order their

NCLEX Program Reports.

NCLEX Program Reports Data Diagram

NCSBN assigns educational program codes

Educational program codes are publicized in NCLEX application

materials, the NCSBN website and the NCLEX Program Reports

website

Candidates take the NCLEX examination

Pearson VUE attaches the educational program code to the

test results data

Mountain Measurement creates NCLEX Program Reports by

associating each candidate to their program via their program code

Candidates use the program code on their NCLEX application

5-Digit Educational Program Code

Where can you find the list of educational program codes? https://www.ncsbn.org/NCLEX_Educational_Program_Co

des.pdf https://reports.mountainmeasurement.com/nclex/faq

Why do you need to know this?

The interpretation of the information is effected by the integrity of the data.

You can impact the integrity of your data by helping us to make sure that your students know their educational program code.

How do you use the NCLEX Program Reports?

Discussion

What do you do with the NCLEX Program Reports?

Why did you use the NCLEX Program Reports for?

What information did you use in the NCLEX Program Reports?

NCLEX® Program Reports Feedback Survey (2008 & 2009) Who uses the NCLEX Program Reports?

Most common users Nursing Program Director (37-48%) Dean of Nursing (29-31%)

74-85% reported at least two other personnel from their institution also use the NCLEX Program Reports

NCLEX® Program Reports Feedback Survey (2008 & 2009) How long do they spend using it

throughout the year? 84-85% spend half a day or more half spent a day or longer

NCLEX® Program Reports Feedback Survey (2008 & 2009) What are they using the reports for?

96-98% indicated using the reports to identify strengths and weaknesses or spot trends in performance

90-93% indicated using the reports to make comparisons to graduates from other programs

90-93% indicated using the reports to plan curriculum

NCLEX Program Reports Walk Through

Chapters 1 & 2: Introduction & Glossary

What's in the report? Report covers first-time graduates during the specified

time period Contents

Summary Overview NCLEX ® Test Plan Report Content Dimension Reports Test Duration/Test Plan Performance Report

The statistics included in each report include percentages, percentile ranks and averages

Median, percentile ranks and averages are only calculated when there are over 10 valid candidates per reporting period

Chapter 3 – Summary Overview

The Summary Overview section summarizes the passing rate for a program

Program performance is compared with performance from up to three comparison groups: Other (RN or PN) programs within your licensing jurisdiction (state or

territory) Other programs (for RN Programs Only) of a similar degree type

(BSN, Associate Degree or Diploma) All (RN or PN) programs within entire U.S. and its territories

Each program’s passing rate from past reporting periods is also provided for interpreting how performance changes over time

Chapter 4 - Test Plan Report The Test Plan Reports show how well the “typical

graduate” from a program performed on each subcategory of clients needs. Compares performance in each test plan area to: The performance of last year’s typical graduate The performance of graduates compared to candidates taking

the test during the same time in each of the three comparison groups

Performance in each test plan area is reported as the percentile rank of the program’s median graduate in reference to the comparison group

Calculating Percentile Ranks

Estimating student ability in each test plan area The measurement model used for the NCLEX® examination

allows for the estimation of student ability on any subset of items by adjusting for their difficulty

A program’s typical graduate’s performance The median (the midpoint in the set of a program’s student’s

ranked scores) score is used to indicate the typical program performance because the mean (average) of scores is more sensitive to extreme scores

A percentile rank for each test plan area indicates how a program’s typical graduate did on each content area in each of the reference groups

Percentile Rank DiagramEach student’s ability is estimated in each content area

The median score from each program is used to indicate typical graduate performance

All students with the same program code are aggregated together

Each program’s median performance is compared to the set of all students from the comparison group

The percentage of students in the comparison group that got the same or a lower score as the program’s median performance is reported as the percentile rank

Interpreting Percentile Ranks• Example: A percentile rank of 50 on Physiological

Adaptation in the sample graph indicates 50 percent of the graduates taking the NCLEX-RN® examination in your jurisdiction from April to September, 2010 scored less than (or equal to) the median of your graduates on items related to Physiological Adaptation

Since the median ability of a program (which tends to be less varied) is compared to individual ability scores, extreme percentile ranks are unlikely

Small differences in the median performance level of your graduates result in larger differences in percentile ranks in the middle of the distribution (near the 50th percentile) than at the extreme ends of the distribution.

NCLEX Percentile Rank Distribution

0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%0

10

20

30

40

50

60

70

80

Percentile Ranks of Programs

Freq

uenc

y

Chapter 5 - Content Dimension Reports The Content Dimension Reports show how well

your program´s typical graduate performed in the each of the content dimensions

Percentile ranks in these reports are determined the same way as in the Test Plan Report and are also used compare performance to the previous year’s typical graduate and with graduates from other programs

When an individual student does not take at least three items in a given content dimension, their score is not included in the calculation of the program’s median test performance

Chapter 6 - Test Duration/Test Plan Performance Report This section provides information about your

graduates NCLEX® examination testing experiences and about their performance in each of the test plan content areas The Test Duration Report provides information about the

number of questions answered and the amount of time spent on the NCLEX ® examination by your graduates

The Test Performance Report shows the percentage of questions the typical graduate from a program would be expected to answer correctly

Test Duration Report

The Test Duration Report reports separately for those who passed, failed and the total group: the average number of questions taken in each examination the average amount of time spent on the examination the percent of students taking the maximum and minimum

number of questions Information on this report can be used to understand

students’ testing experience Students taking the maximum number of items have scores

close to the passing standard Students taking the minimum number of items have scores that

are not close to the passing standard

Test Duration Report

Test Performance Report

The Test Performance Report shows the percentage of all possible questions in each area of the NCLEX® Test Plan that would be expected to be answered correctly, the performance based on the program’s typical graduate’s performance

The reports also include the expected percentages for each comparison group and for a student right at the passing score to enable comparisons to the program’s typical graduate’s performance

Test Performance Report

Future Directions

Annual NCLEX Program Reports!!!

On May 1, 2011, the first NCLEX® Annual Program Report (covering students testing between April 1, 2010 and March 31, 2011) will be released.

The Annual NCLEX Program Reports aggregate data over a one year period as opposed to the six month period used exclusively till now.

Subscriptions to the new Annual NCLEX® Program Reports can be purchased now.

Benefits

Provides a third comparison groups Annual Spring Fall

Larger sample size will add stability Smaller programs will now receive

complete reports

Introductory Offer!

2010-2011 NCLEX subscriptions will include the 2010-2011 Annual NCLEX Program

Report (covering students testing between Apr 1, 2010 & March 31, 2011)

Spring 2011 NCLEX Program Report (covering students testing between Sept 30, 2010 & March 31, 2011)

Introductory Offer!

2010-2011 NCLEX Program Reports $300 – RN $200 – PN

Transition Plan for Annual Reports

April 1st, 2010

Oct 1st, 2010 April 1st, 2011

Oct 1st, 2011 April 1st, 2012

Subscribers to the 2010-2011 Program Reports get two reports: the 2010-2011 Annual and Spring Reports – Both released May, 2011

2010-2011 Annual

Program Report

2010-2011 Spring

Program Report

2011-2012 Fall Program

Report

2011-2012 Spring

Program Report

2011-2012 Annual

Program Report

Subscribers to the 2011-2012 Program Reports can choose either Annual or Semi-Annual subscription or get both for a discounted price

2011-2012 NCLEX Program Reports

Semi-Annual Subscription $300 – RN $200 – PN

Annual Subscription $300 – RN $200 – PN

Joint Semi-Annual & Annual Subscription $450 – RN $300 – PN

Summary

Key Points

Know your 5-digit educational program code and make sure that your students know it and use it properly

The percentile ranks in chapters 4 and 5 of the reports are not traditional percentile ranks and must be interpreted differently

We invite you to subscribe to the New Annual NCLEX® Program Reports

Thank you for your kind attention and interest in the NCLEX® Program Reports

https://reports.mountainmeasurement.com/nclex/1 (800) 261-6227