The NCSBN National Simulation Study: Results from Year 1

2012 NCSBN Scientific Symposium

September 11, 2012

Arlington, VA

Study Collaborators

Jennifer Hayden, MSN, RN NCSBN, Project Director

Pamela Jeffries, PhD, RN, FAAN, ANEFJohns Hopkins University

Suzan Kardong-Edgren, PhD, RN, ANEFBoise State University

Washington State University

Nancy Spector, PhD, RNNCSBN

Study Phases

Phase II-Multi-site, randomized study of simulation/clinical experiences

Purpose

Determine if there are differences in clinical competency among graduating nursing students with 50%, 25% or up to 10% of traditional clinical hours substituted with simulation experiences.

Study Groups

Control Group (clinical as usual) Up to 10% of clinical time can be in simulation

25% simulation in place of clinical 50% simulation in place of clinical

Research Questions

1. Are there differences in clinical competency among graduating nursing students in the three study groups?

2. Are there differences in knowledge among graduating nursing students in the three study groups?

3. Are their perceived differences in how well learning needs are met in the clinical and simulation environments among the three study groups?

4. Are there differences in clinical competency among the three study groups in each of the core clinical courses?

Null Hypothesis

There are no differences between the study groups in knowledge, clinical competency, and learning needs being met.

Substitution Rate

1 hour of traditional clinical is being replaced with 1 hour of simulation time

Time spent in a simulation lab either as an active participant or active observer of a simulated scenario followed by debriefing

This time in the simulation lab is counted towards required clinical hours for graduation

Core Clinical Courses

Randomization will be maintained in the core clinical courses

Foundations Medical/Surgical Nursing Advanced Medical/Surgical Nursing Obstetrical Nursing Pediatric Nursing Mental Health/Psychiatric Nursing Community Health/Public Health Nursing

Study Will Evaluate:

Clinical competencyCreighton Competency Evaluation Instrument

(CCEI)

Nursing knowledgeATI Comprehensive Assessment and Review

Program

How well student learning needs were met in the clinical and simulation environmentsClinical Learning Environment Comparison Survey

(CLECS)

Defining Simulation

Operational definition: An activity or event replicating clinical practice. For the purpose of this study, simulation includes multiple types of scenario based simulation using high fidelity manikins, medium fidelity manikins, standardized patients, role playing, and computer based simulations.

NLN/Jeffries Simulation Framework

Study Management at each School

Team Leader Study Team

Trained in the protocol Trained in the debriefing technique Large part in the development of the simulation

curriculum

Clinical Faculty Training

Clinical faculty rate students each week of clinical and each day of simulation using the Creighton Competency Evaluation Instrument

Each semester clinical faculty undergo online training in the use of the instrument

Calculate inter-rater reliability and intra-rater reliability each semester

Clinical faculty review objectives and “norm” what are competent/not competent expectations

Simulation Day

Entire clinical group reports to the lab with the instructor

Assigned roles: primary nurse, secondary nurse, family member, documenter, med nurse, observers

15-20 minutes for scenario followed by 20+ minutes debriefing

Clinical instructors rate their students with the CCEI

2-3 clinical groups in the lab at the same time

Stations

Manikin scenarios Standardized patients/patient actors Skills Stations Group Activities

Layered Learning Critical Thinking Cards Sensory kits Group Therapy in a Box

Computerized scenarios ATI Real Life Virtual Clinical Excursions

Scenarios Available

CourseNumber of Scenarios

Available

Foundations 26

Medical-Surgical 64

Advanced Medical-Surgical 46

Obstetrical 42

Pediatrics 67

Psychiatric-Mental Health 24

Community Health 23

Total 292

Student “Safety”

Safety data collected each day of simulation and each week of clinical

Oversight Committee at each school Reviews school level safety data, mid-term

grades, final grades, ATI scores, feedback from faculty and study team

Data Safety Monitoring Board established Meets monthly to review school level data and

national data to ensure participants’ education is not being jeopardized by being in the study

Participant Eligibility

18 yo or older Provide informed consent Full-time RN student Complete the nursing program with study

cohort

Ineligibility Licensed as LPN/VN or RN Accelerated RN student

The Students

Gender: 86% Female

Age: 56% 18-24

28% 25-34

16% 35+

Race: 83% White 8% Black

6% Asian 1% Native American

1% Hawaiian/PI 2% Multiracial

Ethnicity: 18% Hispanic

Student Demographics

Previous Degree: 35% (295)

62% (181) Bachelor’s Degree

41% (120) Associate Degree

3% (9) Master’s Degree

Certified Nursing Assistant: 16% Yes

Paramedic/EMT: 19

Military Experience:

28 Yes

8 Medical Corps

Study Numbers

847 Students consented and randomized411 ADN 436 BSN

54 Failed a course39 ADN 15 BSN

96 Withdrew40 ADN 56 BSN

697 Continuing on to Year 2332 ADN 365 BSN

82% rate of continuing in the study

A closer look at attrition

Clinical Instructor Demographics(N=102)

Gender: 95% Female

Race: 79% White 6% Black

7% Asian 1% Native American

Ethnicity: 9% Hispanic

Age: 20% Less than 35

24% 35-44

18% 45-54

39% 55+

Clinical Instructor Demographics

Teaching experience: 94% Yes

average 9.0 years (SD 8.7)

Current position:

43% Instructor

29% Adjunct faculty

13% Asst or Associate Professor

4% Professor

14% Other

Clinical Instructor Simulation Experience

21% No previous experience

55% Observed simulation

47% Run simulation scenarios

50% Debriefed students after simulations

38% Assessed students who have participated in simulations

23% Written simulation scenarios

33% Participated in formal simulation training

Clinical Hours

Range of total clinical hours w/o capstone:

520-776 hours

25%= 130-194 hours of sim

50%= 260-388 hours of sim

What have we learned so far…

Study Model is working Use of a trained simulation team Clear objectives 15-20 minute scenarios Having clinical instructors observe Having students observe their peers Debriefing

Skeptical faculty are now on board

Incidental Findings

Informal training/norming of clinical faculty-- more inclusion of adjunct faculty

Bonding of students in their clinical group Clinical instructors are using DML in their

post-conferences Johns Hopkins University—focus groups of

clinical instructors and how this experience is informing their teaching

Study Timeline

Current Status of Phase II

Study Students are in the senior year of their programs

Students will graduate in May 2013 We will collect NCLEX pass rates through

December 31, 2013 Final results are expected in Spring 2014

Looking Ahead to Phase III

Phase III will follow graduates of the study sites into clinical practice

Purpose of Phase III is to evaluate how well students were prepared for professional practice

Evaluations will include: Clinical competencies Safety competencies Job satisfaction Retention Stress

What can regulators take away

Who is conducting the sims? Do they have training, what kind?

What kind of debriefing? How will students spend their time while in

sim? How long are sims? Are there clear objectives for each sim? Evaluations/graded? Is the sim lab accredited?

www.ncsbn.org/2094.htm

jhayden@ncsbn.org