Using  Simulation  to    Assess  Nurse  Competence    

       

Hinton, J. E., Mays, M. Z., Randolph, P. K., Hagler, D., Kastenbaum, B., Brooks, R., DeFalco, N., & Miller, K.  

   

Disclosure(s)  Measuring  Competency  with  Simulation  Phase  II  

   

Funded  by:    National  Council  of  State  Boards  of  Nursing  (NCSBN)  Center  for  Regulator  Excellence  (CRE)  grant    

Using  Simulation  to  Assess  Nurse  Competence        

   Disclosure(s)  cont’d  

Measuring  Competency  with  Simulation  Phase  II    No  relevant  relationships  with  financial  interests  to  disclose:  

     Randolph,  P.  K.  &  Mays,  M.  Z.  

 Relationship  with  financial  interests  to  disclose:    

 Hinton,  J.  E.,  DeFalco,  N.  &  Miller,  K.  ,  employees  of  SCC:      AZBN,  Nursing  Performance  Evaluation  

   Hagler,  D.,  Kastenbaum,  B.,  &  Brooks,  R.,  employees  of    ASU:    AZBN,  Nursing  Performance  Evaluation  

     

Using  Simulation  to  Assess  Nurse  Competence        

Objective    Explore  efficacy    of  simulation  tests    for  board  decision    support  and    remediation  

 Using  Simulation  to  Assess  Nurse  Competence      

 

Background      v BONs  need  an  independent  and  accurate  assessment  of  a  nurse’s  competence  

 v During  Phase  I  of  “Measuring  Competency  with  Simulation”  our  team  developed  a  41  item  assessment  tool  and  simulation  process  that  resulted  in  a  reliable  and  valid  measure  of  nursing  competence  

 

 Using  Simulation  to  Assess  Nurse  Competence        

Background  cont’d    v Phase  II  focused  on  defining  the  relationship  of  competency  to  educational  achievement,  years  of  experience,  experience  with  simulation,  practice  area,  level  of  practice,  nursing  certification,  supervisor  assessment  and  self-­‐assessment  

 

 Using  Simulation  to  Assess  Nurse  Competence  

 

Phase  II  Method    *  Currently  employed  RNs  (N  =  67)  volunteered  to  be  tested  in  three  acute  care  simulation  scenarios  *  All  201  tests  were  video-­‐recorded    *  Nursing  experts  (N  =  31)  were  trained  to  use  the  assessment  tool  to  rate  competence    *  Each  video  was  independently  rated  by  three  experts  *  Consensus  ratings  were  used  to  create  a  Nursing  Performance  Profile  (NPP)  for  each  nurse  tested  

   Using  Simulation  to  Assess  Nurse  Competence  

 

Phase  II  Participants    *  9%  males,  91%  females  *  Average  age  49  (SD  =  12)  *  Experience  ranged  from  2  –  45  years    *  77%  had  10+  years  *  Education  ranged  from  Associate  to  Doctoral  degree  *  16%  ADN,  37%  BSN,  40%  MSN,  6%  DNP/PhD  *  Experience  with  simulation  varied  *  31%  None,  47%  Occasional,  22%  Frequent  

     

Using  Simulation  to  Assess  Nurse  Competence  

Phase  II  Raters    *  10%  males,  90%  females  *  Average  age  52  (SD  =  9)  *  Experience  ranged  from  3  –  55  years    *  93%  had  10+  years  *  Education:  29%  BSN,  45%  MSN,  23%  DNP/PhD  *  Experience  with  simulation  varied  *  23%  None,  42%  Occasional,  36%  Frequent  

     

Using  Simulation  to  Assess  Nurse  Competence  

Phase  II  Results  1    *  Education:  Nurses  with  BSN  or  MSN  degrees  had  significantly  better  NPP  than  those  with  ADN  degrees.  *  Groups  effect  in  3  x  9  ANOVA,  F(2,186)  =  4.64,  p  =  .01  *  Sim  Experience:  Nurses  with  frequent  simulation  experience  had  significantly  better  NPP  than  those  with  no  or  occasional  simulation  experience.  *  Groups  effect  in  3  x  9  ANOVA,  F(2,189)  =  8.97,  p  <  .001  

Using  Simulation  to  Assess  Nurse  Competence  

Phase  II  Conclusion  1    * Simulation  experience  was  more  clearly  predictive  of  competence  than  years  of  experience  in  nursing,  experience  in  advanced  practice,  or  advanced  education/certification  *  4  factors  explain  this  finding  

 

Using  Simulation  to  Assess  Nurse  Competence  

Phase  II  Results  2    *  Supervisor:  19  participants  provided  supervisor  assessment  data.    Nurses  with  a  supervisor  rating  of  90%  or  higher  had  significantly  better  NPP  than  those  with  low  or  no  supervisor  assessment.  *  Groups  effect  in  2  x  9  ANOVA,  F(1,199)  =  5.04,  p  =  .03  *  Self:  61  participants  provided  self-­‐assessment  data.    Nurses  with  a  self-­‐rating  of  90%  or  higher  did  not  have  significantly  better  NPP  than  those  with  low  or  no  self  assessment  (p  >  .05).  

Using  Simulation  to  Assess  Nurse  Competence  

Phase  II  Conclusion  2    * Nurses  who  were  highly  regarded  by  supervisors  were  highly  competent  in  simulation  testing  *  Supervisor  assessments  were  not  available  for  nurses  who  were  less  competent  

* Self  evaluation  was  unrelated  to  competency  in  simulation  testing  

Using  Simulation  to  Assess  Nurse  Competence  

Conflicting  information  prompted  the  Board  to  order  a  Nursing  Performance  Evaluation  on  7  nurses—Here  are  the  results  of  2  representative  cases  

 A  Tale  of  2  Cases  

 

Nurse  A  

* Nurse  on  probation  pending  completion  of  documentation  course—asked  BON  to  terminate  probation  because  nurse  could  not  find  work  * Documentation  course  (completed)  *  12  months  supervised  practice  (unable  to  complete)  

Nurse  A  Investigative  Information  

* Probation  for  * Medication  errors  * Documentation  errors  * Transcription  errors  * Failing  to  document  assessments  *  Failing  to  assess  * Terminated  from  2  facilities  

Nurse  A  Simulation  Performance  

* Multiple  errors  relating  to  medication  administration  *  Left  a  medication  on  the  MAR  that  pt.  was  allergic  to  * Reported  IV  fluid  incorrectly  to  oncoming  nurse  * Administered  twice  medication  dose  ordered  

* Multiple  errors  related  to  assessment  *  Failure  to  assess  affected  limb  and    respiratory  distress  * Did  not  re-­‐check  blood  glucose  after  treating  for  hypoglycemia  * Reported  lab  value  inaccurately  to  physician  

Nurse  A  Simulation  Performance  

* Multiple  errors  related  to  documentation  * Physician  telephone  order  written  with  a  dose  different  from  prescribed  *  Inadequate  documentation  of  assessment  * Did  not  include  time  and  signature  on  all  entries  * Recorded  medication  doses  different  from  doses  administered  

 

Nurse  A  Simulation  Performance  

* Other  errors  related  to  safety  and  communication  * Did  not  do  appropriate  patient  teaching  * Did  not  use  2  patient  identifiers  *  Lack  of  hand  hygiene  

Nurse  A  Simulation  Performance  

Nurse  B  

* Reported  for  sub-­‐standard  care  * Denies  all  allegations—admits  only  to  documentation  errors  * Completed  documentation  class  

Nurse  B  Investigative  Information  

*  Multiple  instances  of  sub-­‐standard  care  *  Failed  to  document  administration  of  controlled  

substances  *  Failed  to  obtain  ordered  CT  for  bowel  obstruction  

in  timely  manner—Critical  thinking  *  Failure  to  assess  groin  catheter  site  post  cardiac  

cath  *  Discontinued  heparin  drip  inappropriately—

medication  error  *  Numerous  other  practice  incidents  in  other  settings  

Nurse  B  Simulation  Performance  

Assessment  *  No  initial  pulse  check,  cap  refill,  or  sensory  check  on  post-­‐op  

fractured    extremity  

Critical  Thinking  *  Did  not  request  blood  glucose  on  pt.  treated  for  

hypoglycemia  *  Failed  to  note  presence  of  physician  orders  *  Did  not  provide  food  to  hypoglycemic  pt.  c/o  hunger  *  Lack  of  timely  intervention  on  pt.  with  hypoxia.  

Procedural  *  Did  not  position  ortho  pt.  correctly  

 

Nurse  B  Simulation  Performance  

Communication  errors  *  Inaccurate  teaching  *  Lack  of  conflict  resolution  

Medication  errors  *  Administered  3x  ordered  dose  of  medication  *  Failed  to  resume  IV  infusion  after  administration  of  medications  

Documentation  errors  *  Physician’s  name  wrong  *  Wrong  dose  documented  *  Inaccurate  time  reported  

Safety  *  Did  not  read  orders  back  to  physician  *  Did  not  identify  client  prior  to  medication  administration  *  Hand  hygiene  deficits  

 THE  NPE  EXAM  REFLECTS  A  NURSE’S  HABITS—YOU  CANNOT  HIDE  HABITS!      .....So,  how  do  you  change  habits?    

Conclusion  

Simulation:    Competence  &  Remediation  

Simulation:    Competence  &  Remediation  

Simulation:    Competence  &  Remediation  

Simulation:    Competence  &  Remediation  

Memory  systems  and  their  anatomical  loci.      (Modified  from  Squire  and  Knowlton,  1994)    

 

Declarative  (Explicit)  

Nondeclarative  (Implicit)  

Facts   Events   Skills  &  Habits  

Priming  &  Perceptual  Learning  

Simple  Classical  

Conditioning  

Nonassociative  Learning  

Medial  Temporal  Lobe  &/Diencephalon  

Reflex  Pathways  

Emotional  Responses  

Skeletal  Responses  

Striatum  Neocortex  

Amygdala  

Cerebellum  

Long-­‐Term  Memory  

 

SKILLS  DECAY          

                                                         &  REACQUISITION      

 Simulation:    Competence  &  Remediation  

 

CPR  requires  ~50  separate  psychomotor  skills  performed  in  a  specific  sequence    (Flint  et  al.,  1993)        

Simulation:    Competence  &  Remediation  

Number  of  CPR    re-­‐certifications  was  a  better  predictor  of  performance    than  days  trained  prior  to  testing  (Anderson,  2008)      

Discussion  *  It  is  unlikely  the  contribution  of  simulation  experience  to  

competence  was  solely  due  to  familiarity  with  the  testing  modality  

*  Findings  suggested  that  simulation  experience  supported  continued  competence  for  nurses  more  fully  than  other  conventional  methods  

 *  The  NPP  used  in  conjunction  with  high-­‐fidelity  simulation  

showed  promise  for  identifying  unsafe  nursing  practice  in  a  controlled  environment  that  did  not  put  patients  at  risk  

Using  Simulation  to  Assess  Nurse  Competence  

Limitations  *  This  project  did  not  distinguish  between  simulation  

experience  that  occurred  during  formal  education  to  obtain  an  RN  or  APRN  license  versus  that  which  occurred  during  workplace  settings  for  orientation,  credentialing,  or  certification  

*  Nor  did  this  project  assess  what  forms  or  amount  of  simulation  experience  best  support  continued  competence  

Using  Simulation  to  Assess  Nurse  Competence  

Conclusions  *  Nurses  of  substantially    different              levels  of  competence  will  volunteer              for  opportunities  for  high-­‐fidelity              simulation  testing      *  There  is  potential  for  simulation              testing  to  further  the  science  of              assessing  and  promoting              nursing  competence  

Using  Simulation  to  Assess  Nurse  Competence  

References   Anderson,  G.  &  Gaetz,  M.  (2008).  CPR  and  first  aid  skill  retention.  University  College              of  Frazier  Valley,  1-­‐131.  Arne,  M.  (2011)  Experience-­‐dependent  structural  plasticity  in  the  adult  human  brain.                Trends  in  Cognitive  Sciences.  15(10),  475-­‐482.  Byrne,  J.  H.  (Ed.)(2008).  Learning  and  Memory:  A  Comprehensive  Reference.                Academic  Press.  Duvivier,  R.  J.,  Van  Dalen,  Jan,  Muijtjens,  A.  M.,Mulaert,  V.  RMP,  VanDer                Vleuten,  C.  PM,  &  Scherpbier,  A.  JJA  (2011).  The  role  of  deliberate  practice  in              the  acquisition  of  clinical  skills.  BMC  Medical  Education,  11(101).  Retrieved              from  http://www.biomedcentral.com/1472-­‐6920/11/101  Eisenburger,  P.    &  Safar,  P.  (1999).  Life  supporting  first  aid  training  of  the  public              review  and  recommendations.  Resuscitation,  41(1),  3-­‐18.  Flint,  L.S.,  Billi,  J.E.,  Kelly,  K.,  Mandel,  L.,  Newell,  L.,  &  Stapleton,  E.R.  (1993).                  Education  in  adult  basic  life  support  training  programs.  Annals  Emerg.  Med.  22(2,              Pt  2):468-­‐474.  

 

 Using  Simulation  to  Assess  Nurse  Competence  

James,  J.  T.  (2013)  A  new,  evidence-­‐based  estimate  of  patient  harms  associated                with  hospital  care.  Journal  of  Patient  Safety,  9  (3),  122-­‐128  doi:  10.1097/PTS,              0b012e3182948a69  Lamb,  W.  (2009).  We've  had  a  miracle  on  the  Hudson.  North  Jersey  Media  Group.                Retrieved    from:                    http://www.northjersey.com/breakingnews/Reports__of_passenger              _jet_down_in_Hudson.html#sthash.yRmu9EcP.dpuf  McKenna,  S.P.,  &  Glendon,  A.I.  (1985).  Occupational  first  aid  training:  Decay                in  cardiopulmonary  resuscitation  (CPR)  skills.  J.  Occup.  Psych.  58,  109-­‐              117.  O’Neil,  H.  &  Rivera,  N.  (2011).  Educational  psychology  and  human  factor  issues                  involved  in  studying  (or  evaluating)  degradation  of  skills.  Skills  Deterioration                  Symposium.  Newport  Beach,  CA.  Pashler,  H.,  Rohrer,  D.,  Cepeda,  N.  J.,  &  Carpenter,  S.  K.  (2007).  Enhancing                learning  and  retarding  forgetting:  Choices  and  consequences.  Psychonomic                Bulletin  and  Review,  14,  187-­‐193.  

 Using  Simulation  to  Assess  Nurse  Competence