Queensland University of Technology

CRICOS No. 00213J

Not Another Essay!Transforming assessment in an evidence-based

practice nursing unit

Dr Leigh Davis

Ms Moira Cordiner

CRICOS No. 00213Ja university for the worldrealR

Session Overview

• Describe the process we used to design and deliver the first iteration of an EBP curriculum

• Present the first iteration of the curriculum with students’ and teachers’ evaluations

• Suggest implications to inform future iterations so that the effectiveness of the curriculum is improved.

CRICOS No. 00213Ja university for the worldrealR

What is Evidence Based Practice?

Is about making clinical decisions “on the basis of the best available scientific evidence but recognizing patient preferences, the context of healthcare and the judgment of the

clinician.”

Pearson, A., et al., The JBI model of evidence-based healthcare. International Journal of Evidenced Based Healthcare, 2005. 3: p. 207-215.

CRICOS No. 00213Ja university for the worldrealR

Why the move to Evidence Based Practice (EBP)?

• Increasing volumes of health care research.

• Rising health care expenditure

• Health care professionals are accountable for their own practice

• Consumers are becoming more informed

• Reduce the variation in practice decisions

• Important to healthcare quality assurance processes and accreditation

CRICOS No. 00213Ja university for the worldrealR

A model for Evidence Based Practice

• Translation of uncertainty to an answerable question

• Systematic retrieval of best evidence available

• Critical appraisal of evidence for validity, clinical relevance, and applicability

• Application of results in practice

– Incorporation of evidence into hospital policies, guidelines

– Using evidence in our everyday practice

• Evaluation

CRICOS No. 00213Ja university for the worldrealR

EBP and University Curricula

• Universities have begun to incorporate EBP in undergraduate healthcare degrees

• No evidence to design or evaluate an EBP undergraduate nursing curriculum

Ciliska, D. (2005). Educating for Evidence-Based Practice. Journal of Professional Nursing 21, (6), (pp345-350)

CRICOS No. 00213Ja university for the worldrealR

Implementation of first iteration

Team: discipline expert + academic staff developer

3 steps:1. Understand cohort

characteristics2. Design a major assessment

task – ‘not another essay’3. Scaffold students’ learning

CRICOS No. 00213Ja university for the worldrealR

Excerpt of the clinical scenario• You are a newly registered nurse working at Matilda

Memorial country hospital. 

• A 3 year old child with no known pre-existing medical conditions is admitted to the hospital with fever of unknown origin for investigation. 

• Nurses at the hospital manage uncomplicated fever differently.

• The hospital’s Director of Nursing has asked you to:– identify the best available evidence on the nursing

management of uncomplicated fever in children

– propose a draft policy based on the information you find.

CRICOS No. 00213Ja university for the worldrealR

Evaluation of first iteration

Three sources of data: • University-wide voluntary student evaluation of

the unit (quantitative and qualitative data)

• anecdotal information from students and tutors

• the means of students grades/marks for the worksheet and the major task

CRICOS No. 00213Ja university for the worldrealR

Evaluation (cont)

Feature of the unit % satisfied/very satisfied(N=194)

Assessment: Level of difficulty62%

Assessment - Relevance to Topic72%

Relevance of the tutorial sessions72%

Written Assessment Mean

Worksheet 15 marks 10.9

Major Essay “ Submission for a practice change” 50 marks 34

CRICOS No. 00213Ja university for the worldrealR

Limitations1. Problems with the first data source (QUT student evaluation of unit)• not from a randomised sample of students from the unit• we have no input into the generic construction of the survey.• survey administered by the university three weeks before students

have submitted the major task or sat for their exam • small number of comments (44) - qualitative data is not

representative. 2. University policy prevents us from:• matching characteristics of particular students to the feedback they

give us• obtaining details about the cohort, e.g. percentages who were

mature-aged; the tertiary entrance scores of students who started the degree directly from high school, and the ratios of ESL students

CRICOS No. 00213Ja university for the worldrealR

What did the data indicate?

1. most students found the subject relevant and could achieve reasonable results in the assessment tasks

2. only 62% found the difficulty level satisfactory, despite the fact that the subject was at an introductory level

3. diverse and opposing opinions about the unit’s relevance, difficulty, challenge of major task, placement of the unit in the degree

4. many students ‘described’ rather than analysed and synthesised research information

5. failure rate not high – comparable to other units

CRICOS No. 00213Ja university for the worldrealR

Main problem: insufficient alignment

• construct validity not as high as intended• authentic task but• construct validity compromised by insufficient

authentic learning of EBP skills before applying them in the worksheet and major task

CRICOS No. 00213Ja university for the worldrealR

Conclusions

1. change the pedagogy in tutorials to provide more

– scaffolding– practice of analysis and

synthesis skills– reinforcement and

feedback (Caldwell, 2007)

2. remove the requirement for students to produce a ‘policy document’ as part of the major task –is too challenging

CRICOS No. 00213Ja university for the worldrealR

Reflection

• transformed some students views about assessment

• underestimated the amount of practice third year students needed to develop analysis and synthesis skills

• believe that EBP should be incorporated into all units and not be offered as a standalone unit