+ All Categories
Home > Documents > National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade...

National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade...

Date post: 02-Jan-2020
Category:
Upload: others
View: 1 times
Download: 0 times
Share this document with a friend
47
National Approach to Practice Assessment for Nurses and Midwives Exploring Issues in the use of Grading in Practice: Literature Review Final Report (Volume 1) September 2009 Contact person: Prof. Morag A. Gray, Associate Dean (Academic Development), Faculty of Health, Life & Social Sciences, Merchiston Campus, Colinton Road, Edinburgh EH10 5DT Tel: 0131 455 2465 [email protected]
Transcript
Page 1: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

National Approach to Practice

Assessment for Nurses

and Midwives

Exploring Issues in the use of

Grading in Practice:

Literature Review

Final Report

(Volume 1)

September 2009

Contact person: Prof. Morag A. Gray, Associate Dean (Academic Development), Faculty of

Health, Life & Social Sciences, Merchiston Campus, Colinton Road, Edinburgh EH10 5DT

Tel: 0131 455 2465 [email protected]

Page 2: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 2 of 47

National Approach to Practice Assessment for

Nurses and Midwives

Exploring issues in the use of Grading in Practice:

Literature Review

Final Report

(Volume 1)

Prof. Morag A. Gray

Dr. Jayne Donaldson

September 2009

Page 3: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 3 of 47

Contents

Page

Executive Summary 6

1 Final Report 8

1.1 Aim 8

1.2 Objectives 8

2 Methodology 8

2.1 Data Sources included / excluded (phase 1) 8

2.2 Data extraction from included studies (phase 2) 10

2.3 Level of evidence on data extracted 11

3 Introduction to findings 12

3.1 Grading practice 12

3.2 Argument in favour of grading practice 12

3.3 Argument against grading practice 13

4 Issues related to grading of nursing and midwifery practice

14

4.1 Context of grading nursing and midwifery practice 14

4.2 Grading Tools 15

4.3 Grade inflation 24

4.3.1 Reasons for grade inflation: students 22

4.3.2 Reasons for grade inflation: assessors 22

4.3.3 Reasons for grade inflation: student – assessor

relationship

25

Page 4: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 4 of 47

4.3.4 Reasons for grade inflation: tool design 25

4.3.5 Suggested ways of controlling grade inflation 26

4.4 Use of rubrics in grading tools 27

4.5 Challenges related to grading of nursing and

midwifery practice

28

4.5.1 Challenges: Time 28

4.5.2 Challenges: Assessors 28

5 Issues related to validity and reliability in

grading of professional practice

29

5.1 Validity 30

5.2 Reliability 31

5.3 Challenges: Assessment Tool 31

5.4 Improving validity and reliability in assessing

practice

32

6 Implications for mentor preparation and support

in respect of grading of nursing and midwifery

practice

33

6.1 Consistency and accountability 34

6.2 Reliability 34

6.3 Methods and content of training programmes 34

7. Conclusions and Recommendations 37

8. References 39

Page 5: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 5 of 47

9. Tables

Table 1 List of professions found to have published on the

grading of practice

9

Table 2 Number of literature sources within each category 11

Table 3 Grading Tools 16

Table 4 Reasons for grade inflation 26

Table 5 Suggestions for content of assessor training and update sessions

35

Table 6 Suggestions for mode of delivery for training and update sessions

36

10 Appendices (in Volume 2)

Appendix 1 List of search criteria and databases used in electronic searching

Appendix 2a Reference list from the primary search

Appendix 2b Reference list from hand search

Appendix 3 Overview of excluded studies

Appendix 4 Overview of included studies

Page 6: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 6 of 47

Executive Summary

This literature review focuses on exploring issues of grading in practice, including

reliability, validity and the implications for mentor preparation and support.

Recommendations are made to inform the National Approach Working Group in

formulating guidance to Higher Education Institutions around the grading of practice.

Literature review

Literature was collected following a systematic search and spanned across fourteen

professional groups. In total a 119 articles were reviewed following application of

exclusion criteria. Of these, the majority were quantitative in nature (n=66); followed

by descriptive accounts or opinion based (n=28); literature reviews (n=19) and

qualitative (n=6).

Challenges in grading practice

Grading practice reflects the conclusion of a decision making process which

indicates how well a student is progressing in respect to a standard or criteria and

flags up areas where students can improve.

There are a number of documented challenges when grading practice. Some of the

challenges such as time available, consistency and accountability of assessors are

not specific to only grading practice. Those which are specific to grading relate to

validity and reliability issues of the tools used and grade inflation.

Grading Tools

The review of the literature presents a number of grading tools or systems, and we

conclude that evaluation of these tools is under-developed in terms of their

effectiveness, usefulness, reliability and validity. Grade inflation is a well-

documented problem and there are some suggested methods (although not fully

evaluated) from the literature to control this. The most promising of those appears to

be the use of rubrics and focused training of assessors.

Rubrics

Carefully constructed rubrics can ameliorate grade inflation. Rubrics are made up of

three key components: clearly defined performance criteria or elements; detailed

descriptions of what a performance looks like at each level (or grade) of proficiency;

and a rating scale which most commonly uses a three or four points. Rubrics can be

used both formatively and summatively.

Page 7: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 7 of 47

Training of assessors / mentors

If nursing and midwifery were to use grading within practice, there are a number of

training requirements suggested by the literature for mentors: developing an

understanding and interpretation of educational terminology; how to accurately

assess learning; how to assign grades; how to use the tools consistently and

effectively; how to write evidence to support the graded assessment given; how to

deal with borderline students; and how to deliver effective feedback. The time and

methods of training and updating mentors needs to be considered.

Conclusions and recommendations

From the literature reviewed, it is not possible to make any generalisations about

grading of practice. The findings from this literature review come from a wide source

of professional literature that often include limitations of being set in one

geographical area, in one profession, and calculated on relatively small sample sizes.

Most quantitative studies were survey design, and /or presented using descriptive

statistics only. Therefore the generalisability of findings from these quantitative

studies should be necessarily cautious. Only six studies demonstrated the use of a

qualitative theoretical basis for the study. A number of limitations were noted and

could be prone to bias. The literature review articles were mostly descriptive

accounts, of for example a grading tool, or an opinion on grading, or on a developed

tool. It should be borne in mind that opinion can be subject to writer bias. The level of

evidence on grading of clinical practice across the literature tends to sway towards

the lower end of the evidence band i.e. the usefulness, reliability, validity and

effectiveness of grading of practice has still to be proven.

If the grading of practice is adopted, we recommend that consideration is given to

the following:

The development, testing and use of rubrics;

The use of rubrics for formative as well as summative assessment;

The use of a multi-method approach to assessment;

Comprehensive training and updating sessions for assessors;

Ongoing evaluation and monitoring of the grading process used.

Page 8: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 8 of 47

1. Final Report

1.1 Aim

The aim of this final report is to present NHS Education for Scotland (NES)

with the findings from the systematic literature / evidence review aimed to

inform the development of a National Approach to Practice Assessment for

the pre-registration Nursing and Midwifery programmes in Scotland.

1.2 Objectives

Undertake a literature / evidence review that focuses on exploring issues of

grading in practice, including reliability, validity, and the implications for

mentor preparation and support.

Make recommendations based on the above review to inform the National

Approach Working Group in formulating guidance to Higher Education

Institutions around the grading of practice.

2. Methodology

2.1 Data Sources included/excluded (phase 1)

Literature was collected following a systematic search of the literature.

Appendix I (Volume 2) lists the search criteria and databases used in

electronic searching, which revealed 164 references. These articles also

referred to other literature on grading which the electronic search had not

discovered, and these further 28 sources were included in the first phase of

the review as part of a hand searching exercise.

E-mail contact asking for help in the identification of grey literature was made

to the RCN Research and Development Co-ordinating Centre; the Scottish

Heads Academic Nursing & Allied Health Professions and the Council of

Deans of Health; the International Council of Nurses (ICN) Research Network;

the International Network for Doctoral Education in Nursing and Nursing

Knowledge International. As a result, three additional sources were included

within the literature base.

An article list from the initial phase can be found in Appendix 2 (Volume 2).

There were a number of professions found to have published on the grading

of practice and these are demonstrated in Table1. The literature being

reviewed spans the professional groups as specified in Table 1.

Page 9: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 9 of 47

Professional groups

Student Nurses

Registered Nurses

Student Midwives

Registered Midwives

Medical Students

Qualified Medical Practitioners

Speciality Training for Doctors

Physiotherapy Students

Osteopathy Students

Social Work Educators

Social Work Students

Dental Students

Pharmacy Students

Initial Teacher Education

Students

Table 1: List of professions found to have published on the grading of

practice

Every piece of literature was independently reviewed by the reviewers (MG

and JD), and were included or excluded from the study. The data extracted

from each literature source was summarised by each reviewer and where

there was disagreement, reviewers met to discuss and agree

inclusion/exclusion.

Exclusion criteria included literature that did not use a grading tool or system

(n=71 excluded) and not available in the English language (n=3 excluded),

are highlighted within reference list included in Appendix 2 (Volume 2). An

overview of excluded studies (n=71) along with papers (n=23) excluded from

hand searching can be found in Appendix 3 (Volume 2).

None of the literature revealed the use of randomised controlled trails, and the

reviewers considered it appropriate to use literature which employed

quantitative and qualitative methodology, descriptive accounts, text and

opinion. The remaining 119 literature sources were then used in the second

phase of the study. An overview of included 119 studies can be found in

Appendix 4 (Volume 2).

Page 10: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 10 of 47

In summary:

Number of articles found by electronic-searching = 164

o Number of articles included = 119 for phase 2

o Number of articles excluded = 45

Number of articles found by hand-searching = 51

o Number of articles included = 28 for phase 2

o Number of articles excluded = 23

Total number of articles reviewed in phase 1 = 215

Total number of articles included for phase 2 = 119

Total number of articles excluded during phase 1 = 98

2.2 Data extraction from included studies (phase 2)

Data from included studies were extracted and themed into the areas presented

within the findings of this report. Therefore, themes that emerged were:

1. What are the issues in grading of any professional practice?

o Definition of graded practice

o Argument in favour of grading

o Argument against grading

o Grade inflation

o Grading tools

o Importance of context

o Associated dilemmas, difficulties, barriers, challenges

2. What are the issues related to reliability and validity in grading of professional

practice?

o Validity, reliability – depending on type of assessment tool used

o Inter-rater reliability – depending on type of assessment tool used

o Importance of context

o Associated dilemmas, difficulties, barriers, challenges

Page 11: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 11 of 47

3. What are the implications for mentor preparation and support in respect to

grading of professional practice?

o Training courses

o Use of paper-based training packs

o Associated dilemmas, difficulties, barriers, challenges

2.3 Level of evidence on data extracted

Table 2 demonstrates the number of included literature sources using

quantitative methodology, qualitative methodology, literature reviews,

descriptive accounts, text and opinion.

Classification

of study*

Quantitative Qualitative Literature

review

Descriptive

account, Text

and Opinion

Number of

literature

sources

66 6 19 28

Total number of articles reviewed

**7 studies were not reviewed as they were unable to be

sourced due to the short time scale of this review

119**

Table 2: Number of literature sources within each category

**where studies used mixed method, the study was counted as quantitative for

the purposes of distinguishing between levels of evidence.

Once data were extracted, the reviewers collated the findings, which are

presented below. Those articles included within the findings section can be

found within the reference list for this report.

Page 12: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 12 of 47

3. Introduction to findings

For clarity within this report individuals (for example, mentors; co-mentors;

sign-off mentors; preceptors; supervisors; consultants; programme directors)

who assess students in practice are referred to as assessors.

3.1 Grading practice

The grading of practice reflects the conclusion of a decision making process

(Lanphear 1999; Sadler 2005). Grading practice involves making a decision

based on the assessment of performance which allows recognition of merit or

excellence beyond awarding a mere pass (Andre 2000; Williams & Bateman

2003; Hill et al., 2006).

“Grading ... provides feedback to students and staff and because students

like to know how they are getting on rather than only that they have

passed or failed” (Moon 2002: 73).

Moon (2002: 85) defines grade assessment criterion as “a standard of

performance that a learner must reach in order to be allocated a particular

grade within a hierarchy of grades. In this case there is likely to be a series of

grade assessment criteria related to the different grades”.

During the literature review it was apparent that some authors used „grading‟

as a term to rate a single event (such as rating a single task undertaken),

while others referred to this as a combination, or grade point average, of

scores, marks or grades to indicate an overall result (Sadler 2005). Grading

within this document will refer to any scale (numerical, alphabetical or

descriptive) which has been used to rate any type of student performance

whether that was during a specific assessment task or continuous

assessment.

3.2 Argument in favour of grading practice

With the move of Nursing & Midwifery education into Higher Education, Andre

(2000) asserts that grading of practice became increasingly relevant. “A

practice-based discipline such as nursing, that espouses the value of applying

skills to practice, needs to consider how such value is communicated in

academic form” (Andre 2000: 672). Glover et al. (1997) argue that not to

grade students‟ performance in practice is devaluing this important aspect of

their education. Andre (2000) adds to this by stressing that by not grading

practice high achieving students are disadvantaged as their accomplishment

is not overtly rewarded.

Page 13: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 13 of 47

The “...„Holy Grail‟ of being able to fairly assess practice

would achieve what may be seen as a very desirable goal for

measuring the „art‟ of midwifery as well as its science.

Students who excel at practice might benefit from this aspect

being formally acknowledged” (Darra et al., 2003: 44).

Others make an argument for grading practice in terms of improving the

quality of the learning experience. Grading practice improves the learning

experience since it provides students‟ with detailed feedback on their

performance (ElBadrawy & Korayem 2007; Johnson 2007; Johnson 2008),

and allows both students and their assessors to rate students over time and

note patterns of performance (Ben-David et al. 2004; Seldomridge & Walsh

2006; Holaday & Buckley 2008). A number of authors argue that grading of

practice provides students with a strong incentive and motivational driver for

students to perform at their best (Williams & Bateman 2003; ElBadrawy &

Korayem 2007; Johnson 2007; Johnson 2008). Johnson (2007: 27) also

states that grading of competence-based practice is particularly important in

the context of “national and international moves towards developing unified

frameworks for linking qualifications”.

3.3 Argument against grading practice

There are two main arguments against the use of grades. Firstly there are

those opposed to grading on the grounds that it is not compatible with the

principles of competency-based assessment (Andre 2000; Williams &

Bateman 2003). Secondly, there are arguments that focus on the negative

impact of grading students and others as illustrated in Sharp‟s opinion below:

“Medical degrees are usually awarded on a pass/fail or

satisfactory / unsatisfactory basis since while most patients

would be pleased to be treated by a first or possibly upper

second class doctor, the presence of lower second or third

class doctors in the NHS would do little to promote public

confidence in the service. Doctors are either qualified to

practice of they are not” (Sharp 2006: 146).

Sharp (2006) postulates whether the same argument would hold for other

professions.

The negative impact of grading practice on students revolves around the

unfairness of this to less able students who can become demotivated due to a

sense of failure even though they are „passing‟ (Williams & Bateman 2003);

the push for competiveness between students rather than collaboration

(Williams & Bateman 2003) and the cause of anxiety and stress in students

(Ravelli & Wolfson 1999).

Page 14: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 14 of 47

In their comparative study to measure the impact of a change in grading

system in the first 2 years of medical school from graded to pass/fail on

medical students‟ academic performance, attendance, residency match,

satisfaction and psychological well-being, Bloodgood et al. (2009) found that

the move from grading to pass/fail was not associated with a decline in

students‟ academic performance nor attendance. The change to pass/fail

grading system resulted in a statistically significant improvement in students‟

well-being (particularly females). Rohe et al. (2006) suggested that the use of

a pass/fail rather than a grade reduces stress and anxiety in students and

increase group cohesion.

4. Issues related to grading of nursing and midwifery practice

From the review of the literature there are a number of issues identified

related to grading of nursing and midwifery practice: importance of context;

grading tools, grade inflation, use of rubrics in grading tools and challenges.

These will be discussed in turn.

4.1 Context of grading nursing and midwifery practice

The context in which grading nursing and midwifery practice takes place is

argued by a number of authors to be critical (Neary 2000a,b; Clouder & Toms

2005; Cowan et al. 2005; Yorke 2005; Allen et al. 2008; Cassidy 2009). There

is increasing emphasis being placed on using authentic assessments. Mueller

(20081) defines authentic assessment as a form of assessment in which

students are asked to perform real-world tasks that demonstrate meaningful

application of essential knowledge and skills. Yorke (2005) asserts that the

more accurate terminology to use is the assessment of authentic performance.

Whilst acknowledging the importance of context in the grading of practice,

Johnson (2007) warns of the inherent danger of inconsistency in different

assessors‟ judgements.

1 http://jonathan.mueller.faculty.noctrl.edu/toolbox/whatisit.htm (last accessed 28/08/09)

Page 15: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 15 of 47

“Context might interfere with consistency in at least two ways.

First, context might interfere with an assessor‟s ability to

position the qualities of two difference performances on a

common scale. Factors may exist that intrude on the process

of casting consistent judgements (e.g. performance tasks

involving interactions between individuals might be

interpreted differently by judges who accommodate

variations in social dynamics, such as dealing with „tricky‟ as

opposed to „helpful‟ customers). Secondly, context can make

it more difficult to infer the basis on which assessors‟

decisions are being made. Assessors in different contexts

might make different judgements based on different

foundations from each other because their understanding of

competence is based on their different experiences”

(Johnson 2007: 29).

4.2 Grading Tools

From the literature search, details of sixteen grading tools were found. Three

claimed to be valid and reliable and were used in assessing physiotherapy,

dietetics and dental students. One tool used for post-registration nurses is

reported to be valid but lacking in inter-rater reliability. Another tool used in

pre-registration nursing is reported to be „work-in-progress‟. A tool used for

dental students claims to have inter-rater reliability, whilst another two tools

have reduced grade inflation. The valid and reliable tool used in physiotherapy

is a national tool.

In total the grading tools reflect the following professional groups: dental

students (n=3); Dietetic students (n=1); Medical students (n=3); Pre-

registration nursing students (n=3); post-registration nursing students (n=2);

Pharmacy students (n=1); Physiotherapy (n=2); Radiography (n=1).

According to Moon (2002: 91) “grade assessment criteria provide a scaling of

how well learners achieve above the threshold”. For the tools discussed

above, there is a wide variation in the range of scales used from 4 (n=3); 5

(n=6); 6 (n=3); 9 (n=2); 40 (n=1) and one used a visual analogue scale of 0-

100mm. Four tools also incorporated an overall grading score. Details of the

grading tools can be found in Table 3.

Page 16: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 16 of 47

Table 3: Grading Tools

Author(s)

Description of Tool Description of Grading Notes

Burchell et al. (1999)

Revised assessment tool using four generic headings: 1. Relationships with, and

responsibilities to patients 2. Professional integrity 3. Professional relationships and

responsibilities 4. Professional standards

6-point grading scale used with minimum competence specified at level 5. Level 1 = considerable development required; 2 = moderate development required; 3 = adequate performance in some respects; 4 = satisfactory performance in most respects; 5 = professional standard; 6 = exceptional.

Authors acknowledge more work was required in terms of validity and reliability of tool. Grading tool used in the assessment of radiography students in one area in UK.

Chambers (1999)

Competency Rating Scale has 4 sections: 1. Overall rating 2. Four items related to diagnosis and

judgement 3. Technical skills 4. Patient management

Overall grading encourages the adoption of a global and ability-orientated approach. It is not an average rating.

9 point rating scale used.

A grade of 3 or 4 signals that remedial help required. Importantly, a member of Faculty must write what actions need to be taken to help the student improve.

A grade of 1 or 2 signals failure.

Grading tool used in the assessment of dental students in USA.

Positive feedback from assessors using tool and author reports good inter-rater reliability.

Page 17: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 17 of 47

Author(s)

Description of Tool Description of Grading Notes

Dalton et al. (2009a)

Assessment of Physiotherapy Practice (APP) is a practical instrument that reflects the Australian Standards for Physiotherapy. APP has 20 items divided into 7 domains: 1. Professional Behaviour 2. Communication 3. Assessment 4. Analysis & Planning 5. Intervention 6. Evidence-based practice 7. Risk Management

Each item is scored on a scale from 0-4, with 4 denoting greater apparent competence. A score of 2 indicates that the student has achieved a level of competency that would be expected of an entry level graduate on their first day of practice. Scores of 3 and 4 reflect that the student is demonstrating comfort and sophistication respectively. A score of 1 indicates that competence is not yet adequate.

APP items are assessed on student performance of observable behaviours using performance indicators. Performance indicators are used for both formative and summative purposes.

A global rating score is used for summative assessments.

Single instrument with known validity and reliability has replaced 25 distinct assessment forms which were formally in use.

Grading tool used in the assessment of physiotherapy students in Australia.

Dudek et al. (2008)

Completed Clinical Evaluation Report Rating.

14 items on a 5-point scale. Numeric ratings must adequately capture performance and accompanied by well-written comments that are specific, detailed and use concrete examples. Written comments must justify the rating provided.

Authors claim this is a valid and reliable tool. Grading tool used in the assessment of medical students in Canada.

Page 18: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 18 of 47

Author(s)

Description of Tool Description of Grading Notes

ElBadrawy & Korayem (2007)

Clinical Competence Evaluation System which allows students to perform a certain number of clinical procedures from a group of similar procedures within specific defined categories.

Grading uses 5 point scale. 1 = unacceptable; 2 = correctable; 3 = acceptable; 4 = good; 5 = superior.

Grading tool used in the assessment of dental students in Canada.

Fisher & Parolin (2000)

Competency assessment tool with 44 items based on the Australian Nursing Council‟s Competency statements. Three clinical domains used: 1. Medication administration 2. Time management 3. Patient comfort and safety.

Six point scale used. A score of 5 = independent practice whilst a score of 1 = dependent practice.

Grading tool used in the assessment of novice registered nurses in Australia. Authors claim the tool is valid but lacks inter-rater reliability.

Page 19: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 19 of 47

Author(s)

Description of Tool Description of Grading Notes

Fitzgerald et al. (2007)

Clinical Internship Evaluation Tool with two major sections: 1. Professional behaviours (with 18

items) 2. Patient management skills (with 24

items).

Professional behaviours section uses a 5-point rating scale used with 0 = never displays behaviour to 4 always displays behaviour. Monitoring of student behaviour is required for a score of 3. Any score below 3, requires remedial action and if not improved can lead to failure. Patient management skills section also uses a 5-pont scale from well below to well above competence. Well-below equates to a student requiring a great deal of guidance, Below as a student requiring some supervision or increased time to complete task. At that level is defined as a student who is competent, above as a student performing above the level of a competent clinician and well above is reserved for master clinician or clinical specialist. A score of 3 denotes that students are at the „correct level‟. A global rating of the student‟s clinical competence is also made on a scale of 0 to 10 which is accompanied by a written justification for the grade awarded.

Authors claim that it is a valid and time-efficient tool. Grading tool used in the assessment of physical therapy students in USA.

Page 20: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 20 of 47

Author(s)

Description of Tool Description of Grading Notes

Hill et al. (2006)

Competency-based assessment using a detailed grading rubric. Tool contains 19 graded competencies grouped into four categories: 1. Communication / Education 2. Pharmacy Care Plan 3. Professionalism / Initiative 4. Practice Specific Competencies.

5-point scale used with 1 = poorest anticipated performance; 2 = less than expected performance; 3 = average performance; 4 = better than expected performance; 5 = best anticipated performance. Detailed expectations were written for each performance level for all 19 graded competencies.

Authors report moderate success in reducing grade inflation. Grading tool used in the assessment of pharmacy students in USA.

Holaday & Buckley (2008)

Innovative tool-kit to assess, evaluate and measure student performance and growth across clinical settings and at all levels in nursing education programme. Contains 11 consensus-based clinical outcome objectives and competencies.

5-point rating scale used adapted from Bondy‟s (1983) work. Point 1 of the scale relates to novice –assisted; 2 = assisted; 3 = assisted – supervised; 4 = supervised – self-directed and 5 as self-directed. There is a qualitative description of each competency to provide direction for assessors. The level in which each competency is written indicates the expected performance for grading purposes. From the ratings a grade-point conversion scale is used for a global score.

Tool-kit used by 3 major nursing schools in USA. However remains „work in progress‟.

Page 21: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 21 of 47

Author(s)

Description of Tool Description of Grading Notes

Lasater (2007) Lasater Clinical Judgement Rubric based on the four phases of Tanner‟s (2006) Clinical Judgement Model – noticing, interpreting, responding and reflecting. Rubric has 11 total dimensions of the four phases complete with descriptors at each of the four levels. Author claims that the rubric uses language easily understood by assessors and students.

Rubric uses 4 levels against each of the 11 dimensions: Exemplary; Accomplished; Developing and Beginning.

Rubric only used to assess nursing student‟s clinical judgement in one institution in USA.

Norcini (2007) Mini-CEX (mini-clinical evaluation exercise) to assess trainees interacting with patients.

Direct Observation of Procedural Skills (DOPS) is a variation of the Mini-CEX.

Case-based discussions (CbD) used to assess the application of knowledge, decision making and ethical issues.

Mini-PAT (mini-peer assessment technique) with peers as assessors making judgements about both quality and fitness of performance.

Uses 6-point scale. Descriptors require the assessor to judge both the quality of the trainee‟s performance and whether it meets expectations for completion of the year of training.

Grading tool used in the assessment of Foundation Programme medical students in UK.

Page 22: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 22 of 47

Author(s)

Description of Tool Description of Grading Notes

Pender & de Looy (2004)

Assessment tool using a Visual Analogue Scale. Four key core skills assessed: 1. Written skills 2. Interviewing skills 3. Skills associated with dietary

assessment technique 4. Oral/presentation skills Each of the 4 key skills were subdivided into constituent skill performance components.

Assessment was measured using a visual analogue scale consisting of a 100mm horizontal line with a brief description at the 0 and 100mm anchor points. The 50mm point = adequate skill performance and 100mm = optimal skills acquisition and performance.

Authors claim tool to be valid and reliable to assess student proficiency. Grading tool used in the assessment of dietetic students in UK.

Prescott-Clements et al. (2008)

Longitudinal evaluation of performance (LEP) which is used both formatively and summatively.

Uses a 9-pont scale with a fixed reference point for judgements that is the standard expected on completion of training. Ratings 1-3 = need improvement; 4-6 = satisfactory; 7-9 = superior performance.

Grading tool used in the assessment of post-graduate dental students in Scotland. Authors claim that the fixed reference point in the LEP successfully illustrates student progress over time.

Page 23: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 23 of 47

Author(s)

Description of Tool Description of Grading Notes

Schmahmann et al. (2008)

Beside Examination Exercise (BEE) with 4 core competencies: skills; clinical competence; knowledge and professionalism.

Uses a 40-point scoring system: 36-40 = high honours; 30-35 = honours; 24-29 = satisfactory; 22-23 = borderline/conditional; 21 or less = unsatisfactory.

Authors claim BEE reduced grade inflation.

Grading tool used in the assessment of medical students working in neurology in one hospital in USA.

Van Leeuwen et al. (2009)

Spiritual Competence Scale comprising of 6 domains and associated competencies: Assessment & implementation of spiritual care; Professionalism and improving the quality of spiritual care; Personal support and patient counselling; Referral to professionals; Attitude towards patients‟ spirituality; communication.

Uses a 5-point Likert scale for self-assessment purposes. Focus of the scale assessment is to indicate areas for further training as well as assessing whether nurses have developed competencies.

Grading tool used in the assessment of student nurses‟ competencies in providing spiritual care in the Netherlands.

Walsh et al. (2008)

Revised Preceptor Clinical Evaluation Tool based on a rubric based on 3 key components: clearly defined performance elements/criteria, a rating scale and descriptions of what a performance looks like at each level of proficiency.

4 levels of performance used with 4 representing the highest level of performance.

Authors piloting a less personal delivery of feedback on summative grade to students as preceptors were reluctant to use the lower half of the grading scale because they had to deliver the feedback to students themselves. The pilot will involve preceptors sending the completed assessment to Faculty and a member of Faculty will give the students their feedback and grade.

Grading tool used in the assessment of nursing students in one area in USA.

Page 24: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 24 of 47

4.3 Grade inflation

Grade inflation is defined as when there is a greater percentage of excellent

of higher scores than is a true reflection of the students‟ actual performance

warrants (Cacamese et al. 2007; Isaacson & Stacy 2009). The danger of

grade inflation is that it allows students to erroneously belief that they are

more competence than they perhaps are and have weaknesses that need to

be addressed (Cacamese et al. 2007). Seldomridge & Walsh (2006) make the

observation that grade inflation is not confined to nursing but occurs across all

healthcare professions.

There are a number of documented reasons as to why grade inflation occurs.

Reasons can be mapped to those revolving around students; assessors, the

student-assessor relationship and the grading tool itself.

4.31 Reasons for grade inflation: students

Students, according to North American authors, generate pressure on

assessors to give them good grades regardless of the quality of their

performance (Walsh & Seldomridge 2005; Weaver et al. 2007).

4.3.2 Reasons for grade inflation: assessors

A number of possible reasons for grade inflation are attributed to assessors.

Inexperienced assessors often have more difficulty in giving negative

feedback (Cacamese et al. 2007; Walsh & Seldomridge 2005) and it is

suggested that they find it easier to avoid conflict by giving them a good grade

and relying on another assessor in the student‟s subsequent placement to

properly assess the student since they don‟t have the confidence to do so

themselves (Fordham 2005; Yorke 2005; Weaver et al. 2007). Thus, less

experienced assessors are more likely to give students‟ a „second-chance‟.

That said it is not always inexperienced assessors who give students a

„second-chance‟. For over thirty years now, the terms „Hawk‟ and „Dove‟ have

been used to describe the phenomenon of Hawks as assessors who have

high expectations and subsequently fail many students and Doves who are

more likely to pass students as they err on leniency as opposed to hawks who

are more stringent (Alexander 1996, McManus et al. 2006; Seldomridge &

Walsh 2006; Panzarella & Manyrn 2007). The consequences of this can be

that one student can receive a higher grade than one of their peers either

because of better performance or luck in being assessed by a lenient marker

(Iramaneerat et al. 2007).

Page 25: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 25 of 47

More experienced assessors however are not immune to grade inflation.

Again North American authors state that many assessors who are non-

tenured are reluctant to give low grades because they rely on good

evaluations from their students for their continued employment. Assessors

who are tenured are often reluctant to give low grades because they do not

want to waste time in dealing with student appeals (Chambers 1999; Walsh &

Seldomridge 2005; Gill et al. 2006; Isaacson & Stacy 2009).

4.3.3 Reasons for grade inflation: student – assessor relationship

The nature of the student-assessor relationship is seen as a factor in causing

grade inflation. In written assessments there is more distance between

student and assessor and it is possible to use anonymous marking so that it is

less subject to bias. However, in grading practice a closer relationship

develops (Cowan et al. 2005) and the assessor can be unduly influenced by

other factors leading to the „Halo effect‟ (Fisher & Parolin 2000; Iramaneerat

et al. 2007; Fletcher 2008). It is difficult for assessors to give poor grades to

students who model themselves on them or to give high grades to students

they perceive as being difficult to manage (Clouder & Toms 2005; McGrath et

al. 2006). Clouder & Toms (2005) highlight that assessors have their own

picture of how they expect students to be and if they fall short of those then

grades are likely to be lower, than those students who mirror the assessors‟

expectations. Brown‟s (2000) found in his study that 76% (n=115) of mentors

made reference to personal characteristics when assessing their performance.

According to Calman et al. (2002) a student‟s practice assessment is often

dependent on the assessor‟s personality and their knowledge of the student.

Smith (2007) asserts that assessors are more likely to over-grade their

student‟s work than under-grade it.

4.3.4 Reasons for grade inflation: tool design

Tool design can also contribute to grade inflation. If a tool uses the letters A to

D for example with D being a minimum pass, there is a tendency to award

weak students a C grade and average students a B grade (Isaacson & Stacy

2009). Iramaneerat et al. (2007) suggest that some assessors are more likely

to cluster their grade around a particular portion of the scale – either lenient or

severe ends or the mid-point rather than using the whole scale available to

them. Isaacson & Stacy (2009: 136) warn that grading tools that use „equal

weighting of objectives can lead to grade inflation because students can

succeed overall while missing the bigger picture and important components of

the course‟.

The reasons for grade inflation are summarised in Table 4 below.

Page 26: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 26 of 47

Reason Authors

Forming a bond with students Brown 2000; Cacamese et al.

(2007); Clouder & Toms (2005);

Cowan et al. 2005; McGrath et

al. 2006

Having difficulty in giving negative

feedback

Cacamese et al. (2007); Walsh

& Seldomridge (2005)

Pressure of high student expectations Cacamese et al. (2007); Walsh

& Seldomridge (2005); Weaver

et al. 2007)

Being unprepared to challenge

student‟s self-assessment

Fordham (2005); Yorke (2005);

Weaver et al. 2007)

Having low confidence / lack of

experience in failing students

Fordham (2005); Yorke (2005);

Weaver et al. 2007)

Failing a student reflects badly on the

assessor and can affect tenure

Chambers (1999); Walsh &

Seldomridge (2005); Gill et al.

(2006); Isaacson & Stacy (2009)

„Halo effect‟ – where a student is highly

rated in one area, the grade may be

inflated to match

Fisher & Parolin (2000); Clouder

& Toms (2005); McGrath et al.

(2006); Fletcher (2008)

Table 4: Reasons for grade inflation

4.3.5 Suggested ways of controlling grade inflation

A few authors report ways of controlling grade inflation. Chambers (1999) and

Battistone et al. (2001) advocate the need to provide detailed evidence of why

a particular grade has been awarded as this had the effect of reducing grade

inflation. Weaver et al. (2007) found that using explicit criteria for each grade

helped to reduce grade inflation, although it did continue to persist. Some

authors postulate that grade inflation is due to assessors being unable to

discriminate effectively between grades (Hill et al. 2006; Iramaneerat et al.

2007; Hemmer et al. 2008;) so including this aspect in the training of

assessors may help to ameliorate grade inflation. Norcini (2007) suggests the

use of assessors who have not been the student‟s mentor reduces the

amount of prior information available and reduce the assessor‟s personal

stake in the trainee.

Page 27: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 27 of 47

Schmahmann et al. (2008) suggests that no single test is accurate enough to

predict performance in medical interns in neurology, but using a

complimentary set of grading tools to obtain a composite score could limit

grade inflation.

4.4 Use of Rubrics in grading tools

According to Truemper (2004) the origins of the word rubric is derived from

the Latin „rubrica terra‟ meaning the early practice of using red soil to signal

something of importance. Rubrics can be defined as “an assessment tool that

uses clearly defined criteria and proficiency levels to gauge student

achievement of those criteria. The criteria provide descriptions of each level of

performance in terms of what students are able to do” (Montgomery 2000:

325).

There are two types of grading rubrics: analytical and holistic (Truemper

2004). “An analytical rubric allows for the separate evaluation of each

component of the task, while a holistic one views all elements in a combined

manner” (Truemper 2004: 562).

Regardless of the type of rubric, they are made up of three key components:

1. Clearly defined performance criteria / elements

2. Detailed descriptions of what a performance looks like at each level of

proficiency

3. Rating scale (commonly 3 or 4 point) (Moskal & Leydens 2000; Walsh et al.

2008).

The advantages of using a grading rubric are cited as:

A way to reduce subjectivity, improve objectivity and consistency in

grading (Truemper 2004; Walsh et al.2008);

A way to reduce the time burden on assessors (Andrade 2000;

Isaacson & Stacy 2009);

Helping students understand the reasons why they receive a particular

grading (Montgomery 2000; Truemper 2004; Walsh et al.2008);

Well constructed, they can improve intra- rater and inter-rater reliability

(Moskal & Leydens 2000);

When used repeatedly, they can be used formatively as the student

knows exactly what is expected at the highest level (Andrade 2000;

Truemper 2004; Lasater 2007; Walsh et al.2008); and they can note

patterns and /or improvements in their work (Isaacson & Stacy 2009);

Since they are clear and transparent to both assessors and students

they help to improve communication (Truemper 2004; Walsh et al.2008

Isaacson & Stacy 2009);

Page 28: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 28 of 47

Provide “a more level playing field for increasingly diverse groups of

students” (Lasater 2007: 497) supported by Isaacson & Stacy (2009).

“Carefully designed analytic, holistic, task specific and general scoring rubrics

have the potential to produce valid and reliable results” (Moskal & Leydens

2000: 9). Isaacson & Stacy (2009) state that rubrics serve as a blueprint for

grading.

4.5 Challenges related to the grading of nursing and midwifery practice

4.5.1 Challenges: Time

A lack of time for mentors to undertake assessment of students is well

documented in the literature (Knight & Page 2007; Gopee 2008; McCarthy &

Murphy 2008; Webb & Shakespeare 2008). Ben-David et al. (2004) called for

protected time for clinical assessors. The lack of time has several

documented implications. There is often a last minute rush at the end of the

placement (and indeed sometimes left until after the placement has actually

finished) to complete „the booklet‟ (Brown 2000). The feeling of being under

time constraints can lead viewing the assessment of students as a burden

rather than as an important and integral part of the student‟s learning

experience (Buckingham 2000; Norman et al. 2002). Due to time constraints

and the pressure to complete the assessment booklet, there may be a

tendency to make a hurried and favourable assessment despite having

concerns about the student‟s practice (Cassidy 2009). Time pressures are

also identified as fostering a task orientated approach and failing to „test‟ the

student‟s underpinning knowledge, judgement, attitudes or attributes (Scholes

et al. 2004; McGrath et al. 2006).

4.5.2 Challenges: Assessors

It is well documented that clinicians often lack the underpinning pedagogical

knowledge and understanding of the processes involved in assessing

students in the practice setting (Redfern et al. 2002; Seldomridge & Walsh

2006; Price 2007; Luhanga et al. 2008; McCathy & Murphy 2008). Practice

assessors can also have difficulty in differentiating between levels (Brown

2000; Norman et al. 2002; McLean et al. 2005) and determining the standards

required (Brown 2000; Neary 2000a). The implications of this lack of

knowledge and experience has numerous effects on the individual attempting

to assess students in practice. As far back as 1996, Oldmeadow noted that

many clinicians felt anxious about their role in assessing student and worrying

whether they had over-rated or under-rated students.

Page 29: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 29 of 47

Price (2007) reinforces this emphasising that mentors have concerns whether

they have interpreted the assessment documentation properly. Luhanga et al.

(2008) highlight that less-experienced clinicians often lack the conviction of

their assessment decisions. This in turn can lead assessors to place their trust

in their instinct and clinical experience as the basis for their decisions

regarding assessment of students (Isaacson & Stacy 2009). It also perhaps

reflects why such assessors avoid conflict by awarding higher grades than

those warranted (Isaacson & Stacy 2009) and pre-disposed to giving weaker

students a „second-chance‟ because as assessors they lack the knowledge,

skills and confidence to do anything otherwise (Luhanga et al. 2008; Duffy

2004). However, findings from a study by Hale et al. (2006) found that

assessors (n=26) were confident in their assessor role. The difference in

findings may be reflected in that Hale et al.‟s study involved assessors on a

post-registration rather than an undergraduate pre-registration course.

Knight and Page (2007) make the observation that clinicians are happier to

assess technical skills but are less confident to assess softer skills such

communication and inter-personal skills. Buckingham (2000) makes the

observation that assessment of students in practice is often delegated to

junior clinicians who, it could be argued, may have even less knowledge and

experience of assessing students. However, Cassidy (2009) warns of the

danger of assuming that being an experienced practitioner automatically

means that they will therefore be a proficient assessor.

5. Issues related to validity and reliability in grading of professional

practice

It is well documented in the literature, across health professions, that issues

with validity and reliability of practice assessments have posed difficulty over

an extended period of time (Chambers 1998; Neary 2000a; Baulcomb &

Watson 2003; Downing & Haldane 2004; Hale et al. 2006; Prescott-Clements

et al. 2008). Few assessment tools have been tested for validity and reliability

(Andre 2000; Redfern et al 2002) and as yet there is no „gold standard‟

practice assessment tool (Salvatori 1996; Wass et al. 2001; Watson et al.

2002; Walsh et al. 2008).

Page 30: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 30 of 47

5.1 Validity

A valid assessment is defined as one that measures what it claims to

measure and what is important to measure (Chambers 1998; Freeman &

Lewis 1998; Maxted et al. 2004; Clouder & Toms 2005; Coote et al. 2007).

The most important aspect of assessing practice is validity (Stuart 2007),

whether graded or not (Williams & Bateman 2003) because unless all the

appropriate criteria are assessed the entire process is pointless (Cross et al.

2001). According to McKinley et al. (2001: 713) “validity concerns both the

instrument and the assessment process and the challenge with which the

candidate is tested”.

There are three important components to validity: content validity; construct

validity and predictive validity (Hand 2006; Knight & Page 2007; London 2008).

Content validity refers to whether the assessment tests the appropriate and

necessary content (Cross et al. 2001; Clouder & Toms 2005; Hand 2006;

Coote et al. 2007; Stuart 2007).

Construct validity refers to being able to justify that the tool being used

actually reveals and measures specific behaviours or traits (Cross et al. 2001;

Stuart 2007). “Constructs are the qualities, abilities and traits we look for to

explain aspects of human behaviour that cannot be observed directly. There

are a number of constructs that should be tested in healthcare workers such

as professionalism, attitudes and values. Just because a student has

theoretical knowledge about the values does not mean that he or she will

practise them. Continuous assessment increases construct validity as it

allows the student ample opportunity to demonstrate the required constructs

in a variety of care-giving situations and the assessor ample opportunity to

observe those behaviours” (Hand 2006: 51). An individual‟s reasoning

process is an example of a construct which can be displayed through

explanations and results of actions (Moskal & Leydens 2000).

Predictive validity relates to how well the assessment being used can predict

individual‟s future performance (Hand 2006; Stuart 2007). Continuous

assessment is argued to be one way of developing predictive validity (Hand

2006; Stuart 2007). “If there is consistency of performance, there is a higher

chance of validity in our assessment.... the best predictor measure will be to

incorporate and assess those future behaviours which are of interest” (Stuart

2007: 113).

Page 31: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 31 of 47

5.2 Reliability

Reliable assessments can be defined as “... ones where the same marker

reaches the same conclusion on different occasions and different markers

reach the same conclusion when presented with similar evidence (Freeman &

Lewis 1998: 317). In the practice setting this principally relates to inter and

intra-assessor reliability (Cross et al. 2001; ElBadrawy & Korayem 2007).

Inter-rater reliability is seen as a problematic area to resolve (Chambers 1998;

Allen et al. 2008; Holaday & Buckley 2008). Cross et al. (2001) assert that the

differences in assessors‟ decisions are more likely to be due to a lack of

consistency in interpretation of criteria as opposed to their lack of ability in

making good judgements. In order to improve reliability in assessing practice,

it is important to ensure that all assessors are using the same criteria,

however it remains problematic in ensuring consistent interpretation of these

across assessors (Hand 2006; Smith 2007).

Knight & Page (2007) add that to make assessments more reliable, it is

important to remove uncertainty about the marking scheme perhaps by the

use of rubrics, clear criteria and assessor training. McLean et al. (2005) offer

additional suggestions including triangulation of student self-assessment,

assessor verification and written records of diagnostic interviews. In their

study Feeley et al. (2003) found that they required 20 assessors per student

in order to yield reliable results. Having this ratio of assessors to student is

unpractical and Feeley et al. (2003) suggest that efforts should be focused

instead on ensuring better assessor training. This suggestion is reinforced by

Glavin & Moran (2002); McGrath (2006) and Lewis et al. (2008).

Intra-rater reliability can be affected by factors external to the rater. For

example fatigue, mood and other personal inconsistencies that are internal to

the rater/assessor rather than reflective of anything to do with the student‟s

performance (Moskal & Leydens 2000).

5.3 Challenges: Assessment Tool

A plethora of authors identify the use of academic or educational jargon as

one of the key factors in causing practitioners difficulty when assessing

students in the practice setting (Redfern et al. 2002; Knight & Page 2007;

Skingley et al. 2007; Smith 2007; Isaacson & Stacy 2009). Since tools are

often considered complex and unwieldy, there is a temptation for busy

assessors to view their completion as a necessary formality rather than an

effective means of providing students with valuable feedback on their

progress (Norman et al. 2002).

Page 32: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 32 of 47

The use of educational jargon means that often elements of the tool are

interpreted differently by the assessor, student and link lecturer (Scholes et al.

2004; McCarthy & Murphy 2008). With attempts to clarify language,

Seldomridge & Walsh (2006) point out that tools can be worded in such broad

terms that it is difficult to ascertain what exactly is being measured. Skingley

et al. (2007) assert that due to the difficulty of understanding the language

used, assessors find it difficult to link the student‟s performance to the

learning outcomes. Scholes et al. (2004) and Finnerty et al. (2006) argue that

this often results in theory and practice not always being well linked or indeed

assessed at al.

There are a few suggestions in the literature as to how to address these

challenges. Clouder & Toms (2005) suggest that when tools are developed

they are devised in conjunction with those who will be expected to use them.

This is reinforced by McCarthy & Murphy (2008). Knight & Page (2007)

suggest that greater attention is paid to clearly communicating the

assessment tools / methods to be used and Dudeck et al. (2008) suggest that

the expectations of what should be documented is made readily apparent.

Burchell et al. (1999) found that assessors really welcomed such detailed

explanations.

“The apparent lack of clarity about the achievement categories

arises because the assessor is meant to rely on their own

expert judgement rather than on specific cues in the tool. This

remains a significant concern for educators” (McGrath et al.

2006: 54).

5.4 Improving validity and reliability in assessing practice

It is important to note that by the very nature of the context and processes

involved in the assessment of practice, it is impossible to completely remove

subjectivity in making judgements of students‟ performance (McGrath 2006;

Isaacson & Stacy 2009).

There is a great deal of support in the literature for the adoption of a

triangulated approach to assessing practice as a means of increasing validity

and reliability (Chambers 1998; Brown 2000; Buckingham 2000; Murray et al.

2000; Redfern et al. 2002; Silber et al. 2004; Clouder & Toms 2005; Walsh &

Seldomridge 2005; Hand 2006; McGrath et al 2006; Gray & Bradshaw 2007;

Pirie & Gray 2007; Rushforth 2007; Skingley et al. 2007; Abbey 2008;

McCarthy & Murphy 2008; Williams & Rowlands 2009). Calman et al. (2002)

proposed that the introduction of a national practice assessment instrument

would address some of the difficulties associated with problems of inter-rater

reliability.

Page 33: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 33 of 47

This has recently been reinforced by Dalton et al. (2009a) who have been

successful in introducing a standard physiotherapy assessment instrument

across Australia.

“The advantages of a standardised national form are that

clinical educators/supervisors who have students from more

than one physiotherapy program will not have to deal with

multiple assessment forms. Other advantages of a standardised

form will be a means to analyse and document evidence of test

validity and reliability, and the ability to meaningfully compare

scores between students, raters, sites and programs for

benchmarking purposes” (Dalton et al 2009a: 95).

6. Implications for mentor preparation and support in respect to grading of

nursing and midwifery practice

The NMC (2008) Mentor Standards are mandatory for all those who support

the learning and assessment of pre-registration nursing and midwifery

students in practice. As part of implementing these standards, there must be

an approved mentor preparation programme in place. Hyatt et al. (2008)

stress that since 50% of the pre-registration nursing and midwifery students‟

overall programme is situated within practice, the mentors‟ role in conducting

accurate assessments is paramount. Yorke (2005: 39) reminds us that “the

mentor/assessor is (in theory) in a much better position to comment on the

mentee‟s actual performance than an occasional visiting assessor from an

educational institution”.

Webb & Shakespeare‟s (2008: 570) study of how mentors often make their

assessment judgements concluded that mentors make these “...on a relatively

subjective basis, negative outcomes often being described as arising from

„personality clashes‟”.

The imperative preparation of assessors to undertake their role is well

documented in the literature. Interestingly, most of this literature relates to

nursing and midwifery. Silber et al. (2004) highlight the dearth of literature for

training in respect of conducting medical student assessment and emphasise

the need for this despite the fact that it is time consuming and expensive.

The literature mainly focuses on three areas: consistency and accountability;

reliability; and methods and content of training programmes.

Page 34: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 34 of 47

6.1 Consistency and accountability

High quality training and support for assessors is endorsed by many authors

(Phillips & Bharj 1996; Colletti 2000; Watson 2000; Redfern et al. 2002;

Watson et al. 2002; Hanley & Higgins 2005; McLean et al. 2005; Yorke 2005;

McGrath et al. 2006; Seldomridge & Walsh 2006; Pellat 2006; Davies et al.

2008; Gopee 2008; Holaday & Buckley 2008; Hyatt et al. 2008; McCarthy &

Murphy 2008; Dalton et al. 2009b). Calman et al. (2002) make the point that

training courses must not only be improved but be tailored specifically so that

assessors can consistently use the complexity of tools in operation. It is vital

that assessors understand their accountability in the assessment process and

fully engage in the formality required (Calman et al. 2002). Reflecting the

crucial nature of assessor training, a number of authors have suggested that

attendance (initial training course and on-going updates) should be mandatory

(Hale et al. 2006; Seldomridge & Walsh 2006; Gopee 2008; Webb &

Shakespeare 2008). Other authors call for mandatory monitoring of assessors

to ensure that their knowledge and skills remain effective and up-to-date

(Phillips & Bharj 1996; Fraser 2000; Gopee 2008).

6.2 Reliability

The effect of well-constructed and delivered training programmes has been

shown to improve the consistency in which assessment tools are used (Hyatt

et al. 2008) as well as improving (but not eliminating) inter-rater reliability

(Borman 1975; Burchell et al. 1999; Downing & Haladyna 2004; Silber et al.

2004; Hanley & Higgins 2005; Gill et al. 2006; Iramaneerat & Yudkowsky

2007; Gopee 2008; Fletcher 2008; Holaday & Buckley 2008; Lewis et al. 2008)

and controlling grade inflation (Hill et al. 2006; Iramaneerat et al. 2007;

Hemmer et al. 2008).

6.3 Methods and content of training programmes

There is considerable literature advising on the content and methods of

delivery of assessor training and update sessions. The suggestions apply to

all types of practice assessment including grading. A synopsis of these can

be found in Tables 5 and 6.

Page 35: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 35 of 47

Suggested content Authors

How to use assessment forms / tools effectively

and consistently

Williams & Bateman

(2003)

Dudeck et al. (2005)

Seldomridge & Walsh

(2006; 2008)

How to delivery effective feedback Pirie & Gray (2007)

Price (2007)

Davies et al. (2008)

How to write the evidence to support assessment

decision

Knight & Page (2007)

Price (2007)

Hyatt et al. (2008)

How to accurately assess learning (both

formatively and summatively) encompassing both

forms and processes

Price (2007)

Dudeck et al. (2005)

How to assign failing and passing grades Duffy (2004)

Skingley et al. (2007)

How to deal with borderline students Fraser (2000)

Understanding and interpreting educational

terminology

Price (2007)

Table 5 Suggestions for content of assessor training and update

sessions

Page 36: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 36 of 47

Suggested method of delivery Authors

Assessment messages need to be repeated and

conveyed by multiple means

Hill et al. (2006)

Discussions through the use of case studies in

workshops

Gopee (2008)

Visual aids and how to guides about assessment

and grading criteria

Hyatt et al. (2008)

Use of examples of performance criteria Redfern et al. (2002)

Dalton et al. (2009a)

Face to face workshops Walsh et al. (2008)

Think aloud decision making through the use of

role play

Seldomridge & Walsh

(2006)

Training packages / resource guide/ websites Fraser (2000)

Williams & Bateman

(2003)

Seldomridge & Walsh

(2006)

Dalton et al. (2009a, b)

Table 6: Suggestions for method of delivery for training and update

sessions

Dalton et al.‟s (2009a) comprehensive study into the introduction of a national

grading tool for physiotherapy in Australia revealed that the preferred mode of

delivery of training was face to face but acceptable alternatives were online

training; self-directed training manual including a manual and CD/DVD) or

video-conferencing. Dalton et al. (2009a) received positive feedback from

educators and clinical assessors regarding the training and resource manual.

In particular, the frequently asked questions section, information on avoiding

rater bias and training in decision making on scoring items were highlighted as

particularly useful.

Page 37: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 37 of 47

7. Conclusions and Recommendations

There are 5 criteria of good assessment: validity; reliability; practicality and

cost-effectiveness; fairness and usefulness. There is an increasing literature

base across health professions relating to the grading of practice. There are

a number of arguments for its use, but at the same time a number of

arguments against its use. According to Sadler (2005: 191) there are “four

fundamental challenges when considering grading students: coming to grips

with the concept of a standard; working out how to set standards; devising

ways to communicate standards to students and colleagues; and becoming

proficient in the use of standards”. However, on reviewing the literature, these

are hugely challenging.

According to Biggs (1992), “A grading procedure should thus reflect where

students stand in relation to the orderly development of competence rather

than in relation to each other; inform both teacher and student not only where

the student currently is, but what needs doing to improve that position; and be

able to be combined with other grades in order to meet administrative

requirements for awarding levels of pass and the like” Biggs (1992: 2)

The review of the literature presents a number of grading tools or systems,

and we conclude that evaluation of these tools is under-developed in terms of

their effectiveness, usefulness, reliability and validity. Grade inflation is a well-

documented problem within grading of practice, and there are some

suggested methods (although not fully evaluated) from the literature to control

this. The most promising of those appears to be the use of rubrics. Carefully

constructed rubrics can ameliorate grade inflation. Rubrics are made up of

three key components: clearly defined performance criteria or elements;

detailed descriptions of what a performance looks like at each level (or grade)

of proficiency; and a rating scale which most commonly uses a three or four

points. Rubrics can be used both formatively and summatively.

Literature suggests that the use of a multi-method approach to assessment,

and the use of a number of assessment methods which could be graded:

direct observation; interview about practice; performance appraisal; peer-

assessment; simulation; logbook, work diary; portfolio; written test or

assignment.

If nursing and midwifery were to use grading within practice, there are a

number of training requirements suggested by the literature for mentors: how

to use the tools consistently and effectively; how to deliver effective feedback;

how to write feedback; how to write evidence to support assessment; how to

accurately assess learning; how to assign grades; how to deal with borderline

students; and, understanding and interpreting educational terminology. The

time and methods of training and updating mentors needs to be considered.

Page 38: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 38 of 47

The findings from this literature review comes from a wide source of

professional literature, and as demonstrated in Table 2, the number of

quantitative studies is relatively large, and on closer inspection (Appendix 4;

Volume 2), these studies often have limitations of being set in one

geographical area, in one profession, and calculated on relatively small

sample sizes. Most quantitative studies were survey design, and /or

presented using descriptive statistics only. Therefore the generalisability of

findings from these quantitative studies should be necessarily cautious.

Only 6 studies demonstrated the use of a qualitative theoretical basis for the

study. A number of limitations were noted on the literature reviews presented

within this review, and this demonstrates that not all were of a systematic

nature and therefore could be prone to bias.

There appear to be a number of included literature that were descriptive

accounts, of for example a grading tool, or an opinion on grading, or on a

developed tool. It should be born in mind that opinion can be subject to writer

bias. In other words, the level of evidence on grading of clinical practice

across the literature tends to sway towards the lower end of the evidence

band i.e. the usefulness, reliability, validity and effectiveness of grading of

practice has still to be proven.

In summary should grading of practice be adopted, we would recommend that

consideration is given to the following:

The development, testing and use of rubrics.

The use of a multi-method approach to assessment.

Comprehensive training and updating sessions for assessors.

Ongoing evaluation and monitoring of the grading process used.

Page 39: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 39 of 47

8. References

Abbey, H. (2008). "Assessing clinical competence in osteopathic education: analysis of outcomes of different assessment strategies at the British School of Osteopathy." International Journal of Osteopathic Medicine 11(4): 125-131.

Alexander, H. A. (1996). "Physiotherapy student clinical education: the influence of subjective judgements on observational assessment." Assessment & Evaluation in Higher Education 21(4): 357-367.

Allen, P., Lauchner, K., Bridges, R.A., Francis-Johnson, P., McBride, S.G., Olivarez, A. (2008). "Evaluating continuing competency: a challenge for nursing." Journal of Continuing Education in Nursing 39(2): 81-85.

Andrade, H.G. (2000) “Using Rubrics to promote Thinking and Learning.” Educational Leadership 57(5): 13-19

Andre, K. (2000). "Grading student clinical practice performance: the Australian perspective." Nurse Education Today 20(8): 672-679.

Battistone, M. J. P., B., Milnes, C., Battistone, M.L., Sande, M.A., Hemmer, P.A., Shomaker, T.S. (2001). "Global Descriptive Evaluations Are More Responsive than Global Numeric Ratings in Detecting Students' Progress during the Inpatient Portion of an Internal Medicine Clerkship." Academic Medicine 76(10 Suppl): S105-107.

Baulcomb, S. and R. Watson (2003). "The internal consistency and intra-rater reliability of an assessment practice grid for community nurses." Clinical Effectiveness in Nursing 7(3/4): 168-170.

Ben-David, M. F., Snadden, D, Hesketh, A. (2004). "Linking appraisal of PRHO professional competence of junior doctors to their education." Medical Teacher 26(1): 63-70.

Biggs, J. (1992). "A qualitative approach to grading students." HERDSA News 14(3): 3-6.

Bloodgood, R. A., Short, J.G., Jackson, J.M., Martindale, J.R. (2009). "A Change to Pass/Fail Grading in the First Two Years at One Medical School Results in Improved Psychological Well-being." Academic Medicine 84(5): 655-662.

Borman, W. C. (1975). "Effects of Instructions to Avoid Halo Error on Reliability and Validity of Performance Evaluation Ratings." Journal of Applied Psychology 60(5): 556-560.

Brown, N. (2000). "What are the criteria that mentors use to make judgements on the clinical performance of student mental health nurses? An exploratory study of the formal written communication at the end of clinical nursing practice modules." Journal of Psychiatric & Mental Health Nursing 7(5): 407-416.

Buckingham, S. (2000). "Clinical competency: the right assessment tools?" Journal of Child Health Care 4(1): 19-22.

Page 40: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 40 of 47

Burchell, H., Higgs, T., Murray, S. (1999). "Assessment of Competence in Radiography Education." Assessment & Evaluation in Higher Education 24(3): 315-326.

Cacamese, S. M., Elnicki, M., Speer, A.J. (2007). "Grade inflation and the internal medicine subinternship: a national survey of clerkship directors." Teaching & Learning in Medicine 19(4): 343-346.

Calman, L., Watson, R., Norman, I., Redfern, S., Murrells, T. (2002). "Assessing practice of student nurses: methods, preparation of assessors and student views." Journal of Advanced Nursing 38(5): 516-523.

Cassidy, S. (2009). "Interpretation of competence in student assessment." Nursing Standard 23(18): 39-46.

Chambers, D. W. (1999). "Faculty ratings as part of a competency-based evaluation clinic grading system." Evaluation and the Health Professions 22(1): 86-106.

Chambers, M. A. (1998). "Some issues in the assessment of clinical practice: a review of the literature." Journal of Clinical Nursing 7(3): 201-208.

Clouder, L. Toms, J. (2005). "An Evaluation of the Validity of Assessment Strategies Used to Grade Practice Learning in Undergraduate Physiotherapy Students: Final Report to the Health Science and Practice Subject Centre of the Higher Education Academy." from http://www.health.heacademy.ac.uk/projects/miniprojects/clouder.pdf.

Colletti, L. M. (2000). "Difficulty with Negative Feedback: Face-to-Face Evaluation of Junior Medical Student Clinical Performance Results in Grade Inflation." Journal of Surgical Research 90(1): 82-87.

Coote, S., Alpine,L., Cassidy, C., Loughnane, M., McMahon, S., Meldrum, D., O‟Connor, A., O‟Mahoney, M. (2007). "The development and evaluation of a Common Assessment Form for physiotherapy practice education in Ireland." Physiotherapy Ireland 28(2): 6-10.

Cowan, D. T., Norman, I.J., Coopamah, V.P. (2005). "Nurse competency. A project to establish a skills competency matrix for EU nurses." British Journal of Nursing (BJN) 14(11): 613-617.

Cross, V., Hicks, C., Barwell, F. (2001). "Comparing the Importance of Clinical Competence Criteria across Specialties: Impact on undergraduate assessment." Physiotherapy 87(7): 351-367.

Dalton, M., Keating, J., Davidson, M. (2009a). "Development of the Clinical Assessment of Physiotherapy Skills (CAPS): A standardised and valid approach to assessment of clinical competence in physiotherapy ", from http://www.altc.edu.au/resource-development-clinical-assessment-griffith-2009

Page 41: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 41 of 47

Dalton, M., Keating, J., Davidson, M. (2009b). "Development of the Clinical Assessment of Physiotherapy Skills (CAPS): A standardised and valid approach to assessment of clinical competence in physiotherapy : Clinical Educator Resource Manual from http://www.altc.edu.au/resource-development-clinical-assessment-griffith-2009

Darra, S., Hunter, B., McIvor, M., Webber, F., Morse, N. (2003). "Education. Developing a midwifery skills framework." British Journal of Midwifery 11(1): 43-47.

Davies, H., Archer, J., Bateman, A., Dewar, S., Crossley, J., Grant, J., Southgate, L. (2008). "Specialty-specific multi-source feedback: assuring validity, informing training." Medical Education 42(10): 1014-1020.

Downing, S. M., Haladyna, T.M. (2004). "Validity threats: overcoming interference with proposed interpretations of assessment data." Medical Education 38(3): 327-333.

Dudek, N. L., Marks, M. B., Regehr, G. (2005). "Failure to Fail: The Perspectives of Clinical Supervisors." Academic Medicine 80(10 suppl): S84-87.

Dudek, N. L., Marks, M. B., Wood, T.J., Lee, C. (2008). "Assessing the quality of supervisors completed clinical evaluation reports." Medical Education 42(8): 816-822.

Duffy, K. (2004) “A grounded theory investigation of factors which influence the assessment of students‟ competence to practice”. London, Nursing and Midwifery Council

ElBadrawy, H. and M. Korayem (2007). "The flexible requirement system for grading of clinical performance of undergraduate dental students." European Journal Of Dental Education: Official Journal Of The Association For Dental Education In Europe 11(4): 208-215.

Feeley, T. H., Manyon, A.T., Servoss, T.J., Panzarella, K.J. (2003). "Toward validation of an assessment tool designed to measure medical students' integration of scientific knowledge and clinical communication skills." Evaluation & The Health Professions 26(2): 222-233.

Finnerty, G., Graham, L., Magnusson, C., Pope, R. (2006). "Empowering midwife mentors with adequate training and support." British Journal of Midwifery 14(4): 187-190.

Fisher, M. and M. Parolin (2000). "The reliability of measuring nursing clinical performance using a competency based assessment tool: a pilot study." Collegian 7(3): 21-27.

Fitzgerald, L. M., Delitto, A., Irrgang, J.J. (2007). "Validation of the Clinical Internship Evaluation Tool." Physical Therapy 87(7): 844-860.

Fletcher, P. (2008). "Clinical competence examination -- improvement of validity and reliability." International Journal of Osteopathic Medicine 11(4): 137-141.

Page 42: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 42 of 47

Fordham, A. J. (2005). "Using a competency based approach in nursing education." Nursing Standard 19(31): 41-48.

Fraser, D.M. (2000) “Action research to improve the pre-registration midwifery curriculum Part 3: can fitness for practice be guaranteed? The challenges of designing and implementing an effective assessment in practice scheme”. Midwifery 16, 287-294

Freeman, R., Lewis, R. (1998) “Planning and Implementing Assessment”. London, Kogan Page

Gill, F., Leslie, G., Southerland, K. (2006). "Evaluation of a clinical performance assessment tool (CPAT) within a critical care context." Australian Critical Care 19(3): 105-113.

Glavin, R.J., Maran, N.J. (2002)”Development and use of scoring systems for assessment of clinical competence”. British Journal of Anaesthesia 88(3) 329-330

Glover, P., Ingham, E., Gassner, L.A. (1997). "The development of an evaluation tool for grading clinical competence." Contemporary Nurse: A Journal for the Australian Nursing Profession 6(3/4): 110-116.

Gopee, N. (2008). "Assessing student nurses' clinical skills: the ethical competence of mentors." Int J Therapy & Rehabilitation 15(9): 401-407.

Gray, J. and C. Bradshaw (2007). "Assessing general practitioners who may be underperforming: local assessment methods in two English health districts." Quality in Primary Care 15(3): 157-164.

Hale, C., Long, T., Sanderson, L., Carr, K., Tomlinson, P. (2006). "An evaluation of educational preparation for cancer and palliative care nursing for children and adolescents: issues in the assessment of practice arising from this study." Nurse Education Today 26(7): 528-537.

Hand, H. (2006). "Assessment of learning in clinical practice." Nursing Standard 21(4): 48.

Hanley, E. and A. Higgins (2005). "Assessment of clinical practice in intensive care: a review of the literature." Intensive & Critical Care Nursing 21(5): 268-275.

Hemmer, P. A., Papp, K. K., Mechaber, A.J., Durning, S.J. (2008). "Evaluation, grading, and use of the RIME vocabulary on internal medicine clerkships: results of a national survey and comparison to other clinical clerkships." Teaching & Learning in Medicine 20(2): 118-126.

Hill, L. H., Delafuente, J. C., Sicat, B.L., Kirkwood, C.K. (2006). "Development of a competency-based assessment process for advanced pharmacy practice experiences." Am J Pharm Educ 70(1): 1.

Holaday, S. D. and K. M. Buckley (2008). "Chapter 7. A standardized clinical evaluation tool-kit: improving nursing education and practice." Annual Review of Nursing Education 6: 123-149.

Page 43: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 43 of 47

Hyatt, S. A., Brown, L., Lipp, A. (2008). "Supporting mentors as assessors of clinical practice." Nursing Standard 22(25): 35-41.

Iramaneerat, C. and R. Yudkowsky (2007). "Rater errors in a clinical skills assessment of medical students." Evaluation & the Health Professions 30(3): 266-283.

Isaacson, J. J. and A. S. Stacy (2009). "Rubrics for clinical evaluation: objectifying the subjective experience." Nurse Education in Practice 9(2): 134-140.

Johnson, M. (2007). "Is passing just enough? Some issues to consider in grading competence- based assessments." Research Matters(3): 27-30.

Johnson, M. (2008). "Grading in competence-based qualifications - is it desirable and how might it affect validity?" Journal of Further and Higher Education 32(2): 175-184.

Knight, P. P., Anna. (2007). "Assessment of „wicked‟ competences: report to the Practice-based Professional Learning Centre." from http://kn.open.ac.uk/public/getfile.cfm?documentfileid=10623

Lanphear, J. H. (1999). "In support of grading systems." Education for Health 12(1): 79-83.

Lasater, K. (2007). "Clinical judgment development: using simulation to create an assessment rubric." Journal of Nursing Education 46(11): 496-503.

Lewis, L. K., Stiller, K., Hardy, F. (2008). "A clinical assessment tool used for physiotherapy students -- is it reliable?" Physiotherapy Theory & Practice 24(2): 121-134.

London, S. (2008). "The assessment of clinical practice in osteopathic education: is there a need to define a gold standard?" International Journal of Osteopathic Medicine 11(4): 132-136.

Luhanga, F., Yonge, O. J., Myrick, F. (2008). ""Failure to assign failing grades": issues with grading the unsafe student." International Journal of Nursing Education Scholarship 5(1): 1-13.

Maxted, M., Grant, J., Wooding, K., Norcini, J. (2004). "Assessing Healthcare Students in Practice Placements: An Overview of the Literature”. Edinburgh, Association for the Study of Medical Education (ASME)

McCarthy, B. and S. Murphy (2008). "Assessing undergraduate nursing students in clinical practice: do preceptors use assessment strategies?" Nurse Education Today 28(3): 301-313.

McGrath, P., J. Anastasi, et al. (2006). "Collaborative voices: ongoing reflections on nursing competencies." Contemporary Nurse: A Journal for the Australian Nursing Profession 22(1): 46-58.

Page 44: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 44 of 47

McKinley, R.K., Fraser, R.C., Baker, R. (2001) “Model for directly assessing and improving clinical competence and performance in revalidation of clinicians.” British Medical Journal 322(24): 712-715

McLean, C., Monger, E., Lally, I. (2005). "Assessment of practice using the National Health Service Knowledge and Skills Framework." Nursing in Critical Care 10(3): 136-142.

McManus, I. C., Thompson, M., Mollon, J. (2006). "Assessment of examiner leniency and stringency ('hawk-dove effect') in the MRCP (UK) clinical examination (PACES) using multi-facet Rasch modelling." BMC Medical Education 6(42): 1-22.

Montgomery, K. (2000) “Classroom Rubrics: Systematizing What Teachers Do Naturally”. The Clearing House 73(6): 324-328

Moon, J. (2002) “The Module & Programme Development Handbook”. London, Kogan Page

Moskal, B. M., Leydens, J.A. (2000). "Scoring Rubric Development: Validity and Relaibility." Practical Assessment, Research & Evaluation 7(10): 1-11.

Murray, E., Gruppen, L., Catton, P., Hays, R., Woolliscroft, J.O. (2000). "The accountability of clinical education: its definition and assessment." Medical Education 34(10): 871-879.

Neary, M. (2000a). "Responsive assessment of clinical competence: part 1." Nursing Standard 15(9): 34-36.

Neary, M. (2000b). "Responsive assessment of clinical competence: part 2." Nursing Standard 15(10): 35-40

Norcini, J. J. (2007). Workplace-based assessment in clinical training. Edinburgh, Association for the Study of Medical Education.

Norman, I., Watson, R., Murrells, T., Calman, L., Redfern, S. (2002). "The validity and reliability of methods to assess the competence to practise of pre-registration nursing and midwifery students." International Journal of Nursing Studies 39(2): 133-145.

Nursing and Midwifery Council. (2008). "Standards to support learning and assessment in practice: NMC standards for mentors, practice teachers and teachers." 2nd. from http://www.nmc-uk.org/aDisplayDocument.aspx?documentID=4368

Oldmeadow, L. (1996). "Developing clinical competence: a mastery pathway." Australian Journal of Physiotherapy 42(1): 37.

Panzarella, K. J. and A. T. Manyon (2007). "A Model for Integrated Assessment of Clinical Competence." Journal of Allied Health 36(3): 157.

Page 45: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 45 of 47

Pellatt, G. C. (2006). "Nursing mentors. The role of mentors in supporting pre-registration nursing students." British Journal of Nursing (BJN) 15(6): 336-340.

Pender, F.T., de Looy, A.E. (2004) “Monitoring the development of clinical skills during training in a clinical placement”. Journal of Human Nutrition and Dietetics 17(1): 25-34

Phillips, M. and K. Bharj (1996). "Developing a tool for the assessment of clinical learning." British Journal of Midwifery 4(9): 471-475.

Pirie, E. S. and M. A. Gray (2007). "Exploring the assessors' and nurses' experience of formal assessment of clinical competency in the administration of blood components." Nurse Education in Practice 7(4): 215-227.

Prescott-Clements, L., van der Vleuten, C. P. M., Schuwirth, L.W.T., Hurst, Y., Rennie, J.S. (2008). "Evidence for validity within workplace assessment: the Longitudinal Evaluation of Performance (LEP)." Medical Education 42(5): 488-495.

Price, B. (2007). "Practice-based assessment: strategies for mentors." Nursing Standard 21(36): 49.

Ravelli, C., Wolfson, P. (1999). "What is the 'Ideal' Grading System for the Junior Surgery Clerkship?" American Journal of Surgery 177: 140-144.

Redfern, S., Norman, I., Calman, L., Watson, R., Murrells, T. (2002). "Assessing competence to practise in nursing: a review of the literature." Research Papers in Education 17(1): 51-77.

Rohe, D. E., Barrier, P.A., Clark, M.M., Cook, D.A., Vickers, K.S., Decker, P.A. (2006). "The Benefits of Pass-Fail Grading on Stress, Mood, and Group Cohesion in Medical Students." Mayo Clinic Proceedings 81(11): 1443-1448

Rushforth, H. E. (2007). "Objective structured clinical examination (OSCE): review of literature and implications for nursing education." Nurse Education Today 27(5): 481-490.

Sadler, D. R. (2005). "Interpretations of criteria-based assessment and grading in higher education." Assessment and Evaluation in Higher Education 30(3): 175-194.

Salvatori, P. (1996). "Clinical competence: A review of the health care literature with a focus on occupational therapy." Canadian Journal of Occupational Therapy 63(4): 260-271.

Schmahmann, J. D., Neal, M., MacMore, J. (2008). "Evaluation of the assessment and grading of medical students on a neurology clerkship." Neurology 70(9): 706-712.

Page 46: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 46 of 47

Scholes, J. Freeman, M., Gray, M., Wallis, B., Robinson, D., Matthews-Smith, G., Miller, C. (2004) “Evaluation of Nurse Education Partnership” London, Department of Health

Seldomridge, L. A. and C. M. Walsh (2006). "Evaluating student performance in undergraduate preceptorships." Journal of Nursing Education 45(5): 169-176.

Sharp, S. (2006). "The grading of placement in initial teacher education in Scotland." Scottish Educational Review 38(2): 145-157.

Silber, C. G., Nasca, T. J., Paskin, D.L., Eiger, G., Robeson, M., Veloski, J.J. (2004). "Do global rating forms enable program directors to assess the ACGME competencies?" Academic Medicine: Journal Of The Association Of American Medical Colleges 79(6): 549-556.

Skingley, A., Arnott,J., Greaves, J., Nabb, J. (2007). "Supporting practice teachers to identify failing students." British Journal of Community Nursing 12(1): 28-32.

Smith, J. (2007). "Assessing and grading students' clinical practice: midwives' lived experience." Evidence Based Midwifery 5(4): 112-118.

Stuart, C. C. (2007) “Assessment, Supervision and Support in Clinical Practice”. 2nd ed. Edinburgh, Elsevier Churchill Livingstone

Truemper, C. M. (2004). "Using Scoring Rubrics to Facilitate Assessment and Evaluation of Graduate-Level Nursing Students." Journal of Nursing Education 43(12): 562-564Van Leeuwen et al. (2009)

Walsh, C. M. and L. A. Seldomridge (2005). "Clinical grades: upward bound." Journal of Nursing Education 44(4): 162-168.

Walsh, C. M., L. A. Seldomridge, et al. (2008). "Developing a practical evaluation tool for preceptor use." Nurse Educator 33(3): 113-117.

Wass, V., C. Van Der Vleuten, et al. (2001). "Assessment of clinical competence." Lancet 357(9260): 945-949.

Watson, S. (2000) “The support that mentors receive in the clinical setting.” Nurse Education Today 20: 585-592

Watson, R., Stimpson, A., Topping, A., Porock, D. (2002). "Clinical competence assessment in nursing: a systematic review of the literature." Journal of Advanced Nursing 39(5): 421-431.

Weaver, C. S., Humbert, A. J., Besinger, B.R., Graber, J.A., Brizendine, E.J. (2007). "A more explicit grading scale decreases grade inflation in a clinical clerkship." Academic Emergency Medicine 14(3): 283-286.

Webb, C. and P. Shakespeare (2008). "Judgements about mentoring relationships in nurse education." Nurse Education Today 28(5): 563-571.

Page 47: National Approach to Practice - NHS Education for …...Page 4 of 47 4.3.4 Reasons for grade inflation: tool design 25 4.3.5 Suggested ways of controlling grade inflation 26 4.4 Use

Page 47 of 47

Williams, M., Bateman, A. (2003). Graded assessment in vocational education and training. Leabrook, Australia, Australian National Training Authority: 1-66.

Williams, S. Rowlands, A. (2009) “Evaluation of National pilot of Longitudinal Evaluation of Performance in Psychology”. Edinburgh, NHS Education for Scotland

Yorke, M. (2005). "Issues in the assessment of practice-based professional learning: A report prepared for the Practice-based Professional Learning CETL at the Open University." from http://www.open.ac.uk/cetl-workspace/cetlcontent/documents/464428ed4aa20.pdf


Recommended