AN EVALUATION S T U D Y OF I N D I V I D U A L P E R F O R M A N C E REVIEW AT M O U N T V E R N O N
CENTRE FOR CANCER T R E A T M E N T
Sarah James
Mount Vernon Centre for Cancer Treatment, Mount Vernon and Watford Hospitals' N.H.S. Trust, Northwood, Middlesex ttA6 2RN, U.K.
(Received 29 April 1996 and accepted 10 March 1997)
Purpose." Individual performance review (IPR) was formally introduced into the National Health Service in 1986. Since its introduction many health care professionals have become involved in IPR schemes as both appraisers and appraisees. At Mount Vernon Hospital a working party was established to design and implement a suitable appraisal scheme throughout the hospital, which commenced in 1992. Three years later, this study evaluated how effectively the scheme has been implemented within the Cancer Treatment Centre. Methods." Three different questionnaires were designed and used to survey the appraisees, appraisers and managers of each of the professional groups partici- pating in the scheme. Results. Approximately 80 questionnaires were returned, from which the results have highlighted many differences in the way in which the scheme is being implemented and how it is viewed by different groups of employees. Conclusions: The results from this study indicated that there are many variations in the extent and the manner with which the appraisal scheme has been implemented in the Cancer Treatment Centre. Recommendations, such as a review of the number of appraisees allocated to each appraiser and the format of the documentation, along with the need for more focused, regular training sessions are all suggested to increase participating personnel's commitment towards the IPR scheme in the future.
Key words: appraisal; human resources; organizational management.
INTRODUCTION
Widely held views that employees are the largest operational cost and therefore the most important resource of any organization has led to the acknowledgement that a lack of commitment towards this valuable resource could contribute to organizational failure [1]. Changes in the operational environment of any organisation, for example increased competition and the increasing role of management, will emphasize the need for any system or technique which would improve the supervision and effectiveness of its human resources. As employees have become more accountable for their perform- ance, managers have become more answerable for developing and supporting their' employees through these changes.
For many years, performance appraisal has been acknowledged to be an effective tool for human resource management. A survey by The Institute of Personnel Management
[2] indicated that performance appraisal procedures have been increasingly used at all organizational levels.
The theory of performance appraisal is based upon three psychological principles [3]; people work, learn and achieve more when they are given adequate feedback as to how they are performing; they need clear attainable goals, and involvement in the setting of these tasks and goals.
These basic principles have wider implications than just promoting improvements in individual performance and personal development. They are also concerned with the development of an effective organizational culture by helping to achieve the broader goals of creating satisfying, effective jobs, and encouraging the involvement of people in the organization to attain organizational objectives. If carried out effectively, an appraisal scheme will not only promote increased employee commitment and moti- vation, but will also stimulate good working relationships and lead to improved manager and employee communication. Performance appraisal is both a personal event between two people who have an ongoing relationship and an organizational event that is needed to form part of a human resource management system. It is therefore a major mechanism for integrating the individual and the organization.
Methods of appraisal The effectiveness of any appraisal system is dependent on reliable ways of assessing behaviour and performance. There are many different methods employed where an individual scheme may consist of a combination of features.
Evaluation may be made by measuring performance in terms of strengths and weaknesses. This looks at what the appraisee puts into the job, but fails to place emphasis on the need to achieve. Objective setting may be used, where assessment is made based on what has been achieved in the previous period.
Some systems use a rating scale, where levels of performance and ability are graded or ranked; other schemes may rely on narrative comments where the appraiser is prompted by headings within the documentation. These types of assessment have limitations. The use of a rating scale is dependent on all appraisers using the same standards by which to grade appraisees, and therefore relies on all appraisers being equally and adequately trained and objective. Both systems rely on impartiality and fair assessment of all appraisees by all appraisers. There are also difficulties about who should appraise whom. It is usual for the line manager to undertake the appraisal, but other options include assessment by senior management, peer colleagues, self appraisal and upward appraisal.
Appraisal schemes may also be pay related, where levels of performance influence pay awards given for the following year. Within the NHS this has only been introduced for senior management, and only for a small percentage of their salary. For the majority of staff the motivational effects are thought to be considerably weaker because outcomes and performance levels are not under the direct control of the individual. There would be too many unquantifiable objectives, and assessment may even depend upon measurable targets which do not reflect effective performance [4].
Motivation is known to be related to social needs and self fulfilment as well as economic motives. Rewards which motivate people not only differ between individuals but also within the same person at different times. Performance related pay has not therefore been regarded as appropriate to provide incentives to promote high performance for all levels and professions of employees [4].
Individual Performance Review 295
The introduction o f appraisal into the N H S Individual Performance Review (IPR) was formally introduced into the National Health Service in 1983 as part of the development of management techniques after the Griffiths' review 1983 [5]. In 1986, it was made obligatory for all general managers, gradually extending downwards to all personnel with some management responsibili- ties. More recently, the NHS White Paper 'Working for Patients' 1989 [6] has highlighted the need for more emphasis to be placed on the IPR process to enable the successful implementation of the recommendations within this White Paper. Many self governing Trusts have also pinpointed IPR as a future management tool to enable organizational goals to be achieved.
The introduction o f appraisal at Mount Vernon N H S Trust Until 1991 Mount Vernon Hospital was managed by The North West Thames Regional Health Authority under the guidance and directives of the Department of Health. In April 1991, Mount Vernon Hospital was granted Trust status within the first wave of appointments and in 1994 became a joint Trust with Watford Hospital. The achieve- ment of Trust status has enabled Mount Vernon to become a more strategic and autonomous organization.
The organizational culture of the hospital ~as had to adapt and managers have been given three prime tasks:
1. to obtain the highest possible performance from their staff ensuring that activities are commensurate with the business plan and objectives.
2. to ensure that their staff are providing a quality service to all their customers. 3. to develop individuals for greater capability and flexibility in their present jobs or for
more responsibility in the future.
Due to the commitment of the hospital to provide a quality service and to fulfil the hospital's mission of 'providing quality health care enhanced by research, training and development to meet the needs of the patients and staff', the management decided to introduce a Total Quality Management initiative.
During the initial stages of implementing this system the management became aware of communication problems within the hospital with many staff unclear or ignorant about the hospital's management structure. Many managers were quick to highlight mistakes and there was little recognition for tasks done well, with almost no praise.
As a result of these deficiencies, it was decided that IPR would be introduced throughout the hospital. A working party was established with the responsibility of implementing a suitable system for all departments and all grades of staff employed at Mount Vernon Hospital.
The system of appraisal which was devised took the form of IPR that would not be pay related. The same system was to be used for all employees in all departments with its main objectives of providing a user friendly system, to strengthen employee commitment and motivation, stimulate good working relationships and improve communication. The prime tasks for managers were to be as previously described.
The scheme would apply to all staff within the hospital and it would be the responsibility of every manager to review the performance of those staff directly accountable to him/her.
Performance discussions took place as frequently as thought necessary by the manager and the member of staff, with the recommendations of a formal review of an
296 James
individual's performance at least once a year (or more frequently if both parties agree). These would be accompanied by regular informal reviews at least every 3 months so that both parties had the opportunity to discuss progress and perhaps review objectives in the light of current circumstances.
The review period would be timed to coincide with the financial year i.e. 1 April to 31 March, with managers to undertake formal IPRs between April and June. At the latter, performance in the previous financial year would be discussed and objectives for the next year set. All IPR discussions would be completed by July each year.
The documentation of the scheme included: (a) Personal preparation forms to be completed by every individual prior to their
formal annual interview. This form included questions referring to the individual's objectives of the previous year and whether they had been achieved. Questions were asked about the level of job satisfaction gained by the employee and what career and personal developmental aspirations they had.
(b) Performance Appraisal forms were to be completed by the appraiser during and after the interview. This form included a section to agree objectives for the oncoming year and a section for rating the individual's level of performance over the preceding year. This was done by grading the individual's ability towards attributes such as knowledge, understanding, skill, dependability, initiat- ive, creativity, quality, quantity and accuracy. This section also included an opportunity for narrative comments about the appraisee's performance and achievements during the preceding year.
This scheme has been in use within the Cancer Treatment Centre since 1992.
Aims of evaluation study This study was designed to evaluate the effectiveness of this IPR scheme in operation at Mount Vernon Centre for Cancer Treatment by surveying all radiographers, nurses, physicists, secretarial and counselling staff. Collectively these personnel constituted the majority of participants of the scheme within the Centre.
The aims of this study were to: (i) identify strengths and weaknesses of the IPR system from the perspective of the
appraisee, appraiser and manager. (ii) analyse whether the system was being implemented uniformly across the
professions being surveyed.
M E T H O D O L O G Y
The use of questionnaires was chosen as the tool to gather the required information, since it is efficient and effective in surveying the number of personnel involved. Mount Vernon Centre For Cancer Treatment is a busy department, where staff have high levels of work to achieve, and it was felt that if it was conducted in an appropriate manner a questionnaire survey would yield high response levels with minimum disruption to the workload of those participating.
Three different questionnaires were developed to enable the appraisees, the appraisers and the managers to be surveyed. All three questionnaires were carefully designed using clear and simple language to eliminate confusion, ambiguity or bias from the responses and to ensure the best rate of completion and return. Easy to follow instructions were
Individual Performance Review
Table 1. Sample size of professions surveyed within the evaluation study
used with a logical flow through the topics of information required to ensure that the respondents completed the entire questionnaire [7].
Topics included within the questionnaires were: • level and format of training undertaken and whether it was considered adequate. • number of appraisees allocated to each appraiser. • number of formal interviews and informal review interviews undertaken. • length of interviews and preparat ion time for them. • individual's views on the scheme structure such as the use of informal review
interviews, the fo rmat of the documentation. • individual's views on whether the scheme was considered to be effective and reasons
for the views held.
Study population All staff in relevant categories were included, Table 1 details the sample size of each profession.
Timescale The distribution of the questionnaires was timed to occur after each individual had undergone their formal annual review interview or 3 monthly review. This ensured that staff had recent involvement in the scheme and that their views and opinions were easily remembered and therefore reflected in their responses to the questions.
The results f rom the completed questionnaires were reviewed to establish their collective answers and comments.
RESULTS
Seventy nine questionnaires were issued and 71 were returned, a response rate of 90%.
Questionnaires returned All the managers and appraisers returned their questionnaires, but only 88% of the appraisees.
Overall, the analysis of the data indicated that there were variations seen in many aspects of the scheme's implementation. These included factors such as: • the level of training undertaken. • the time taken to prepare and conduct interviews. • the number of interviews undertaken.
298 James
• use of personal preparation forms. • number of appraisees allocated to each appraiser • views of the appraisal scheme. Many of these variations occurred not only across the different professions but also within the same profession.
Training All managers, 90% of appraisers, but only 59% of appraisees received training. There were mixed views about the usefulness of this training, managers thought it adequate, but both appraisers and appraisees felt if could be improved.
Three monthly reviews Managers felt they were useful, but one was unable to undertake them due to the high number of staff within her responsibility. Appraisers and appraisees agreed they were useful, but both categories acknowledged that there was a wide variation in their use, ranging from 0-4 per annum. This was felt to be partly due to some appraisers being allocated too many appraisees.
Documentation All categories of respondents agreed that the forms were too long, repetitive, and both appraisers and appraisees felt that some aspects/terminology were inappropriate for their work.
Effects of interviews upon job satisfaction Only 56% of appraisees felt that their appraisal interviews had any effect upon their job satisfaction; 90% of these effects were described as positive.
Views of the scheme All managers were in favour. Ninety percent of the appraisers were in favour, but only 64% of appraisees, with 30% not sure. The main reasons given were a dislike for being appraised by somebody who did not work with them enough and were therefore considered unable to review their performance. There was also a general feeling that if they did not see many or any benefits then why bother with it.
Table 2 summarizes the results found from the study.
DISCUSSION
The high return rate of questionnaires from the sample population was felt to be due to several factors. The researcher attempted to minimize the known poor response rate to questionnaire surveys [7] by careful design of the questionnaires and the method by which they were distributed. This high response rate achieved may also have been influenced by the staff awareness of the ongoing changes within the health service, for example the current round of pay negotiations, the possible loss of national pay bargaining and the instigation of trust contracts, with the possible loss of standard employment terms and conditions which have always been taken for granted in the past within the NHS.
The fact that all the managers and appraisers returned their questionnaires reflected their acknowledgement that IPR was a required part of the management of employees
within the NHS [5]. In comparison, the appraisees level of response rate of 88% indicates that the commitment and motivation of this latter group of participants towards the organization and/or the appraisal scheme was not so high. These response rates are particularly interesting when compared to the results of the number of individuals who have attended any type of training about appraisal. All the managers, 90% of the appraisers but only 59% of the appraisees had undertaken any such training.
It was also noticed that the views of the scheme varied across these groupings. All managers, 90% of appraisers and 64% of appraisees view the appraisal scheme as a good thing. These three factors may be linked. Where there was lack of involvement, due to less training for appraisees, this led to a reduced ownership by this group and resulted in a lower response rate for the study. This under-investment towards employees, by not ensuring training for all staff, possibly confirms the discussion of West and Anderson [1]. They highlight the fact that when employees are overlooked, this leads to a lack of ownership of their organization.
Appraisal training Although all of the managers and the majority of the appraisers (90%) felt that the training they received was adequate, there was an indication that there was room for improvement. The finding that 10% of appraisers had not been trained indicates that the level of commitment towards IPR is not sufficient at present. The suggestion for follow-up sessions after an initial training session would seem very logical. Regular follow-up training sessions would allow all participants to discuss current situations/ problems in which they are involved. Such discussion sessions with peer staff, who may be experiencing similar circumstances, may help them to solve their own situations and/or provide assurance that others are experiencing similar problems.
Employers cannot expect workers to have a high level of commitment towards a scheme that has not sufficiently trained those who are going to implement it. The current level of commitment to IPR by the hospital management reflects the changes that have occurred within Mount Vernon Hospital in the last year. Since becoming joint Trust status with Watford Hospital there has been a very low emphasis placed upon IPR across the hospital generally, although the Cancer Centre chose to continue with the original scheme. Any change in organizational structure and/or culture will necessitate a review of management processes in use [9].
The 24% of appraisees who felt that training was not adequate, also suggests that the format of the future training sessions needs to be reviewed. The dislike for multi- disciplinary sessions and the lack of follow-up sessions was highlighted on several appraisee questionnaires.
Three monthly interviews The views and use of 3 monthly review interviews was found to be very varied. Four of the managers viewed them as beneficial, whilst the fifth manager acknowledged that she was unable to undertake them due to the high number of staffthat she appraises. All the appraisers agreed that 3 monthly review interviews were useful, but the extent to which they were carried out varied ranging from none to three per year. The results from the appraisee questionnaires indicated that there were very mixed feelings about their usefulness. Views range from a feeling that these interviews were a waste of time, to the comment that they were more useful than the annual review because of feeling less anxious about its occurrence. This may be a reflection of their limited experience of the scheme.
Individual Performance Review 301
These varied comments may also suggest that the number of informal review interviews should be specified for each individual, to suit their current needs. Any future scheme would need to allow more flexibility, and need to guarantee an equal level of supervision to all staff. Comments from the appraisee questionnaires requesting a more disciplined approach towards undertaking all interviews may imply that some individ- uals feel that the scheme is not uniformly implemented across all staff. The suggestion for more focused and regular training sessions would solve this problem whilst also helping to increase ownership of the scheme by both appraisers and appraisees.
Documentation Dissatisfaction with the design of the paperwork was expressed by managers, appraisers and appraisees. All groups felt that the format of both the personal preparation form and the assessment form were inappropriate for the following reasons, Some sections of the performance assessment form are inapplicable to specific jobs or professions. It is very repetitive and therefore unnecessarily lengthy. The pitfalls of design may encour- age both appraisers and appraisees to feel that it was not worth participating in the scheme. The personal preparation form was viewed as generally difficult to complete and not appropriate to certain professions and/or grades of staff. If staff considered the forms inappropriate their motivation towards using them would be very low, but these problems may be overcome by redesigning their structure. The indications that individuals were not comfortable using the documentation implied that this aspect of the scheme was not being implemented very well or consistently with the possible consequence that appraisees were not being fully encouraged to set their own objectives. This reflects Fletcher and Williams' [8] discussion about the acknowledgement that appraisal schemes are only as effective as the people using them, and this lack of commitment to the scheme design would indicate that an overhaul is required to increase the employees ownership and therefore hopefully improve commitment.
Number o f appraisees per appraiser The variation in the number of appraisees for each appraiser is very considerable. The manager who appraised over 20 employees adapted the scheme herself to a level that she could comply with. She accepted that each appraisee would only have been interviewed once a year.
This may be reflected in the views of the scheme by the individuals she appraised. A logical consequence for individuals having only experienced one interview is that insight or experience of any benefits or disadvantages may be restricted. Another effect of having only one interview per year may also necessitate that the interview be longer, with these appraisees left feeling unattended in comparison with colleagues who received a greater number of interviews. As a consequence they would be unable to modify objectives if circumstances changed during the review period. It may be appropriate to consider limiting the number of appraisees per appraiser in order to try to ensure that no one appraiser is over-burdened and that all appraisees get equal levels of attention. This may necessitate modifications to the scheme design. Suggestions such as delegation to more junior members of staff may be appropriate. It may be worth considering peer or self-appraisal for this staff group.
The use of self-appraisal also has its own problems as it can lead to appraisees feeling that they have less influence over the appraisal. Both of these suggestions would also reduce or possibly eliminate the occurrence of another grievance felt by some
302 James
appraisees, that of being assessed by somebody who does not work directly with them. The comment was made that this seems unfair and could possibly be an unreliable method of assessment.
Fletcher and Williams [8] discuss the many factors that may influence the accuracy of any appraisal scheme and they acknowledge that different appraisers may apply different standards when assessing individuals. The ability and motivation of any appraiser may be influenced by lack of experience and/or training. These factors are likely to affect the quality of the appraisal and it is for these reasons that the immediate supervisor of an appraisee may not be the appropriate person to be their appraiser.
These views confirm the need for good training to all staff likely to be delegated the role of appraiser and that this responsibility is not to be allocated to any individual until adequate training has been undertaken. The follow up training sessions previously suggested would assist newly allocated appraisers whilst they are gaining their initial experience.
Effects of in ter views A high majority of respondents acknowledged that they felt frank discussions had taken place with proposals agreed, leading to increased motivation. It was disappointing that only 56% of appraisees felt that their appraisal interview had affected their job satisfaction. This implies that these interviews had not achieved the organizational and individual objectives. This could be an underlying problem of insufficient training resulting in a lack of ownership/commitment of the appraisers. There is room for considerable improvement with this scheme when it is remembered that only 56% of appraisees acknowledged any effect of their appraisal interview upon their job satisfac- tion. This effect rate may indicate that it is questionable that the scheme in its present format is worthy of the resources that it demands. However, it was encouraging to find that of this 56% the vast majority of comments regarding their satisfaction having been affected were positive factors. This could imply that where the scheme was being fully carried out it was being implemented well and/or having mainly beneficial effects. This suggests that the scheme would be worthy of greater time and resource allocation to gain more benefits.
Views of the appraisal scheme The appraisees expressed many reasons for their views on the appraisal scheme. They acknowledged the positive benefits specified by the managers and appraisers. From the 64% of appraisees who felt that the appraisal scheme was a good thing, there were many suggestions made that improvements could be implemented to ensure that the 5% who were not in favour of it and the 30% who were not sure, could see some benefits for themselves. Many appraisees described feeling anxious when they know that the appraisal interview is approaching. There were concerns about impartiality, fairness and the process possibly being dependent on the individuals concerned. These com- ments could reflect a lack of understanding of the theory behind appraisal schemes (see introduction) where emphasis is to be placed on involvement in objective setting and positive feedback. This lack of knowledge about the appraisal scheme was the result of insufficient and or poor quality training. The interviews are not or should not be an opportunity to reprimand employees, but this was perhaps what appraisees were concerned about.
Individual Pelformance Review 303
Other comments from the appraisees questionnaires reflect general doubts as to how the scheme is implemented and puzzlement concerning the number of interviews that should take place. Their comments suggested that they would like to have seen more standardization in the implementation of the scheme for all employees. These doubts wouldal l be solved by more resources and commitment, in the form of longer training and time to carry out the interviews.
Conclusions The results from this study indicated that there are many variations in the extent and the manner with which the appraisal scheme had been implemented within the Cancer Treatment Centre.
The overall high response rate of the questionnaires implies that there was a high level of interest about its future. The detailed results from the questionnaires of each category and profession surveyed indicated that there were differences in all aspects of how the appraisal scheme was viewed and the method in which it was implemented.
Some of these effects may be due to the formation of the joint Trust with Watford Hospital in 1994 which resulted in the hospital managers placing a lower emphasis on the IPR scheme within Mount Vernon Hospital. Departmental managers were encour- aged to continue with the IPR scheme where it had already been implemented, but there was an acknowledgement that no developments or change would occur to the scheme until its future had been decided within the new Trust.
The Cancer Treatment Centre decided to continue the use of the IPR scheme with the designated appraisers taking on new appraisees as new employees commenced work within the Centre. It could be speculated that although there was a general view that this scheme was worthy to be continued at the time, the reduced input from the human resources department could result in a lower level of training sessions for new staff. This may have left the current employees feeling lethargic or unmotivated towards the scheme, as there was an anticipation of possible future change. This was reinforced by the fact that departments that had not already commenced 1PR were not encouraged to do so. This acknowledgement by the hospital management that the IPR scheme would change will have a demotivating effect upon those who continue to use it, especially if they feel dissatisfied with the current format, which is the case when considering the results of this study.
A large section of the participants within this study were in favour of the IPR scheme. It would seem that where the scheme was implemented well it may have resulted in many benefits. However, the variation in length and number of interviews along with the dissatisfaction with some aspects of the scheme design i.e. the delegation of appraisers and the documentation suggested that a complete review of the scheme was required.
Recommendations As a result of this study the following recommendations were made:
1. Increased levels of training to be provided for all participants. With consideration of the need for regular sessions for newly designated appraisers where they could discuss their experiences after the first round of appraisal interviews.
2. The number of appraisees allocated to each appraiser should be limited to that which is manageable for any one individual. Managers of each profession must review their
304 James
current list of allocated appraisers and consider designating more individuals with this responsibility to ensure that all employees are provided with equal levels of supervision. This would probably result in delegation of appraising to the next downward level of management thus ensuring that appraisees are appraised by direct managers.
3. A review of the forms currently used. Modifications to both the personal assessment form and the personal preparat ion form are required. It is essential that appropriate formats are implemented to increase the level of motivation and commitment by all personnel towards their use, and therefore to the scheme in general.
4. Consideration should be given to the number of informal review interviews required. It may be appropriate for the number of these interviews to be decided on a more individual basis so that they are useful and achievable without burdening appraisers with further work which reaps no further benefits.
References
1. West MA, Anderson NR. Fire Fighting. Health Service Journal, 28 October 1993: 2~4. 2. Long P. PerJormance appraisal revisited: third I.P.M. survey. Institute of Personnel Manage-
1993. 4. Orchard C. Mindyour language. Health Services Management. December 1993: 24-5. 5. Department of Health and Social Security. The National Health Service Management Inquiry
(Griffths Report). Department of Health and Social Security 1983. 6. Secretaries of State for Health, Wales, N. Ireland and Scotland. Working for patients. the
Health Service. London: HMSO 1989. 7. Oppenheim AN. Questionnaire design and attitude measurement. London: Heinemann, 1992. 8. Fletcher C, Williams R. Performance appraisal and career Development, 2nd edn.
Cbeltenham, 1992: Stanley Thornes (Publishers) Ltd.
