Peer Supported Review Evaluation (PSR) - An inclusive approach to

professional development in learning, teaching and assessment

 BERA 2011

Sheffield Hallam UniversityRajesh Dhimar - llrrd1@gmail.com

• In 2004, Academic Development Committee (ADC) approved a framework for Peer Supported Review of Learning, Teaching and Assessment

• In 2006, ADC advised on the paper, The Implementation of Peer Supported Review of Learning, Teaching and Assessment

• In January 2010, ADC reviewed Peer Supported Review: Update and Overview of Activity March-December 2010

Context

Context...

• In May 2010, ADC approved the Peer Supported Review Policy for Academic and Learning Support Staff. This policy provides a definition and key principles underpinning PSR. It forms part of the implementation of Professional Standards in Teaching and Learning: an integrated approach to appraisal, professional development and peer supported review, endorsed by ADC in 2009.

 • The Professional Standards Steering Group has responsibility for

the development, implementation and evaluation of the Professional Standards work.

This work was supported and resourced by key staff from the Learning and Teaching Institute and those with PSR responsibility at Faculty and/or Department level.

Resource Implications

Internally:• The University's Corporate Plan (2008-2013); sections 2

(Improving the Student Experience) and 5 (Managing and Developing our People).

• SHU QAA Institutional Audit (2010), through provision of evidence to address the previous QAA Institutional Audit (2005) which required the University to "reassess how the staff appraisal and peer supported review of Learning, Teaching and Assessment systems might be more effectively used for the assurance of teaching quality in addition to the enhancement of teaching standards".

Key Reference Points

Key reference points cont...

and externally:• UK Professional Standards Framework (HEA, 2006)• DIUS Inquiry into Students and Universities (October 2008) • Future workforce for HE (HEFCE commissioned report

(December 08)• The Higher Education Academy reports on Reward and

Recognition of Teaching in HE (February and December 2009)

• The outcomes from this evaluation paper will feed into the further development and implementation of the PSR Policy.

• The outcomes will be shared with relevant Faculty and departmental groups/committees.

• Monitoring will be through regular updates, reporting to ADC and the Improving Student Experience Professional Standards Group.

Dissemination, Implementation and Monitoring

Areas covered:• information on the background and context of PSR

evaluation• key findings from an evaluation into the practice of PSR

over 2009/10• recommendations for the enhancement of PSR

Evaluation of Peer Supported Review (PSR): 2009/2010

• Following the Peer Supported Review of Learning, Teaching and Assessment policy in 2004, a number of evaluations (including HWB, 2004/5 and D&S, 2008/9 identified variable practice. The subsequent rejuvenation of PSR arose from wider university discussions and evaluation during 2009, building on existing effective practice.

• This resulted in a refreshed policy, Peer Supported Review Policy for Academic and Learning Support Staff (May 2010), which defined PSR as:

….. a developmental process through which individuals review, reflect on and enhance their practice, with the support of colleagues.

Background and context

The aims of Peer Supported Review are to: - promote and enhance the scholarship of learning and teaching through reflective practice - improve the quality of student learning through a process of review and development of professional practice.

The main aims of the evaluation were to: • capture the current range of PSR activity and experience

across the University• provide evidence to inform the development of PSR

policy and practice• provide evidence for institutional objectives in the

Corporate Plan (2008-13) • provide information for internal and external

dissemination.

The 2010 Evaluation

The evaluation was conducted in the Faculties of Health and Wellbeing (HWB), Arts, Computing, Engineering and Sciences (ACES) and Student and Learning Services (SLS).  The Faculty of Development and Society (D&S) had an external evaluator from the HEA and Sheffield Business School (SBS) had carried out their own evaluation.

• an online survey in Blackboard (VLE) for colleagues involved in the process. This included questions on staff profiles; PSR groups and process; topics and focus; reflection; engagement; impact; issues and benefits.

• semi-structured interviews with staff in key roles, including Heads of Department; Heads of Learning, Teaching and Assessment (LTA); Heads of Quality; LTA leads; Line Managers. Topics covered staff roles and responsibilities; PSR structure and processes; issues and benefits; overview of practice and impacts; future plans.

The evaluation began in June 2010 and used:

• PSR is perceived to be a positive and constructive approach to the Professional Development of academic practice, providing opportunity and structure within which to develop practice. Its core principles were validated and visible in practice.

• PSR was seen as an opportunity to confirm and validate reflective practice, and for colleagues to more openly engage with learning and teaching. The vast majority of respondents felt their practice had improved.

• There were observations that some staff have a history of reflecting on their practice and discussing teaching with colleagues; thus, change/development already occurs. The challenge is to use PSR to build on existing practice and move developments on further.

Key Findings1.Response to Peer Supported Review

• PSR groups seemed to work well as a mechanism for support and collegiality. A wide variety of models was illustrated, each designed according to context and need. Some were 'manufactured', others self-selecting and of varying sizes. They were seen as flexible and adaptable.

• Most groups formed within their subject area, with few examples of cross-Department/Faculty activity.

• Associate Lecturers, Part-time and Technical staff were rarely included in PSR groups.

• PSR groups used a variety of activities to support their PSR colleagues, most frequently:

- identifying useful resources/ideas- helping with reflection and evaluating practices- giving feedback e.g. on materials; e-learning approaches; teaching- observing teaching.

2. PSR Groups and Activity

• Whist many interviewees felt staff had already developed the necessary skills e.g. reflection; critical friendship; giving feedback; observing colleagues, it was felt that further guidance would be useful.

• Clear triggers informed PSR activities, many relating to university activities and processes e.g.:

- students' feedback e.g. NSS; first year experience; course committees- module/course review (often in teams)- validation.• Some staff, individually or as a group, had recorded their own

development, through:- paper-based materials e.g. Portfolio; learning diary; reflective writings- electronic resources e.g. group wiki; web-based personal portfolio; blogs.

PSR Groups and Activity…

• The choice of topic was primarily based on personal reflection, generally in relation to Subject/Departmental contexts. The most effective process of determining topic seems to start with a collegial approach, with discussion and exploration of relevant possibilities.

• As might be expected, actual topics varied considerably, according to context. The most common were:

- e-learning e.g. audio feedback; Web2.0; mobile learning; electronic feedback- assessment and feedback e.g. peer assessment; peer feedback; re-design of tasks- curriculum development e.g. employability skills; incorporating research - teaching e.g. large groups; changing approach to reduce dependency.

PSR Groups and Activity…

• At all levels, staff identified current or potential links between PSR and existing University/Faculty processes, commenting that explicit connections and synchronisation are essential to the success of PSR as a developmental tool. Examples included:

- Appraisal, Self-Managed Time, Work Planning- Keep-in Touch meetings (KITs), Departmental/Subject Group meetings- Staff Development, Continuing Professional Development (CPD)- Course design, Validation, Course/module evaluation- Annual Quality Review.

3. PSR links with other processes

• There was clear consensus about the importance of explicitly connecting PSR with Appraisals. This had occurred in some areas, although only half the questionnaire respondents had experience of it.

• There was a strong view that following on from Appraisal, PSR should be clearly identified within both Annual Work Planning and Self-Managed Time, and align more with teaching loads. This had occurred in only a few areas.

• PSR was clearly connected with Professional Development, as a vehicle to develop practice. PSR has enabled some Subject Groups to identify broad professional development needs via the collation of information (PSR forms). This enabled a more coherent professional development strategy at Department/Faculty level.

• PSR was seen as a good source of evidence for Professional Body CPD requirements and for continuing registration (often via a portfolio). There was no duplication of activity reported, and some observations about complementary activity. However, there were issues where CPD requirements focus on the discipline e.g. for research-active staff.

PSR links with other processes…

Key enabling approaches included:• individual ownership of the process and personal

responsibility• flexibility - designed to suit the purpose, the individual and

the context • understandable and transparent process • dialogue with colleagues (including managers) and peer

support • structurally visible PSR leads at differing levels

(Faculty/Department/Subject).

4. What helps it work?

• Dissemination was identified as a key activity in the PSR process, for sharing ideas and practice, 'closing the loop', deeper reflection and developing greater collegiality. This occurred at different levels, Faculty/Department/Subject, and included:- Faculty LTA conferences and showcase events- Subject Group Away days; staff meetings; collective course reviews

• In general, the paper-work and forms were seen as minimal, appropriate and working well. Issues related to avoiding a 'tick box' approach and ensuring the information collected was used appropriately.

• Spreadsheets and visual diagrams were used to collate information from the forms, and were usually accessible by LTA and Quality Heads/co-ordinators, Subject Group Leaders and Line Managers. Cross-departmental priorities/resources were identified.

What helps it work…

The main issues included:• time and timeliness

o insufficient time. (However, some staff had 10 hours on work plan.)

o some complaints that the process started too late (Oct/Nov). • lack of clarity of process and expectations e.g.:

- the perception that it is a management imposed process - level of confidentiality within the process - understanding and interpreting information about PSR; confusion about the purpose and intended outcomes e.g. is it peer observation again?- how best to support PSR group colleagues; responsibility and expectations

5. What hinders?

• existing culture and practice within staff group. e.g.:- level of collegiality, communication and openness - motivation and engagement with LTA- skills and confidence of staff- staff identity - some see selves as practitioners first and academics second - how does it work for those who primarily undertake research?

What hinders…

• Actual impacts were difficult to ascertain as yet. The few identified related more to staff than to students. Likely impacts and the most useful aspects of PSR were:

- greater collegial working; team building; improved communication - improving staff confidence and increasing engagement in LTA- seeing reflection as a professional activity- greater coherence - PSR feeds into departmental priorities. with further benefits of:- helping develop a culture of peer support - encouraging and raising awareness of professional development- staff questioning and evaluating practice- giving recognition and value to teaching activities- increasing accountability and quality.

6. Impacts and Plans

• Managers had clear plans to develop PSR, and raise its profile. All were interested in practice elsewhere. Key areas for development were to :

- develop cross Subject/Department/Faculty PSR activity- further work with staff (groups and individuals; ALs; Technical staff)- evaluate approaches and impacts- improve information e.g. examples; documentation; dissemination- develop links with other processes e.g. work planning; Appraisal; validation- explore rewards and recognition.

Impacts and Plans…

The following recommendations have arisen from comments and suggestions, and are intended to: • maintain the momentum of PSR as a developmental tool • clarify, improve and disseminate information and

guidance • enable equity, accessibility and entitlement to engage

with PSR • increase engagement in areas where it is lower or under-

valued.

Key Recommendations

• Ensure PSR is a feature in Appraisal, as a core aspect of Professional Development. Develop guidance for Appraisers and Appraisees

• Annual Work Planning and Personal Scholarly Activity/Self-Managed Time to explicitly include PSR

• Within Faculties/Departments, PSR cycle to match other appropriate cycles.

1. Link PSR with University/Faculty/Department processes

• Develop a university-wide strategy for the inclusion of Associate Lecturers and Part-time staff

• Develop strategies to include Technical staff and Graduate Teaching Assistants, trial and evaluate.

2. Enable equity and access to the process

• Develop a web-based repository of materials, guidance and information

• Review guidance information, for clarity e.g. expectations/responsibilities

• Identify and/or develop relevant materials and resources, based on best practice e.g. reflecting on your practice; supporting PSR colleagues; evaluating impact

• Identify and trial appropriate recording tools for staff, including electronic portfolios.

3. Further develop materials and resources

• Continue to work with the cross-university PSR Advisory Group as a means for development and dissemination of ideas and practice

• Facilitate interest groups - e.g. Subject Groups with Professional Body requirements; Line Managers

• Identify and promote effective mechanisms for cross Department/Faculty working

• Consultation around the inclusion of staff with a primary focus on research.

4. Further cross-university facilitation and consultation

• Ensure appropriateness of documentation for the range of staff using it; fitness for all purposes, needs and contexts

 • Identify strategies to increase effectiveness

e.g. electronic forms; combining 'forms' e.g. Appraisal, PSR and Research forms.

5. Review PSR forms/documentation

 • Further investigate the management of PSR

 • Develop guidance/suggestions for those

managing PSR, to include the findings of this evaluation e.g. responsibility allocated at appropriate levels.

6. Identify, disseminate and support effective approaches to manage PSR

• Design mechanisms to evaluate the outcomes of the PSR process at different levels and its impact on both staff and students' experience.

7. Develop evaluation and impact strategies