Interim Evaluation Report - Integrated Classroom...

April 12

Interim Evaluation Report: Year 1

w w w . m a c a u l a y a s s o c i a t e s . c o . u k

Tony Macaulay Activity and Progress Report of the Evaluation of Year 1 the Northern Ireland Council for Integrated Education’s ‘Sharing Classrooms: Deepening Learning’ Project

Table of Contents

1 Introduction......................................................................................................3

2 Background.......................................................................................................4

3 Evaluation Methodology...................................................................................5

4 Activities...........................................................................................................6

5 Project Baseline Data......................................................................................15

6 Learning..........................................................................................................226.1 Project Participants.............................................................................................................................226.2 Project Delivery Team........................................................................................................................28

7 Recommendations..........................................................................................29

Appendix I: Evaluation Framework........................................................................30

Appendix II: Online Learner Survey.......................................................................34

Appendix III: Training Course Evaluation Form Comments....................................39

2

1 Introduction

This is the Interim Activity and Progress Report for the Evaluation of Year 1 of the Sharing Classrooms: Deepening Learning Project, which is funded by the International Fund for Ireland (IFI) and delivered by the Northern Ireland Council for Integrated Education (NICIE).

The project’s independent evaluator, Macaulay Associates Network1, gathered and analysed information for this internal evaluation2 between during February and April 2012.

The objectives of this report are:

To summarize the activities of the project to date. To summarise levels of participation and evaluation feedback

from participants. To present the findings of the analysis of pre-participation

surveys completed by learners. To highlight key learning to date. To make any relevant recommendations as a result of the

interim evaluation of the first year of the project.

This report includes:

A description of the background to the project.

An explanation of the project evaluation methodology.

A description of project activities and participation to date.

An analysis of training and conference participant evaluation forms and the pre participation learner surveys.

A presentation of key learning gained from the project to date.

A series of recommendations for the project based on the findings.

A set of associated appendices

1 www.macaulayassociates.co.uk

2 The project is also being evaluated by DETI.

3

http://www.macaulayassociates.co.uk/

2 Background

Sharing Classrooms: Deepening Learning is a shared education initiative, managed and delivered by NICIE and open to all post-primary schools in Northern Ireland.

The IFI made a grant of £702,790 to support the project for the period May 2011- December 2013.

The project has four strategic objectives:

(a) To increase awareness of the benefits of cross-sectoral collaboration in terms of ‘deepening’ the learning experience for pupils involved in Area Learning Communities (ALCs);

(b) To break down barriers and develop trust between educators on a cross-sectoral/cross-community basis;

(c) To widely disseminate a model for ‘integrating’ pupils taking part in shared classrooms; and

(d) To create a mechanism to ensure the sustainability of cross-sectoral shared classrooms and relationship building at a deeper level.

SCDL aims to enrich and deepen learning through developing teachers’ skills for creatively using the range of diversity found in shared classrooms. The project intends to provide professional development for teachers, who, as a result of Entitlement Framework (EF) collaboration arrangements between schools, teach classes comprised of students from different schools. As full implementation of the Entitlement Framework is required by 2015, the increasing movement of students between different schools and across sectors presents unprecedented opportunities for trust building, community relations/reconciliation work and learning.

In practical terms the project was designed to offer free professional development/training for teachers, who teach (now or in future) classes comprised of students from different schools as part of EF collaboration arrangements. In the first year participants from ALCs across Northern Ireland were offered a 3 Day Diversity Training Course and professional development on the theme of deepening learning in shared classrooms. The training was provided by independent facilitators who have been recruited on the basis of experience and understanding of community relations work in schools. Additional training, conferences and follow up support are also being made available, as the project develops.

4

3 Evaluation Methodology

The independent evaluator in consultation with NICIE, designed a comprehensive Evaluation Framework (see Appendix I), to meet the aims of the evaluation by measuring the impact and outcomes of the project.

The overall purpose of the independent evaluation is to evaluate progress against the strategic objectives of the Sharing Classrooms: Deepening Learning Project. The aims of the evaluation are:

To capture the impact of the SCDL project on the whole school, teachers and pupils involved in shared classrooms.

To assess the impact of the SCDL Project on relationships within and between participating schools.

To assess the impact of the SCDL Project on teachers. To assess the impact of the SCDL Project on pupils.

The Evaluation Framework includes the outcomes, measures or indicators and data collection methods devised to assess the effectiveness of the delivery of each of the strategic objectives of the project. Each strategic objective has a set of desired outcomes, and for each outcome there are measures or indicators of impact. Data collection methods include a range of approaches such as:

Pre and Post Participation Learners, Teachers and Leaders Surveys (both written and online).

Focus Groups and Interviews with Participants, Facilitators, Project Staff and the Management Group.

Participant Journals of a Sample of Participating Teachers. Participative Research with Young People.

The evaluation methodology used for this Report has been as follows:

Discussions with the Project Manager. Desk Research including analysis of project reports. Analysis of training and conference participant evaluation forms

and key learning forms. Analysis of Learner On Line Pre Participation Surveys (See

Appendix II) Observation of Facilitator Review and Planning Meetings (15th

December 2011 and 20th April 2012). Observation at the Conference on 29th March 2012.

This Activity and Progress Report follows the initial Baseline Report of March 2012. There will be third Year 1 Report in August 2012, which will include analysis of interviews, participant focus groups, comparative analysis of pre and post participation surveys, a summary of the impact of the first year and recommendations for the second year of the project.

5

4 Activities

The level of participation in project activities until 31st March 2012 was as follows:

A total of 96 people participated in Training Day 1 in November/December 20113.

A total of 47 people participated in Training Day 2 in March 20124. A total of 42 people participated in Training Day 3 in March 20125. Approximately 100 people participated in the Day Conference in

Armagh City Hotel, on 29th March 2012

The analysis of evaluation feedback forms completed by 86 participants at the end of the first day of training found that the majority of participants rated the programme, facilitation/delivery, learning, planned follow on training and the venues/catering/resources either good or excellent. The details are show in the bar charts below and the additional comments are in Appendix III.

3 The training was offered on five separate days in various locations: Corrymeela (twice), Greenhill YMCA (twice) and North West Training Resources Centre.4 The Day 2 & 3 training took place in four locations: Antrim Forum, Corrymeela, Greenhill YMCA and Tollymore.5 The Day 2 & 3 training took place in four locations: Antrim Forum, Corrymeela, Greenhill YMCA and Tollymore.

6

7

8

9

The group evaluation discussions following Days 2 & 3 of the training highlighted the following positive aspects of the course6:

Participants described the course as a ‘safe and secure environment’ where ‘you don’t have to watch what you say.’

A lot of useful content covered in a relatively short period of time. Participants appreciated how issues that had been identified

during Day 1 of the training were addressed during Days 2 and 3. Participants appreciated the opportunity to clarify and discuss

problems and benefits of shared classrooms. The discussions were described as stimulating and the variety of

approaches used (such as drama) maintained an engaging and very enjoyable learning space.

The icebreakers were described as meaningful. The activity on flags/symbols was particularly valued. There were positive comments about the quality of the venues and

the food provided. The supply teachers group valued the opportunity to meet and

learn together and were interested in further training of this kind. The experienced teachers group valued the sharing of ideas and

experience that took place among the teachers.

The group evaluation discussions following Day 2 & 3 of the training highlighted the following aspects of the course that could be changed7:

Participants wanted to spend more time working on strategies and solutions to all of the challenges they identified.

More time needed on debriefing of some exercises. It was suggested that a wider range of issues be covered in

addition to the traditional Catholic/Protestant differences.

The analysis of the conference evaluation feedback forms found that:

The conference increased understanding of ways in which school policies and processes support and deepen learning in classrooms.

The conference increased participants’ understanding of teachers’ experiences in shared classrooms.

The conference increased the majority of participants’ understanding of pupils’ experiences in shared classrooms.

The conference increased participants’ understanding of strategies that can build relationships/be used to manage difficult issues in shared classrooms.

The majority of participants found the key speakers and the workshops very useful or useful.

The majority of participants were either very satisfied or satisfied with the organisation/administration, the quality of the resources provided and the venue/facilities of the conference.

6 Source: Flip chart notes from Training Days 2 & 3.7 Source: Flip chart notes from Training Days 2 & 3.

10

11

12

13

14

15

5 Project Baseline Data

A total of 34 learners completed the pre-participation online survey. The findings are presented below.

16

17

18

19

20

21

22

These results will be used alongside the findings of the post participation survey for comparative analysis at the end Year 1 of the project. The comparative analysis of teacher, leader and learner pre and post participation surveys will be included in the final Year 1 Report in August 2012.

23

6 Learning

6.1 Project Participants

In addition to the learning highlighted above, the conference participants were asked to provide specific feedback on their learning from the conference workshops. This feedback is presented below:

Workshop A: In what ways can school policies and processes support and deepen learning in shared classrooms?

Key policies that impact on collaboration:

We feel it will affect all policies/majority of policies. Pastoral care policies need discussed and agreed by collaborating

schools. Discussion on elements of collaboration – joint agreements. Service level agreements. Procedures and building trust. Intrinsic agreement on what key outcomes will be. Recognition of demands on schools. Induction policy is key to effective collaboration. Uniforms, emblems, respect and tolerance. Citizenship Learning for Life and Work. Curriculum. RE. CEIAG careers. Board of Governors. Transport to and from and at various times throughout the day. Homework. Finance. Funding of lessons. Timetabling. Child Protection. Discipline/Positive behaviour. Assessment. Teaching & Learning. Equal opportunity policy which sets out the over arching aims of

collaboration. Every other policy would sit within the context of this. Staff Development Policy. Reporting. Reviewing. Pathways of progression. Communication – informing staff about policies and procedures

regularly. School development.

24

Ways in which policies and processes can be more effective in supporting collaboration:

All stakeholders understand the process involved in collaboration. This cannot be faked or forced – must become whole school

acceptance. Set out on a normalisation journey (Prospectus making

collaboration explicit). Agreed dates for terms, holidays etc. Spreading of good collaborative practice. Moving beyond the policies to bespoke induction processes. Create awareness in the local community and raising profile in ALC. Developing trust and agreement between staff and pupils. Staff development. Provides a joined up approach to deal with issues faced by the ALC. Each ALC context is different/unique. Issues in relation to attendance/assessment/tracking – SIMS Funding and timetabling. Broader support for policy development. Are the policies worthwhile/workable? Deciding as a group where do we start? (induction, pastoral care,

homework, behaviour). Joint working and development from all stakeholders. Raising awareness of all collaborative arrangements. MER (in particular the student voice). Deregulation of transport/barriers this places on timetabling. School calendar implications – work experience, mock exams etc. ALC Agreement – provision- discussion. Relationships between schools and staff – improving. Support from Department of Education (especially financial). Awareness in the community. BOG/principal influence. Involving all stakeholders. Assessment/tracking performance. Induction programme for pupils. More pupil centred induction procedures for foster greater

awareness and shared understanding and sense of belonging. ALC can have a greater impact on government and other agencies

(Translink, Council) when they lobby as a unit. Joint approach to policies –shared understanding. Clear expectations.

Challenges in developing policies and processes to enhance collaboration:

Practical issues, trust, time, transport, accountability, BOG attitude and influence, community perceptions.

Sharing information (SMS/c2K) Evaluating models and outcomes – assessment and review. Dealing with negativity, conflict, breakdown in relationships.

25

Consider looking at non exam/curriculum based model of collaboration.

Key lessons from SEN collaborations already established. Financial/ funding. Cost of enhancing curriculum provision.

Overcoming financial constraints and ensuring parity of esteem with numbers attending each school.

Timetable. Lobbying of ministers, DE etc. Alignment of calendars. Promoting local awareness – stakeholders. Logistical saturation. Spread of more courses. School sectors. Parents’ evenings/reporting/tracking. Operational/organisational/logistical issues. Shared days – what do they look like/are they effective. Induction for adults and students. Challenge of integrating technologies – SIMS How to promote ‘softer issues’ whilst delivering the EF within ALC. Ethos/values – individual schools – shared all schools. Range of different sectors. Differences in school ethos. Understanding ethos/aims of other

schools. Monitoring progress. Value for money vs outcome. Staff training (and morale?). Time restraints (staff). Duty of care/responsibility for pupils. Leadership within schools (principal to lead). Building trust among partners and especially among staff. Ensuring that the concept of sharing education and the sense of

collaboration moves fully down from principals to all staff.

Workshop B: What do we know about teacher experiences of shared classrooms?

What are the issues for you in the shared classroom?

Developing relationships with students. No communication with parents. Reporting and parents’ evenings. No formal feedback on progress. Formal links need to be established between participating teachers

via face-to-face meetings. Examination centres. Submission of coursework. Disciplinary issues and procedures. An ability gap – based on both differences in prior knowledge of the

subject and sectoral differences. Timetabling, co-ordinating holidays, travel allowed for in

timetabling.

26

To what extent does the presence of pupils from a different school influence teaching and learning?

Teaching style – awareness of abilities more so that before. Student profiling is crucial. Any individual education plans see special education needs co-

ordinator. Flexibility to adapt to change in pupil numbers. Emphasis on pupil evaluations. Classroom management – seating arrangements. Classroom assistance, behaviour expectations, class induction and

rules. Pastoral level – need a learning contract in a shared classroom to

have a common ground. Teaching styles – differentiation and awareness of abilities.

What are the key challenges in developing relationship that are conducive to deepening learning?

Time to plan. Communication. Getting to know your colleague. Lack of understanding of difference and sometimes unwillingness to

learn. Full induction. Pupils need to feel safe in their environment. Need to see the pupils more than just once a week. Is there a real need to develop relationships? Hard to develop relationships if the subject is not taught in both

schools.

Is there a role for teachers in developing relationships between pupils from different schools?

Yes. Facilitator role – guiding learning processes- responsibility of inclusion and acceptance of difference.

Teacher: providing the spark, induction, preparation and planning, resources, support.

Important to develop social relationships as well as passing exams. Breaking down gender/religious barriers.

What additional skills, if any, might be required for teaching the shared classroom?

Patience including the use of time to help understand differences in culture, ethos, identity and background.

Flexibility and willingness to adjust school practices and policies to meet the needs of the class.

Negotiation skills (to facilitate compromise). Open-mindedness about what is contentious! Tolerance of difference.

27

Communicating positions, interests, expectations, perceptions, needs. Courage to face up to how students and parents feel about not being

able to study a subject at their first choice school. Attention to differentiation. Possible medical training. Communication on classroom level, school level and ALC level. Motivating strategies for pupils passed on to teachers. Full disclosure about lots of pertinent individual pupil information is

vital to enabling this. Foresight to anticipate and prepare for issues likely to arise. Empathy. Courage to face up to underlying divisions/competitions etc.

In what ways might inclusion and respect for difference be fostered in the shared classroom?

Acknowledge differences, don’t ignore them! Time allocated for induction, contracting, creating a safe place –

which can be revisited as necessary if/when conflict arises. Social and non-curriculum time together – possibly a residential Mixed group work dynamics. Welcoming environment, displays to promote respect and diversity. Permission to talk about differences – create understanding and

mutual respect. LLW shared activities? A whole school approach!

Workshop C: What do we know about learner experiences of shared classrooms?

The school environment

School bell/statues/symbols/signage New technology access codes needed to get into schools.

Classroom environment

Seating arrangements. Shared display boards. Tea/coffee together.

Teachers not directly involved with the shared class.

Need an all school approach. Registers updated by secretary –SIMS. Management lead by example.

Teachers in shared classroom.

Build relationship. Preparation essential. Make classroom a safe and secure environment for learning.

28

It was clear during the conference that participants had benefited from the opportunities to explore the issues and challenges of sharing classrooms.

There was evidence of participants indicating how they had developed skills and confidence through SCDL. As two key speakers explained:

‘The training gave us the opportunity to discuss issues and problems with other teachers and to take part in a range of activities which we will be able to use with our classes- activities where pupils get a chance to interact and talk with others. Having taken part in the activities we have gained confidence and skills to undertake them in shared classrooms to enable pupils to integrated and achieve a sense of belonging, not just visiting a classroom.’ (Conference Speech, Mary Joyce, Teacher, St Colman’s, Ballynahinch)

‘Collaboration is not all plain sailing…We have received training through Shared Classrooms Deepening Learning Programme that has developed our skills and confidence to deal with the issues within the classroom and the whole school. We have also been given time to plan.’ (Conference Speech, Angela Casey, Teacher, High School, Ballynahinch)

The conference heard examples of how to deal with conflict in the classroom around community relations issues such as symbols and emblems which may be contentious .

‘The range of activities covered have given us the confidence and skills to perhaps try to resolve conflicts that may arise and have arisen in the past such as the wearing of the poppy or St Patrick’s Day.’ (Conference Speech, Mary Joyce, Teacher, St Colman’s, Ballynahinch)

During the conference there was active sharing of learning within and between ALCs in the workshops and there was evidence of enthusiasm of teachers towards sharing. The workshop discussions consistently highlighted issues such as the need for good induction, overcoming practical barriers, the importance of building trust and relationships and the need to ensure there is a whole school approach. Many participants at the conference expressed a desire to improve the quality of collaboration. As one participants put it:

‘We are now moving from the mechanics of collaboration to effectiveness.’ (Comment at Conference Workshop)

Participants expressed a desire for further support to improve the quality of their collaboration.

‘In May we will meet and plan an induction programme. We will receive and very much welcome the support of the staff from NICIE which we now know is available.’ (Conference Speech, Mary Joyce, Teacher, St Colman’s, Ballynahinch)

29

‘Our hope is that collaboration will not just meet the requirements of the Entitlement Framework or the learning of academic subjects but bring out a deeper learning of values such as respect, tolerance and understanding that will extend beyond the classroom.’ (Conference Speech, Mary Joyce, Teacher, St Colman’s, Ballynahinch)

6.2 Project Delivery Team

At the review meetings in December 2011 and April 2012, the project delivery team (staff and facilitators) discussed the feedback from schools, the evaluation to date and their own experiences of delivering the programme to date.

Their key learning can be summarised as follows:

a) Although the feedback from participants was largely positive, the running of centralised training courses proved problematic due to the wide diversity within each group (different types of school, different levels of experience, different areas and different roles), difficulties in engaging ALCs and in recruiting participants and cancellations prior to the training.

b) The programme has more potential to have a greater impact through a more contextualised, localised, needs led and bespoke approach to support, training and facilitation. This would require a change of approach away from providing centralised programmes towards a more open consultative process that offers facilitation support for conversations between leaders/partners (e.g. as requested by the Foyle ALC) and more specific and targeted training and classroom support for clusters/schools. This would require a stronger focus on leadership in ALCs in the first instance to see what support is required/ requested.

c) The ALCs/schools involved would request support, share responsibility for shaping the support and determine the level and depth of training and support they require at different levels. The content and approach of training needs to be appropriate, relevant and targeted to the specific needs and roles of the participants. Training should take place at a time and venue that suits the schools requesting the support. It is vital that the programme rationale, objectives and content are explained clearly at the outset of training.

d) A change of approach to offering bespoke facilitation, training an support will change the role of the facilitation team and may require additional skills to those currently available within the team.

30

7 Recommendations

As a result of this interim evaluation report the following recommendations are suggested for the planning of Year 2 of the project:

1) The project should develop a more local contextualised, needs led and bespoke approach to support, training and facilitation8.

2) The project should develop a consultative process that offers independent facilitation support for conversations between leaders/partners in ALCs.

3) The project should work closely with ALCs/schools to identify needs and develop appropriate training and support.

4) The project should work closely with ALCs/schools to develop more specific and targeted training and classroom support for teachers within clusters/schools.

5) All training should provide adequate time to focus on exploring strategies and working out practical solutions to the real issues and challenges identified within classrooms/schools.

6) The project should carry out a skills audit of the current facilitation team and extend the team if required to include any additional skills that may be required for this adapted approach.

7) The project should develop a resource for teachers to assist with all of key areas of collaboration that have been identified such as policies, preparation, induction, facilitation and dealing with conlfict/contentious issues in shared classrooms.

8) The project should support participating schools to connect the collaborative work of the school with other initiatives on cohesion, sharing and integration in the wider community.

9) The project should continue to prioritise how to deepen the cross-cultural learning dimension in school collaboration and in shared classrooms in line with the IFI funding priorities.

10) The project should continue to network and share learning with the other existing and emerging shared education initiatives across Northern Ireland.

11) The project should continue to consider sustainability strategies for the training and support being provided.

8 This shift in approach will require a different approach within the evaluation. The methodology will require more qualitative methods to assess impact.

31

Appendix I: Evaluation Framework

Objective 1: To increase awareness of the benefits of cross-sectoral collaboration in terms of “deepening” the learning experience for pupils involved in ALCs.

Outcomes Measures/Indicators Data Collection

There is increased awareness of the nature of the benefits of cross-sectoral collaboration by participating schools.

There is increased awareness of the nature of the benefits of cross-sectoral collaboration by participating teachers.

There is increased awareness of the nature of the benefits of cross-sectoral collaboration by participating learners.

The benefits and “deepening” of learning perceived by the learners involved.

There is an impact of the SCDL Project on relationships within and between participating schools.

The number of schools that the project helped to collaborate.

School leaders indicating increased awareness of the benefits of cross-sectoral collaboration.

Teachers indicating increased awareness of the benefits of cross-sectoral collaboration.

Learners indicating increased awareness of the benefits of cross-sectoral collaboration.

Participating learners indicating a deepening of their learning (KS3 attitudes/dispositions; respect, tolerance, community spirit)

School leaders, teachers and learners indicating a positive impact on relationships within and between participating schools.

Project participation records.

Pre and Post Participation Leaders Survey

Focus Group/Interviews

Pre and Post Participation Teachers Survey


Participant Journals

Pre and Post Participation Learners Survey

Young people’s research

Pre and Post Participation Learners Survey


Pre and Post Participation Leaders/Teachers/Learners Surveys


Objective 2: To break down barriers and develop trust between educators on a cross-sectoral/cross-community basis.


There is an increase in cross sectoral/cross community barriers between educators being identified.

There is an increase in strategies for removing cross sectoral/cross community barriers between educators being implemented.

There is an increase in cross sectoral/cross community barriers between educators being broken down.

There is an increase in trust between educators on a cross-sectoral/cross-community basis.

There is an increase the visible outworkings of trust between participating teachers.

Participating teachers have an increase in the attitudes, knowledge and skills required to build trust, break down barriers and deepen learning through collaboration.

The number of teachers involved in cross-sectoral collaboration.

Participating leaders and teachers identifying barriers.

Participating leaders and teachers implementing strategies for removing barriers.

Participating leaders and teachers reporting barriers being broken down.

Participating leaders and teachers reporting an increase in trust.

Participating leaders and teachers reporting examples of the visible outworkings of trust.

Participating teachers reporting an increase in the attitudes, knowledge and skills required to build trust, break down barriers and deepen learning through collaboration.

Project participation records.

Pre and Post Participation Leaders/Teachers Survey


Pre and Post Participation Leaders/Teachers Surveys



Pre and Post Participation Leaders/Teachers Surveys








33

Objective 3: To widely disseminate a model for “integrating” pupils taking part in shared classrooms through ALCs.


A model for “integrating” pupils taking part in shared classrooms is developed through the work of the project.

The model developed is widely disseminated through a range of publicity/communications.

Evidence of a model with characteristics that deepen learning.

The number of schools and teachers who receive information.

The range and number of publicity/communication methods used.

Baseline Desk Research/Literature Review

Project records


Project records

Project records

34

Objective 4: To create mechanisms to ensure the sustainability of cross sectoral shared classrooms and relationship building at a deeper level.


Fit for purpose mechanism/s created.

The mechanisms created are sustainable

There is an improvement in relationship building at a deeper level.

Clear mechanism/s in place at a range of appropriate levels.

There is a commitment by schools to use the mechanisms beyond the lifetime of the project.

The number of teachers who opt for a more intensive training programme to become teacher trainers.

There is a commitment by teachers to participate in the mechanisms beyond the lifetime of the project.

School leaders, teachers and learners indicating an improvement in relationship building at a deeper level.

Project records.

Pre and Post Participation Leaders Surveys






35

Appendix II: Online Learner Survey

37

38

39

40

Appendix III: Training Course Evaluation Form CommentsDay 1 Comments

41

42

43

44

45

46

47

48

49

50

51

Date post:	09-Apr-2018
Category:	Documents
Upload:	lamdang
View:	212 times
Download:	0 times

Interim Evaluation Report - Integrated Classroom...

Documents