ArtFULL – finding and using evidence of learning

Centre for Education and Industry

University of Warwick

Learning Outcomes for Pupils

(The DfES criteria)• fulfilment and satisfaction from achievement

• increased learning within the subject area

• increased understanding of connections between subjects

• increased learning across subjects

• increased self-confidence and self-esteem

• increased cultural understanding and respect and tolerance of others

• increased ability to work with others

• increased involvement in class, school community events

• the ability to make informed choices within and beyond the planned experiences

• positive attitudes to the experience and a desire for further experiences

DfES Objectives for teachers

• increased confidence

• increased expertise

• increased professional satisfaction

DfES Access Objective

• Ensure that more pupils and, where relevant, teachers, benefit from access to objects and images from museums and galleries during the school day

Scale of MGEP2

• 130 projects – 16 galleries• 165 museums and galleries involved• Around 20, 000 pupils participating – 880 in

galleries• £1 million • 9 regional museum, library and archive

agencies plus engage• Timeframe: October 2002 to March 2004

Methodology

• Initial Project Description, Quarterly & Final Report from 130 projects • Project Portfolio: pupil responses and resources (61)• Simple exit questionnaire for pupils (4093 & 1436) (472 in galleries)• Selected entry questionnaires (2745) (235 in galleries)• Exit questionnaire for teachers, museum staff, assistants (536) (59 in

galleries)• Observations(71), group interviews (59), teacher and staff interviews as

appropriate for 51 projects (85) (7 of which were galleries)• Entry/Exit Interviews with regional co-ordinators, observation of training

and networking sessions (129)

Evidence of Learning

Teacher assessment of student work: quality of work, before & afterComparison with non-project classes: teacher and pupil judgmentsObservation of activity: response to different learning styles,

independence New concepts, skills, knowledge: greater complexity/challengeObservation and recording of participation: inclusion, concentrationDevelopmental work: progress, group self-evaluationPupil work or response: how personal is response, how complete?Notes, sketchbooks, drafts: creative and original work, developmentResponse or requests for further opportunities: behaviour, decisionsPupils’ judgement and feedback: attitudes, satisfaction

• What and how much was learned?

• How did they learn it?

• Who and how many learned it?

Analysis

• learning gain

• objectives and outcomes

• associations between outcomes

• galleries vs. museums

• character of gallery learning

• explanation of sustainability

• There was substantial aggregate impact on subject learning

– 64% of entry/exit respondents experienced “learning gain”

– 70% of teachers judged that pupils had learnt a lot about the specified topic

A1

don't knownot muchsomethinglots

120

100

80

60

40

20

0

B4

don't know

not much

something

a lot

Chart 3.31: Percentages of respondents reporting the extent of learning about “subject” at exit grouped in terms of extent of knowledge at entry. (n=2617). (A1*B4 Cross-tabulation)

The Relative Impact of a Gallery Education Programme Base: MGEP2 – 2923, ArtFULL – 235/379

0

10

20

30

40

50

60

ArtFULL All MGEP2

Programme

%

Knew a lot - entry

Learnt a Lot - exit

• Activities were enjoyed

– 61% of all pupils enjoyed activity very much (65% for ArtFULL)

– 73% of those who reported highest learning also reported highest enjoyment

• Working with adults

– Overall pupils judged relationships with other pupils and adults on projects positively but similar to normal

– Teachers judged as more positive

– Some groups of pupils reported relative progress

– Projects involving artists/performers working intensively with pupils were relatively successful at improving experience of “learning with adults” – 59% of these projects showed a learning gain in this area as against 51% for all projects.

• Fulfilment– 91% of all pupils were either very pleased or content with the

work they did themselves– Across MGEP2 fulfilment levels did not show significant value

added– Some identified value added for those least satisfied with their

own work at school

A3

don't knowdisappointedokpleased

120

100

80

60

40

20

0

B3

don't know

disappointed

ok

Very Pleased

Chart 3.53 Percentages of respondents expressing satisfaction with their own work at exit grouped in terms of how satisfied they were with their own school work at entry (n=2630)

Impact of Programmes on FulfilmentBase: MGEP2 – 2630, ArtFULL – 235/379

0

10

20

30

40

50

60

ArtFULL All MGEP2

Programme

%

Usually very pleased withschool work - Entry

Very pleased withachievement -Exit

• Confidence

– 58% of pupils reported a gain in confidence (67% for ArtFULL)

– 77% teachers reported some or general gain for pupils

– Some gains in confidence for least confident

A4

don't know

not very confident

fairly confident

very confident

120

100

80

60

40

20

0

B6

don't know

unchanged

less confident

more confident

Chart 3.42: Percentages of respondents reporting changes in confidence at exit grouped in terms of how confident they were at entry (n=2609)

• Choices

– Experience of choice relates to satisfaction

B11

not sure

hardly any choice

sometimes choice

often make choices

120

100

80

60

40

20

0

B3

don't know

disappointed

ok

pleased

Chart 3.108 Percentages of Key Stage 2/3/4 pupils expressing satisfaction with their own work in the project grouped in terms of their experiences of making choices on their project at exit (n=3755) (B3 * B11 Cross-tabulation)

Opportunities for Making Choices Base: MGEP2 – 2609/3937, ArtFULL – 235/379

0

10

20

30

40

50

60

ArtFULL All MGEP2

Programme

%

Often make choices inschool work - entry

Made choices in project -exit

Outcomes for Teachers

• 94% of teachers reported a gain of satisfaction• 84% of teachers reported an increase in expertise in using

this kind of project• Teachers contributed to design and development of projects• Teachers valued collaboration with other teachers and other

professionals• Teachers valued projects which made curriculum more

relevant for their pupils (and for them)• No general difference between museum and gallery projects• Rare for projects to have wider impact on schools

Characteristics of Museum and Gallery Learning

• Situated learning/immersion in context

• Distinctive social context for learning

• Use of performance

• Enhanced opportunities for creativity

• Relatively long blocks of time and an extended treatment of a particular topic

• Cross curricular learning

• Learning in special places

• Object handling and interpretation

Sustainability

– Meeting a curriculum need– Existence of ready schools market– Cost and future funding– Charging– Flexibility of project– Professional development– Embedding in school curriculum– Quality of marketing

Issues for Evaluators

– Cost and time to collect evidence

– Reconciliation of project and programme evaluation

– Involvement of schools – use of assessment data

– Usefulness of evaluation – scope for more informed discussion about what various stakeholders want to know from evaluation and therefore about what kinds of evaluation should be gathered and analysed

– Collaborative evaluation