Using CEM Data for Self-Evaluation and Improvement

Running Your School on Data

7th June 2011

Peter.Hendry@cem.dur.ac.uk

Running your school on data

CEM data includes:

• School profile bands• Baseline test acquired ability data, inc. IPRs• Predictive data inc. chances graphs• Value-added data• Attitudinal data• Curriculum assessments (Insight)• Software programmes (PARIS) with databases

So much data................................!

1. What data is needed? (What data is not needed?)

2. And for Whom is the data ?

3. What does the data mean? (What does the data not mean?)

Some key questions:

1. What data do I need? A key first step!

e.g. using MidYIS GCSE predictive data for target setting

• Point and/or grade predictions? (e.g. 5.8/grade B)• Based on whole cohort?• Based on the 75th percentile?• Based on prior value-added?• The chances graphs?

Individual Chances Graph for ADAM BECKSMITH - GCSE EnglishMidYIS Score 121 MidYIS Band A

Teacher's Adjustment : 0 grades / levels / points

0 0 0 0

3

17

34 34

12

0

5

10

15

20

25

30

35

40

U G F E D C B A A*

Grade

Per

cen

t

Prediction 6.3 A/B

MidYIS Score 121 MidYIS Band A Prior VA Adjustment : 1 grades / levels

0 0 0 0 0

4

16

3941

0

5

10

15

20

25

30

35

40

45

50

U G F E D C B A A*

Grade

Per

cen

t

Prediction 7.4 A*/A

Discussion: setting the targets...........

• From your perspective, assess the merit of each type of predictive data and the associated chances graphs

• Which type of predictive data would you use to set the targets, and why?

• Would your students be involved as part of the target setting process?

• Would parents be informed about the process and outcome?

• How would you ensure that HoDs were involved to ensure overview the process?

2. CEM data: who is it for? For example:

• SMT/SLT: summary data, attitudinal• HoDs: Subject VAD, predictive data• Subject teachers:• FormTutors:• Head of Learning/Year/House:• Parents:• Pupils:

and not forgetting:• Governors:• Media:

3. What does the data mean?

e.g. value-added data:

• The difference between raw and standardised residuals

• The use of confidence limits to distinguish between average and statistically significant data, and to allow for small entry subjects

• Can ‘zero’ or even ‘negative value-added’ be acceptable?

Raw residual Bar chart

(MIDYIS and YELLIS only)

Standardised Residual Bar Chart

0.0

-0.2

0.3

-0.1 -0.2 -0.2

0.1 0.1

0.9

0.00.1

-0.3

0.3

-0.3

0.2 0.2 0.2 0.1

-0.3

0.1 0.1 0.20.0

-2.5

-2.0

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

2.0

2.5

Art

& D

esig

n

Bio

logy

Bus

ines

s S

tudi

es

Che

mis

try

Des

ign

& T

echn

olog

y

Dra

ma

Eng

lish

Eng

lish

Lite

ratu

re

Fre

nch

Geo

grap

hy

Ger

man

His

tory

Hom

e E

cono

mic

s

ICT

Mat

hem

atic

s

Mus

ic

Phy

sica

l Edu

catio

n

Phy

sics

Rel

igio

us S

tudi

es

Wel

sh

Sci

ence

: O

ther

Sci

ence

: G

CS

E

Sci

ence

: G

CS

E A

dditi

onal

SC

IC

T

SC

Phy

sica

l Edu

catio

n

SC

Rel

igio

us S

tudi

es

Voc

Hea

lth &

Soc

ial C

are

Av

era

ge

Sta

nd

ard

ise

d R

es

idu

al

99.7% confidence limit

95% confidence limit

ANY VALUE IN THE INNER SHADED AREA IS CONSIDERED TO BE AVERAGE VALUE ADDED

Raw residual Bar chart

(MIDYIS and YELLIS only)

Standardised Residual Bar Chart

0.0

-0.2

0.3

-0.1 -0.2 -0.2

0.1 0.1

0.9

0.00.1

-0.3

0.3

-0.3

0.2 0.2 0.2 0.1

-0.3

0.1 0.1 0.20.0

-2.5

-2.0

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

2.0

2.5

Art

& D

esig

n

Bio

logy

Bus

ines

s S

tudi

es

Che

mis

try

Des

ign

& T

echn

olog

y

Dra

ma

Eng

lish

Eng

lish

Lite

ratu

re

Fre

nch

Geo

grap

hy

Ger

man

His

tory

Hom

e E

cono

mic

s

ICT

Mat

hem

atic

s

Mus

ic

Phy

sica

l Edu

catio

n

Phy

sics

Rel

igio

us S

tudi

es

Wel

sh

Sci

ence

: O

ther

Sci

ence

: G

CS

E

Sci

ence

: G

CS

E A

dditi

onal

SC

IC

T

SC

Phy

sica

l Edu

catio

n

SC

Rel

igio

us S

tudi

es

Voc

Hea

lth &

Soc

ial C

are

Av

era

ge

Sta

nd

ard

ise

d R

es

idu

al

99.7% confidence limit

95% confidence limit

ANY VALUE IN THE INNER SHADED AREA IS CONSIDERED TO BE AVERAGE VALUE ADDED

Standardised Residual Bar Chart

Average Standardised Residuals by SubjectMidYIS Year 7 2004/2005 to GCSE 2009

0.0 0.2

-0.1

0.8

0.4 0.30.0 0.0

0.5 0.50.2

0.9

-0.9

0.0 0.1

0.6 0.50.6

0.3 0.30.3

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

Additional S

cience

Art &

Design

Bus. &

Com

m.

System

s

Business S

tudies

Design &

Technology

Dram

a

English

English Literature

French

Geography

Germ

an

History

Hom

e Econom

ics

Latin

Mathem

atics

Music

Physical E

ducation

Religious S

tudies

Science

Spanish

Statistics

Ave

rag

e S

tan

dar

dis

ed R

esid

ual

Case study: Alis value-added dataFour sets of VAD are available!

From average GCSE baseline:• all Alis cohort • type of Institution• Syllabus

From the baseline test• all Alis cohort

SPC Chart with confidence limits: WHOLE SCHOOL

All Alis Cohort: Institution

Syllabus

-0.3

-0.2

-0.1

-0.0

0.1

0.2

0.3

0.420

04

2005

2006

2007

2008

2009

2010

Ave

rage

Sta

ndar

dise

d R

esid

ual

Exam Year

-0.3

-0.2

-0.1

-0.0

0.1

0.2

0.3

0.4Final_Result

Using PARIS software: Baseline Test

Whole School

From your perspective, which set of VAD would you use for the different user groups? (Governors, HoDs, Media, Parents, SMT/SLT...)

0.8

0.5

0.5

0.7

0.4

0.6

0.2

-0.1

0.4

1.2

-0.2

0.5

0.3

0.3

0.8

0.2

-0.7

0.7

0.6

0.7

0.5

-3.0 -2.5 -2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0 2.5 3.0

Spanish

Science

Religious Studies

Physics

Physical Education

Music

Mathematics

Latin

History

German

Geography

French

English Literature

English

Drama

Design & Technology

Classical Civilisation

Chemistry

Biology

Art & Design

Additional Science

Average Raw Residual

A final thought..........

Is it possible to keep adding value at each key stage?

0.5 0.6 0.6

-0.6

0.2

0.6

0.3 0.3 0.3

-0.2

0.9

0.3

-0.1

0.2

0.50.3

0.6

0.30.5 0.6

0.4

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

Additional S

cience

Art &

Design

Biology

Chem

istry

Classical

Civilisation

Design &

Technology

Dram

a

English

English Literature

French

Geography

Germ

an

History

Latin

Mathem

atics

Music

Physical E

ducation

Physics

Religious S

tudies

Science

Spanish

Ave

rag

e S

tan

dar

dis

ed R

esid

ual

2009 A2 2010 A2

2009

2010

• USE ONE YEARS DATA WITH CAUTION!

• Better to use three years data as patterns over time are more significant.

• Teachers to help learners• Curriculum and staffing decisions• Target setting and monitoring pupil progress• Inspection evidence• Self-evaluation• Monitoring changes over time such as pupil

ability intake profiles and VAD• Asking the question ‘can we do better?’

i.e. the data is used to aid and to support professional judgement

CEM data is used for:

with due consideration to:

• Ethos and tradition of ‘my’ institution• Accountability • Parental expectations• Staff training in use of, and ability to cope with, data (data

overload)• Management of data: integrating the data into school

procedures, storage, retrieval, distribution and access……policy?

i.e. doing our best to help every pupil to at least achieve, if not exceed, their potential