Evaluating the national“Action Research for Physics”

programme- Follow up of impact on post-16

physics uptakeWilleke Rietdijk

Caro GarrettMarcus Grace

Science Education Research Conference, York 3 July 2012

The Action Research for Physics Programme (ARPP)• Research commissioned by the National Network of

Science Learning Centres and the Department for Education (DfE).

• Organised and managed by the nine Regional Science Learning Centres between September 2009 and February 2011.

The ARP programme followed on from the findings and

recommendations of the Girls in the Physics Classroom

(Hollins et al., 2006) and Girls into Physics (Daly et al.,

2009) projects.

The course• A model of professional development incorporating

action research, with, as its aims:

– to try out new approaches to teaching physics...

– ...which lead to an increase in young people’s engagement with the subject...

– ...and pursuit of physics post-16.

• 3 professional development days

• Main interventions in – Careers – Teaching and Learning

Main aims of the evaluation:

i) to examine the effectiveness of the programme in changing students’ attitudes to physics, and their aspirations for further studies

ii) to document and categorise the nature of effective practice across the action research case studies.

Reflections at the end of main study (2011)• Significant increase in participating students’ stated

likelihood of taking physics post-16, but monitoring of this proved problematic (and too early at that stage)

• Impact of the CPD programme on teachers’ practice came through very strongly – more confidence, more enthusiasm, improved teaching techniques

• Teachers felt students were more interested, motivated and engaged; contradictory results from student questionnaires; focus group findings were mixed

Follow-up of the ARPP evaluation

Part 1: development of 20 case studies.

Part 2: research questions:

1.What were students’ reasons for choosing physics post-16?

2.What aspects of the classes and the teaching had contributed to this decision? (Focus groups)

3.What was the 2011 physics AS uptake compared to 2009 and 2010?

4.How did students feel about Physics AS so far? (particularly related to expectations and GCSE teaching experiences)

Research Methods

Part 1, case studies booklet: Interviews with the teachers of the top 20 action research projects, to explore the details and successes of their intervention – part of the report and will return to briefly later.

Part 2:

Quantitative:

• Student questionnaire for AS Physics students (whole of AS cohort including previous Year 11 ARPP participants)

– N=147, from 6 colleges

Qualitative:

• Revisiting a sample of Year 11 ARPP participants now in Year 12

– 4 student focus group meetings across 3 SLC regions; 1 phone interview

• All teachers participating in ARPP with a Year 11 group were asked about their physics post-16 uptake numbers in 2011 as compared to 2009 and 2010.

ARPP teacher interventions used in the 6 sample schools

• Introducing electrical concepts before the maths.

• Questioning strategies:

1. no-hands up and mini whiteboard strategies;

2. using Bloom’s taxonomy, letting students come up with and answer each other’s questions.

1. Increasing the relevance of physics by bringing in anecdotes about real-life physics.

2. Increasing the amount of small group discussions in lessons.

3. A variety of techniques to increase engagement, in and outside of lessons.

FINDINGS

Student questionnaire sample (N=147)• All year 12, AS physics students

• 35% girls and 64% boys

• 31% had been part of an ARPP intervention during Year 10/11

• 80% Triple science students of which 80% A or A* grade

• 20% Additional science students: 65 % A or A* grade

• November 2011: 11% ‘still deciding’ about continuing AS Physics choice in Year 12!

• Not all students remembered correctly whether they had been part of an intervention

• Focus groups: None of the students in the focus groups could remember the teacher’s specific intervention

1. Reasons for choosing Physics AS

Why did you choose Physics AS?% whole group

% boys % girls Gender differencesignifican

t toI am interested in the subject as a whole

60.5 67.4 49.0 p=.031*

I am interested in particular parts of the subject

58.5 57.6 62.7 n/a

I think it is a good challenge 57.1 55.4 62.7 n/aI am good at it 51.0 57.6 41.2 p=.06I need it for further university studies 50.3 50.0 52.9 n/aI particularly enjoyed physics last year

40.1 46.7 29.4 p=.043*

I have always enjoyed my physics lessons

38.8 43.5 29.4 n/a

It looks good on my CV 37.4 45.7 21.6 p=.004**I need it for future jobs 36.7 38.0 37.3 n/aI had an inspiring teacher 27.9 29.3 23.5 n/aI find it easy 19.0 25.3 9.8 p=.026*I need it for a work experience placement

5.4 6.5 3.9 n/a

It goes well with mathematics (reason given in open space with ‘other reasons’)

4.1      *=p<.05; **=p<.01; ***=p<.001

Reasons for choosing physics

• “Most important” reasons for choosing physics were: an inspirational teacher, interest/enjoyment of the subject, and needing it for future studies/careers

• GCSE topics that motivated students to pursue physics at AS level: astrophysics and nuclear physics

• Girls more often indicated they had chosen physics because they were interested in parts of the subject (as opposed to the whole subject), and because they thought the subject provides a good challenge (but differences here not significant)

2. Focus group findings

• Reasons for choosing physics the same as in questionnaire responses

• Difference teacher GCSE/AS mentioned quite often – not as enthused by current teacher compared to ARPP intervention teacher, less engaged, more difficult explanations, more book work

• Teachers pivotal in decision according to quite a few students: passion and enthusiasm rubbing off; instilling confidence, making subject more enjoyable

• Simple explanations, varied ways of teaching/presenting materials, looking for ways to help digest big chunks of information were mentioned as important teaching strategies to help students understand physics and engage them.

3. School data on post-16 uptake after ARPPSchool Number opting for Physics

2009-10 2010-11 2011-12 School A

No data given none 6 boys out of 15 (all boys school).

School B 49/266 = 18.4% 

47/273 = 17.2% 55/272 = 20.2%

School C usually 120 students meet the entry requirement for AS physics

9/90 students in lower 6th = 10%

19 /100 students in lower 6th = 19%

34/100 in lower 6th = 34%

School D No data No data Of the 56 pupils in last year's cohort, 11 are now doing AS level physics

School E 2 boys 5 boys 9 boys and one girl

School F 19 29 

31

School G 18 out of 207 AS level students = 8%

36 out of 262 AS level students = 14%

30 out of 221 AS level students = 14% 

4. Feelings about AS Physics so far (open question)

• “A good challenge” (11 respondents)

• Being able to go into GCSE topics in more detail and further complexity (11)

• Topics enjoyed: Electricity/circuits (16), particle physics (12), practicals (11), quantum physics (7), the mathematics involved (5), SUVAT (4), and mechanics (4)

“Very interesting - news ideas coming out, understand more about how things work.”

 “Working in smaller classes where everyone wants to participate, unlike GCSE”

 “You get to do the in-depth side of GCSE. You learn more complex things than at GCSE.”

• Less interesting about AS physics so far: circuits and electricity (31), calculations and equations (14); materials (5); motion (5); fluids (5)

 “[I enjoy] Not a lot - it is so difficult! The concepts of motion are interesting, just too hard!”

 “[I enjoy] little at the moment. But it will get better, I have read the AS specifications/year outline.”

 “Old content.’” “Repetitive, feels unimportant” “Same as GCSE just more work.”

• Focus groups: corresponding findings.

Impact of ARPP teacher intervention

• Statistical comparisons made:

– Answers from students who were part of ARPP compared to those not part of ARPP

• When they chose physics• Reasons for choosing physics (Topics? Teacher? Intervention-related

(indirect)?• What contributed most to their decision to choose physics AS• What they liked about physics GCSE (trying to gauge if intervention-related)

• No statistical differences in any questionnaire item between ARPP and non-ARPP students – so statistically no clear impact of the programme on reasons for choosing physics post-16

• No specific impact of the type of interventions could be found in terms of statistical differences in motivations to choose physics, or answers to the open questions.

• HOWEVER: students did not seem to remember very well whether they had been part of an intervention, and did not seem to remember at all what this intervention was if they did

Conclusions

• Some schools report a distinct increase in Physics AS numbers in the year after the ARP programme

• Difficult to find impact of ARPP intervention on students’ reasons for choosing physics from students’ responses; BUT

• general feedback from focus groups shows that teachers are very important in students’ choices for AS physics (passion, enthusiasm, engaging teaching techniques, instilling confidence)

Indirect impact?

ARPP teacher participation

greater teacher confidence & enthusiasm about physics / better teaching strategies

students more enthused/motivated/confident?

more students opt for Physics AS?

Limitations

• Small sample (147 students from only 6 schools/ intervention studies)

– Findings difficult to generalise

– Not easy to find statistical impact from a small sample

• Self-selection of teachers

Beyond the programme and evaluation• Bearing in mind the effect of teacher effects on

student decisions, as in the conclusions:

Impact of the programme on teachers’ CPD

Impact of the teachers’ CPD on their colleagues

Impact on the teachers’ practice and sustainability

Potential for effective teacher development via collaboration between Teaching Schools and Science Learning Centres……… how long is a piece of string?

Beyond the programme and evaluation• Bearing in mind the effect of teachers effects on

student decisions, as in the conclusions:

Impact of the programme on teachers’ CPD

Impact of the teachers’ CPD on their colleagues

Impact on the teachers’ practice and sustainability

Beyond the programme and evaluation• Bearing in mind the effect of teachers effects on

student decisions, as in the conclusions:

Impact of the programme on teachers’ CPD

Impact of the teachers’ CPD on their colleagues

Impact on the teachers’ practice and sustainability

Finally…

• A case studies booklet is currently being compiled for publication – 20 most successful action research studies

• Final (main) ARPP report is available from the NSLC website: https://www.sciencelearningcentres.org.uk/research-and-impact/research-reports