UNAWE evaluation
Session aims
To explain the UNAWE Evaluation approach
To show the UNAWE Evaluation handbook materials
To show examples of UNAWE evaluation evidence
Grace [email protected]
UNAWE evaluation
UNAWE evaluation
1.3 To learn about evaluation research and examples-Research about evaluation and examples (GK)
Programmes have the same issues
“Furthermore, due to budget and time constraints,the evaluation of initiatives is often
limited. The interconnections between initiativesare very rare which effectively eliminatesthe possibilities for scaling-up and for the
disseminationof the new ideas: the dynamics of
“economy of scale” and the huge potential forreal impact are simply not being exploited”.
http://ec.europa.eu/research/science-society/document_library/pdf_06/report-rocard-on-science-education_en.pdf
1.3 To learn about evaluation research and examples-Research about evaluation and examples (GK)
local human agency perspective
Integration with structural perspectiveCategorisation for communication
PerspectivesCriteria Ontological perspective:
StructureHuman Agency
Function Control, supervision, accountability
Learning, understanding
Goal Standardisation, universality Variance, difference, diversity and peculiarity
Frame Structural/ macro perspective
Diagnostic i.e. pupil level
Focus Products, conceptual definitions
Processes, local meanings
Benefit Sorting, accountability Strengthening, autonomy
Outcomes Knowledge/professionalism Strengthening/ autonomy
Methodology Scientific, quantitative, RCT Responsive, diversified
Inquiry Analytic Holistic
Locus External Internal
Levin-Rosalis et al., 2009:191
Context
ChildGroup Class
ParentTeacher
Group of parentsGroup of teachers
InterventionResource
E Learning programmeEnrichment Programme
CurriculumInformal learning organisation
Cluster of schoolsNetwork of (in)formal learning organisations
Regional policyNational Policy
International Policy
Structural perspective
Human agency perspective
Evaluation strategy: triangulation
Teacher surveys for many; mixture of quantitative and qualitative information
Level of insight
Number of people
Pupil drawings/ surveysQualitative information analysed quantitatively
Observations/ interviewsQualitative information
analysed quantitatively and reported both qualitatively
and quantitatively
Journey: UNAWE evaluation materials
Project initiation:Project goals
March 2012:Collaborative goals
Aug 2012:Summative
evaluation materials produced
Nov 2012:NPMs amend
framework
March 2013:UNAWE Evaluation materials published
Journey: UNAWE evaluation materials
Journey: UNAWE evaluation materials
National Science Foundation
Basis: UNAWE evaluation materials
Generic Learning Outcomes
Basis: UNAWE evaluation materials
Basis: UNAWE evaluation materials
Domains of active learning
Objectives Evidence
CuriosityTenacityEnjoymentInspiration
-Children are doing the tasks with pleasure-Children seem enchanted-Children react with diligence in front of the proposed activities-Children demonstrate attention-Children apply perseverance / tenacity-Children manifest inquisitiveness-Children introduce some complex questions
Motivation
Objectives EvidenceDevelop Scientific Thinking & Problem Solving Techniques; Planning & Conducting Investigations Observation, Identification, Classification, Making interconnections, Changing Perspective & Communication Discussing and Questioning Planning Observing InterpretingIdeas and EvidenceRecordingEvaluation
-Correct use of vocabulary or gesture to name objects and phenomena observed in sky-Grouping objects/ phenomena to indicate developing understanding of astronomy concepts-Making conjectures available to be contrasted-Developing some experiences related to the hypothesis-Linking new information with existing conceptions of the same or different areas-Removing previous points of view according to new inputs-Sharing with others their new knowledge
Domains of active learning
Scientific skills
Domains of active learning
Awareness of the UniverseObjectives EvidenceObserving, exploring and discovering:1. The Sun, Sun relative position, Sun light (shadows), Day/night cycle, time zones, the seasons, the Moon, the Earth as a planet, awareness of the existence of water and of the Earth atmosphere and Sun light for the development of life on Earth, Solar and Moon eclipses,... 2. The Solar system: planets characteristics and movements, wharf planets, asteroids, comets,... 3. The Stars in the night sky, the constellations, orientation, the Life-cycle of stars, the formation of stars and planets,... 4. Our place in the Milky Way, Family of galaxies,... 5. Current developments in astronomy6. Magnetic fields (compass, northern lights,...)7. Several complementary questions
Direct observation and/or recording of naming, first explanations, discussing, drawing, construction, creative responses, movements and dances, etc. to demonstrate knowledge of one of the features listed. left
Objectives Evidence
Valuing different cultural perspective. Recognising different physical perspectives. Positive attitude towards astronomy. Valuing inclusive education Working Individually & in Teams
-Demonstrating awareness of different cultures-Ability to observe and explain differences in phenomena in different countries-Statements of future activity with regards to astronomy-Act on an appropriate way in a frame of diversity
Domains of active learning
Intercultural awareness
Objectives Evidence
Disseminating knowledge to other teachersRecommendations for improvementsEmbedding new materials in curriculum
CPD sharingWritten or spoken statementsNew short, medium or long term planning
Domains of active learning
Legacy
Examples of evidence
Objectives EvidenceCuriosityTenacityEnjoymentInspiration
-Children are doing the tasks with pleasure-Children seem enchanted-Children react with diligence in front of the proposed activities-Children demonstrate attention-Children apply perseverance / tenacity-Children manifest inquisitiveness-Children introduce some complex questions
Motivation
Examples of evidence
Examples of evidence
MotivationCase study: Germany- Teacher training responses
Workshop 13a and 13b , 2012 n=78 comments
Objectives EvidenceDevelop Scientific Thinking & Problem Solving Techniques; Planning & Conducting Investigations Observation, Identification, Classification, Making interconnections, Changing Perspective & Communication Discussing and Questioning Planning Observing InterpretingIdeas and EvidenceRecordingEvaluation
-Correct use of vocabulary or gesture to name objects and phenomena observed in sky-Grouping objects/ phenomena to indicate developing understanding of astronomy concepts-Making conjectures available to be contrasted-Developing some experiences related to the hypothesis-Linking new information with existing conceptions of the same or different areas-Removing previous points of view according to new inputs-Sharing with others their new knowledge
Scientific skills
Examples of evidence
Objectives EvidenceDevelop Scientific Thinking & Problem Solving Techniques; Planning & Conducting Investigations Observation, Identification, Classification, Making interconnections, Changing Perspective & Communication Discussing and Questioning Planning Observing InterpretingIdeas and EvidenceRecordingEvaluation
-Correct use of vocabulary or gesture to name objects and phenomena observed in sky-Grouping objects/ phenomena to indicate developing understanding of astronomy concepts-Making conjectures available to be contrasted-Developing some experiences related to the hypothesis-Linking new information with existing conceptions of the same or different areas-Removing previous points of view according to new inputs-Sharing with others their new knowledge
Scientific skills: groups
Examples of evidence
Examples of evidence
Scientific skillsCase study: The Netherlands- Teacher training responses March 2013 n=65 comments
Did you develop any new skills or practice existing skills?
Teacher quotes; many focussed on pedagogical approach as a skill they had acquired:
“new skills! inquiry learning and how I can use it in my classroom”
“how to explain clearly to children using visuals”
“the usefulness of discovery learning”
Examples of evidence
Communication skills: teachersCase study: The Netherlands- Teacher training responses March 2013 n=65 comments
Did you develop any new skills or practice existing skills?
Teacher quotes; many focussed on pedagogical approach as a skill they had acquired:
“new skills! inquiry learning and how I can use it in my classroom”
“how to explain clearly to children using visuals”
“the usefulness of discovery learning”
Awareness of the UniverseObjectives EvidenceObserving, exploring and discovering:1. The Sun, Sun relative position, Sun light (shadows), Day/night cycle, time zones, the seasons, the Moon, the Earth as a planet, awareness of the existence of water and of the Earth atmosphere and Sun light for the development of life on Earth, Solar and Moon eclipses,... 2. The Solar system: planets characteristics and movements, wharf planets, asteroids, comets,... 3. The Stars in the night sky, the constellations, orientation, the Life-cycle of stars, the formation of stars and planets,... 4. Our place in the Milky Way, Family of galaxies,... 5. Current developments in astronomy6. Magnetic fields (compass, northern lights,...)7. Several complementary questions
Direct observation and/or recording of naming, first explanations, discussing, drawing, construction, creative responses, movements and dances, etc. to demonstrate knowledge of one of the features listed. left
Examples of evidence
Awareness of the UniverseCase study: Spain- Teacher training responses
Improvement over time
Examples of evidence
Awareness of the Universe
Examples of evidence
Objectives Evidence Valuing different cultural perspective. Recognising different physical perspectives. Positive attitude towards astronomy. Valuing inclusive education Working Individually & in Teams
-Demonstrating awareness of different cultures-Ability to observe and explain differences in phenomena in different countries-Statements of future activity with regards to astronomy-Act on an appropriate way in a frame of diversity
Intercultural awareness
Examples of evidence
Intercultural awareness
Case study: Italian and South African exchange partnershipJuly- August 2012 n = 5 schools
Examples of evidence
Objectives Evidence
Disseminating knowledge to other teachersRecommendations for improvementsEmbedding new materials in curriculum
CPD sharingWritten or spoken statementsNew short, medium or long term planning
Examples of evidence
Legacy
Examples of evidence
Case study: United Kingdom Teacher training responses
Nov 2011 n = 48 participants
Initial Action Plan - as a result of this training I will:“Endeavour to deliver the programme to my own P6 class within our World
Around Us planning /teaching. Disseminate today’s programme to my other 3 year group colleagues in P6 and possibly to the P7 year group. Explore
resources indicated to us today and try to secure funding for those requiring purchases”.
Follow up: Have you ever had a chance to talk to your colleagues about EU-UNAWE to disseminate what you have
learned?“I disseminated some of the materials at a staff meeting. I have also
incorporated the materials into a unit called Razzle Dazzle – all based on light, planets, constellations etc and have shared it with two colleagues from
other schools who have also used it”.
Objectives Evidence
Disseminating knowledge to other teachersRecommendations for improvementsEmbedding new materials in curriculum
CPD sharingWritten or spoken statementsNew short, medium or long term planning
Domains of active learning
Legacy
Domains of active learning
Legacy
UNAWE evaluation
Review aims
To explain the UNAWE Evaluation approach
To show the UNAWE Evaluation handbook materials
To show examples of UNAWE evaluation evidence
Grace [email protected]
UNAWE evaluation
