| Date post: | 02-Jan-2016 |
| Category: |
Documents |
| Upload: | gregory-dean |
| View: | 213 times |
| Download: | 0 times |
1Contact: [email protected]
From Evaluation to ResearchFrom Evaluation to Research
Description of a continuum Description of a continuum
in the field ofin the field of
Global EducationGlobal Education
2Contact: [email protected]
1. Evaluation and Research
2. From Evaluation to Research – a continuum
3. Models of Evaluation in the field of Global Education
and their relevance for Research
4. The analysis of learning effects as a challenge for
Evaluation and Research
Overview
4Contact: [email protected]
“The activity of scientific research is to establish rates or systems of propositions and to systematically review, in the empirical sciences in particular hypothesis, theory, systems which are installed and checked against experience by observation and experiment." (Popper, 1966: 3)
Research …
1. is a systemic activity (about a "system of records" or a "system of knowledge");
2. is directed towards reality (empiricism);
3. works with the help of hypotheses, theories, and other means, and
4. arrives at general statements ("systems theory" be established and verified, or "causal relationships and regularities")
Research
5Contact: [email protected]
Standards:
1) The results need to be provable.
2) The results need to be intersubjectively verifiable.
Research
6Contact: [email protected]
Evaluation is …
1. is a systemic activity
2. is directed towards reality (empiricism);
3. works with the help of pre-determined quality standards
4. arrives at project-related statements (assessment of strengths and weaknesses of a certain project reality)
Standards:
1) Utility
2) Feasibility
3) Propriety
4) Accuracy
Evaluation
7Contact: [email protected]
Evaluation and Research
Evaluation Research
Concern Valuation of practice based on verifiable criteria
Generation of knowledge about a field of practice
Focus Focus on a case Generalization as perspective
Standards Standards of carrying out an evaluation on a case
Intersubjectivity of the valuation needs to be verifiable
Sufficient data base for the generalization
Generation of knowledge needs to be intersubjectively verifiable
9Contact: [email protected]
From Evaluation to Research
Evaluation From Evaluation
to Research
Research
Concern Valuation of practice based on verifiable criteria
Using the knowledge from the evaluation without inheriting the attitude to improve
Generation of knowledge about a field of practice
Focus Focus on a case Several similarily arranged evaluations
Generalization as perspective
Standards Standards of carrying out an evaluation at a case
Intersubjectivity of the valuation needs to be verifiable
Sensitivity for the evaluation‘ s limitations
Data collection with methods that permit intersubjectivity
Sufficient data base for the generalization
Generation of knowledge needs to be intersubjectively verifiable
11Contact: [email protected]
We can distinguish three models…
1. Concept Evaluation
2. Performance Evaluation
3. Impact Evaluation
Models of Evaluation
12Contact: [email protected]
Examples:
1) Evaluating the quality of an educational programme’s concept …
2) Evaluating the quality of teaching material …
…with normatively defined quality standards/criteria
Possible aspects of concept evaluations:
- Objectives
- Content
- Educational Methodology ...
Concept Evaluation
13Contact: [email protected]
Possible questions:
• Concerning the objectives: Do the educational objectives of an activity relate adequately to the target group? Are these objectives expressed in an understandable way?
• Concerning the content: Does the content relate appropriately to the objectives / the target group? Does the chosen content and its didactic format comply with the quality criteria of Global Education?
Concept Evaluation
14Contact: [email protected]
Examples:
1) Evaluating the implementation of a Global Education project in a school
2) Evaluating the organisational structures of an institution
Possible aspects of performance evaluations:
- Project management
- Transparency
- Educational Methodology ....
Performance Evaluation
15Contact: [email protected]
Possible questions:
• Concerning the project management: Does the allocation of responsibilities correspond to the working conditions?
• Concerning the educational methodology: Is the learning material used properly / adequately for the particular target group?
Performance Evaluation
16Contact: [email protected]
Examples:
1) Evaluating the widespread impact of a certain material
2) Evaluating the impact of an activity / project on a particular target group
Possible aspects of impact evaluations:
- Spread
- Satisfaction
- Learning effects …
Impact Evaluation
17Contact: [email protected]
Possible questions:
• Concerning satisfaction: In which aspects are the stakeholders / target groups content with the performance / conduction of the project?
• Concerning the learning effects: How do the target groups assess their own attainment of knowledge? What real learning effects can be recognized after a certain project / activity?
Impact Evaluation
18Contact: [email protected]
Concept Evaluation and its relevance for Research
Concept Evaluation: comparison of concept and quality standards
Research: comparison of concept A and quality standards
comparison of concept B and quality standards
comparison of concept C and quality standards
To which extend do the practical concepts comply with the
theoretical discourse or the normative quality standards/ criteria in the field of GE/DE?
Sociological interest in the correspondance of norms of a practical field
19Contact: [email protected]
Performance Evaluation and its relevance for Research
Performance Evaluation: How is the project being carried out?
Has the performance been aim-oriented?
Research: Performance Evaluation A
Performance Evaluation B
Performance Evaluation C
Organizational Development
Project Management
Interest from the perspective of theories on organizational development and theories of project management
20Contact: [email protected]
Impact Evaluation and its relevance for Research
In the field of GE/DE 3 different types of impact can be distinguished:
1. the widespread impact through a project or activity, e.g. the number of people reached, the number of material distributed, the number and type of media coverage etc.,
2. the impact on the institutional level, e.g. the strengthening of a field of activity or the extension of projects in GE/DE within a NGO,
3. the impacts on the level of the target groups; in this case, another distinction can be made between the effects on the level of satisfaction statements, self-mentioned/reported learning and real learning effects.
21Contact: [email protected]
Impact Evaluation and its relevance for Research
Impact Evaluation: Widespread impactImpact on institutional levelSelf-identified/reported learning(Cognitive) Learning effects
Research: Impact Evaluation A
Impact Evaluation B
Impact Evaluation C
distribution/acceptanceself-reflective competenciesreal learning effects
Interest from the perspective of communication and media research
Interest from the perspective of research into teaching and learning
22Contact: [email protected]
The analysis of learning effects as a
challenge for Evaluation and
Research
23Contact: [email protected]
Analysis of learning effects as a challenge
Input
e.g. quality of material
Impact
Learning activities
e.g. active use of time
Individual preconditions
Context
Mediators
e.g. motivation
Teacher/
Trainer/
Educator
24Contact: [email protected]
Analysis of learning effects as a challenge
1) Learning is very complex.
There is no linearity of teaching and learning.
Learning (esp. change of human behaviour) isn’ t always a logical response to a certain activity.
2) Learning has different dimensions.
- Knowledge
- Competencies concerning values and judgement
- Competencies/Strategies concerning communication and action
26Contact: [email protected]
Analysis of learning effects as a challenge
Analysing the acquisition of knowledge:
27Contact: [email protected]
Analysis of learning effects as a challenge
Analysing the acquisition of competencies concerning values and judgement:
Decision and action alternatives by giving short tales concerning topics, which the participants have to weigh against each other or state reason for their choice
Focus:
To what extent does an activity create the necessary cognitive preconditions / qualifications (!) for the intended change in values and judgement to become most probable?
28Contact: [email protected]
Analysis of learning effects as a challenge
Analysing the acquisition of competencies concerning communication and action / strategies:
Scenic challenge-situations for which future solutions have to be found
Focus:
Examination of certain prevailing conditions like e.g.
- the capability / ability to predict future developments
- the capability / ability to set individual goals
- the capability / ability to shape change processes
29Contact: [email protected]
Conclusion
We need …
… to gain experience with the synopsis of evaluations which are carried out according to similar criteria.
… to complement evaluation with research by taking out small areas that can be dealt with according to the standards of research.
To do so it is necessary …
… to find more adequate methods concerning different evaluation targets
… that evaluators and researchers get into contact with each other.
