Date post: | 08-Jul-2015 |
Category: |
Education |
Upload: | centre-for-distance-education |
View: | 368 times |
Download: | 0 times |
Feedback as dialogue and learning technologies: can e-assessment be
formative?
Stylianos Hatzipanagos
King's College London
April 2010
The seminar will:
Put forward a conceptual model of formative assessment and examine how this can be made to work purposefully within the specific constraints of ODL;
Explore the relationship between formative assessment and learning technologies and the opportunities for computer mediated communication and dialogue in these environments.
Closing the loop: two projects
explored policy, teaching practices and tutor and student views in three Open and Distance Learning environments.
investigated relationship between formative assessment and learning technologies
Projects team
♣ Paul Black (King’s)
♣ Stylianos Hatzipanagos (King’s)
♣ Ana Lucena (IOE)
♣ Bob McCormick (OU)
♣ Steven Warburton (King’s)
Types of assessment
summative (SA)
numerical results about student performance
generally carried out at the end of a course/programme
formative (FA)
a continuous/periodic appraisal of students’ performance
may inform and improve competence and contribute to further learning
(Sadler, 1989; Boston, 2002; Black et al., 2003; York, 2004)
Formative or assessment for learning
(Albon, 2003; Wiliam et al., 2004; Nicol & MacFarlane-Dick, 2004; Black, 2005)
Duality “SA/FA” may not represent opposite poles of assessment
(Hargreaves, 2005; Dylan, 2006)
FA is SA with feedback, which can be used by the learner (Taras 2005)
positive implications for student learning
allows students to play a more active role in management of own learning (Nicol 1997)
Purposes of assessment types(Hatzipanagos & Warburton 2009, adapted from Bull & McKenna 2004)
Type of
assessment
Feedback to
student
Feedback to
tutor
Self testing Determining
level/progression
diagnostic 2 2 1
formative 1 2 1 2
summative 2 1
Primary purpose: 1, Secondary purpose: 2
Dimensions of FA
Power (autonomy and ownership)
Dialogue
Peer/self assessment
Feedback (attributes)
Visibility
Reflection
Action
Black et al. (2003), Gibbs & Simpson (2004),
Giroux (1992), McConnell (2006).
Feedback Comments – what messages are being conveyed?
! ‘be more critical’
‘you’re argument needs to be more academic’
X ‘More use of analysis would have helped’
‘requires further elaboration’
‘you need to think carefully about your structure’
‘You write in a journalistic style’
?Medland 2009
FA and technologiesLearning technologies promote innovative assessment practices and lead to deeper thinking about how tutors conceptualise assessment in higher education (McCormick 2004).
assessment practices have been supported by technology for many years. However…
….main focus on developing tools such as objective tests rather than addressing fundamental issues, such as how they can be used to support effective assessment approaches (Nicol and Milligan, 2006).
Characteristics of FA technologies: potential
Can be used as living record of student learning
Blur boundaries between formal and informal learning spaces
Challenge dominant model of hierarchical, tutor centred education
Embrace an ideology of openness, dialogue and ownership.
Open & Distance Learning
ODL environments : necessity for FA practices. proactive in FA practices out of need to provide
systematic feedback to students.
e-literacy in pedagogical terms
Required set of competencies for teaching practitioners is more complex and consists of:
Updated knowledge of e-learning resources and their role in enriching the learner experience.
Expertise of range of tools and of the pedagogical affordances that each provides
Ability to evaluate technologies and engage in reflection on how they impact on learning
Staff Perspective‘...part of the feedback given to pupils in class is like so
many bottles thrown out to sea. No one can be sure that the message they contain will one day find a
receiver’
(Perrenoud, 1998: 87)
Student Perspective
Dearing Report (1997)
National Student Survey (2004-present)
Methodology Open-ended interviews with 17 tutors and 20 students to
explore perceptions of assessment.
Informed content of online questionnaire that was administered to students within the three ODL environments.
In the online survey, students were asked to express their level of agreement to a number of statements about a five-point Likert scale.
Qualitative data using also open-ended questions in the questionnaire.
Institutional differences
Huge diversity in practices in all three of the environments.
From the three environments, two were broadly similar, and they were characterised by consistent elements of good practice.
One of the three environments had in place an infrastructure to provide more systematic provision of feedback. In addition, there was in place a framework, which emphasised periodic assessment rather than end of year assessments.
Institutional cultures and the nature of an institution determine assessment practices:
King’s ODL and the External programmes broadly similar.
External programmes : there is an indication of huge diversity in practices.
Good practice in policy : policy documents that set the frameworks within which the tutors will operate
and increased use of e-assessment. Infrastructure to provide more systematic provision of feedback. Framework, which emphasises periodic assessment rather than end of year
assessments. Particularly at the OU tutors are continuously monitored about their assessment
practices.
Tutor engagement
Disjunction between beliefs, ambitions and pragmatic approaches to the use of FA. Practitioners’ attitudes to FA context-dependent and sometimes discipline-oriented.
Classification of approaches in two substantial groups and one smaller:
The first two claimed that they were proponents of FA; they either used FA in their practices (group 1) or claimed that they did not, however they would consider it if pragmatic constraints allowed it (group 2)Third smaller group did not consider FA necessary for their context.
Significant number of tutors engaged in FA but not extensively in what we consider FA.
Notion of FA varied e.g. often was equated to ‘continuous
assessment’.
Formats of assessment have changed because of the
possibilities new technologies can afford.
Student engagement Target audience of institutional groups were diverse in terms
of perceptions.
Students’ attitudes to assessment were not discipline dependent. It was the broad context (the ODL environment) that determined attitudes.
Facilitation of feedback mechanisms using computer mediated communication was recognised as a significant component of the assessment process.
Majority of learners recognised the challenges in providing a suitably formative environment in these settings.
Difficulties in defining their personalised learning environment and the affordances of the tools they used.
Q31. I am interested in the marks and not in the feedback.
0
50
100
150
200
250
300
350
400
3.8% 13.4% 41.8% 41.0%
Always Often Sometimes Never
Series1
Q24. I would learn more if I received feedback
0
50
100
150
200
250
300
350
400
28.9% 27.1% 44.0%
more frequently with longer comments shortly after submission ofmy work
Series1
Q28. The feedback guides me on what I need to do to improve my work.
0
50
100
150
200
250
300
350
400
37.9% 43.5% 16.9% 1.6%
Always Often Sometimes Never
Series1
Q13. I have used a computer to receive feedback.
0
100
200
300
400
500
600
700
800
83.6% 13.2% 3.8%
Yes No Not sure
Series1
e-Assessment
The projects considered whether current formative assessment practices can cope with emerging technologies.
Learning technologies (a fundamental channel of student support)
Received favourably, though overall there was no sense of ownership or always a clear perception of purpose regarding the affordances of each technology.
e-assessment: range of technologies
Non-formative
Objective tests (non adaptive and they ‘disagree’ with certain disciplines)
Model answers received or revealed after students submit an answer, as non-personalised feedback
Electronic submission of coursework
Formative
Communication tools in VLEs Online tutorial environments Games that allow monitoring and intervention Audio to canvas opinions/understanding of concepts/issues (audio more
meaningful conceptually than video) Tools such as certainty based marking Videoconferencing Social software: Blogs, Wikis e-portfolios
Key processes of formative assessment
Traditional technologies
Social software
email discussion
board quiz blog wiki
Power (ownership and autonomy)
H L L H H
(distributed)
Dialogue
L to M H L H (informal
tone) M
Timeliness
L to M H variable H H
Visibility
L variable L H H
Action
L to M L to M M H H
Community (peer support)
L variable L H (via
aggregation) H
Reflection
variable M M H M
Relationship between social software & FA
…social dimensions of social software
allow for formative assessment practices to be re-invented or at the very least facilitated by participative and student focussed interventions.
(Hatzipanagos & Warburton 2009)
Four Stages to embed FA practices in the assessment cycleStage Zero Assessment mostly through exams and end of assessment term projects.
No provision of feedback.
Limited or no peer/self assessment opportunities.
Limited or no use of learning technologies to support assessment practices.
Stage One Generalised feedback on student work but of limited customisation to the needs of the
individual learner.
Examiner reports with model answers for monitoring/evaluating assessment practice.
Use of learning technologies to support assessment through objective tests.
Stage Two All the above in Stage One plus:
Periodic/continuous assessment for learners to rehearse arguments that they will use in end
of assessment period assessments.
Feedback is monitored, to ensure that students will act upon negotiated targets and the
feedback loop is closed.
Learner responses to feedback become an essential part of the assessment cycle.
Use of learning technologies such as computer mediated communication to facilitate the
assessment cycle.
Stage Three All the above in Stage Two plus:
Peer/self assessment.
Student involvement in setting marking criteria.
Use of learning technologies to peer review, and to construct knowledge collaboratively.
Hatzipanagos 2010
Conclusions
Towards an understanding of the significance of formative assessment in ODL.
A conceptual model of formative assessment and how this can be made to work purposefully within the specific constraints of ODL environments.
FA practices can be problematic in courses with emphasis on end of year assessments as the ‘closing the loop’ component of the assessment process very rarely takes place.
FA can enrich e-learning approaches by making the feedback central to all e-assessment activities.
e-assessment tools promote a dialogue in relation to feedback, peer and self assessment activities which by their nature place the student at the centre of the educational process.
Some questions to explore
1. Are assessment practices in HE stifling innovation?
2. How formative SA practices must be to have an impact on student learning?
3. Can self or peer assessment practices be summative or only formative?
4. How important is context on assessment practices and student performance?
5. How do we accredit learning that takes place in informal spaces afforded by learning technologies
Further work
♣ Comparison of current technology supported FA mechanisms across the ODL landscape.
♣ identify types of processes and learning technology tools that might best support effective feedback
♣ encourage effective feedback approaches that both empower and enhance the learning experience for the distance learner.