Date post: | 17-Nov-2014 |
Category: |
Education |
Upload: | centre-for-distance-education |
View: | 320 times |
Download: | 4 times |
Writing course materials for successful flexible learning
Ormond Simpson Visiting Fellow CDE
CDE seminar - 24 June 20141
Variations in OU course module retention
T302
T331
MU120
W300
K224
60
65
70
75
80
85
90
95
100
70
% getting to exam
% p
assi
ng
exa
m
40 50 60 80 90 100
2
3
A successful distance course keeps a student’s learning motivation
switched on
Course Design- possible influences on retention
1. Course Choice - how students choose their course
2. Course workload – how much is in the course
3. Course structure – how the course is organised
4. Course writing - the writing of the course
5. Course assessment strategies - how students are assessed
6. Course evaluations - how courses are assessed
4
1. Course Choice and retention
The second biggest
reason students give
for dropping out (after
time issues) is that
they were on the
wrong course.
5
2. Course workload and retention
No link between students’ reported course workload and dropout - Crooks, 2005
But that’s: Counter intuitive?Counter experience?
Possibly a methodology problem?
Or the variety of concepts and media?
6
3. Course structure and retention
‘Flexibility’
- choice of material to study
- choice of time to study
- choice of assessment
Crooks 2005
7
John Sweller ‘Cognitive
Load Theory’
John Keller ‘ARCS’ theory
John Hattie‘Self –
Reporting’
4. Course writing - theories
Maintaining a student’s motivation to learn
Keller’s ARCS theory
A = Get and keep their Attention
R = Ensure everything is Relevant to their needs
C = Ensure they have Confidence in what they’re
doing
S = And that they are continually Satisfied with their learning experience
9
Getting attention
- Empathy – eg stories sharing personal thoughts, acknowledging difficulties
Incongruity – eg humour
Readability – Flesch tests
Relevance
ConfidencePersonal, Approachable, Using ‘I’
Satisfaction Ensuring students feel progressWorked example
Ensuring relevanceAvoiding redundancy
Conclusions from theories
Keeping attention
Keller – Keeping Attention – Readability
’Flesch Reading Ease’ Score depends on sentence length and number of
syllables per word
Readability score Interpretations
0 - 20 Very difficult
20 - 50 Difficult
50 – 60 Fairly difficult
60 – 70 Plain English
70 – 80 Fairly easy
80 – 90 Easy
90 - 100 Very easy11
Open Poly of New Zealand course on communication
The field of communication studies runs wide. As a discipline, it borders on academic specialities such as linguistics, psychology, media studies, cultural studies, sociology, philosophy, marketing, and business studies. Its diverse components include interpersonal communication, intercultural communication, workplace writing, organisational studies, and mass communication. It can, at times, be difficult to limit the scope of communication studies: it seems to involve pretty much most things human beings to together. This is an indication of the obsession in modern times with communication. As Peters (1999) notes, communication has been viewed as the solution to humanity’s diverse and profound troubles. How many times, for instance, during local or international conflicts have you heard talk of communication breakdowns, or of the need to open channels of communication? In this optimistic view, communication entails connecting with others, expressing our true selves, alleviating loneliness, preventing misunderstandings and conflict. The other side of the attention to communication in the modern world is a focus on the dilemmas and apparently intractable problems of communication. Peters (1999) points to the way that so much twentieth century humour focuses on human miscommunication, where humans are left dazed and lonely – gesticulating wildly, impotent, ridiculous.
12
Open Poly of New Zealand course on communication
The field of communication studies runs wide. As a discipline, it borders on academic specialities such as linguistics, psychology, media studies, cultural studies, sociology, philosophy, marketing, and business studies. Its diverse components include interpersonal communication, intercultural communication, workplace writing, organisational studies, and mass communication. It can, at times, be difficult to limit the scope of communication studies: it seems to involve pretty much most things human beings do together. This is an indication of the obsession in modern times with communication. As Peters (1999) notes, communication has been viewed as the solution to humanity’s diverse and profound troubles. How many times, for instance, during local or international conflicts have you heard talk of communication breakdowns, or of the need to open channels of communication? In this optimistic view, communication entails connecting with others, expressing our true selves, alleviating loneliness, preventing misunderstandings and conflict. The other side of the attention to communication in the modern world is a focus on the dilemmas and apparently intractable problems of communication. Peters (1999) points to the way that so much twentieth century humour focuses on human miscommunication, where humans are left dazed and lonely – gesticulating wildly, impotent, ridiculous.
FRE = 13.5 = Very difficult
13
• The field of communication studies runs wide. As a discipline, it borders on academic specialities such as: – linguistics, – psychology, – media studies, – cultural studies, – sociology, – marketing, – business studies.
• Its diverse components include: – interpersonal communication, – intercultural communication, – workplace writing, – organisational studies, – mass communication.
• It can sometimes be difficult to limit the scope of communication studies. It seems to involve pretty much most things human beings do together. This shows the obsession today with communication.
FRE = 62 = Plain English
14
15
Evidence for Keller’s Theory?
‘Motivational measure of the instruction
compared: Instruction Based on the ARCS
Motivation Theory vs Traditional Instruction
in Blended Courses’
Ozgur M. COLAKOGLU & Omur AKDEMIR
Turkish Online Journal of Distance Education
(TOJDE) 2010
Cognitive Load Theory (Sweller, 1998)
Transfer of learningInitial learning goes into the
working (short) memory and needs to be transferred to long term memory
16
Learning Working memory
Long term
memory
Three types of cognitive load
1. Intrinsic – due to inherent difficulty of subject. Should be managed by e.g. ‘segmentation’ and using worked examples
2. Extraneous – due to way information is presented. Should be minimised by e.g. Ensuring relevance avoiding redundancy
3. Germane – due to way info’ relates to previous info’ Should be maximised by making clear links between new and old information 17
‘Assessing Cognitive Load Theory to improve Student Learning for Mechanical Engineers’ - Impelluso,V. American Journal of Distance Education 23 (4) (2009)
- claimed increased retention and learning
18
5. Course assessment strategies
‘Assessment drives learning’But…
Does assessment also drive dropout?
19
1
100
62 57 52
38
43
48
7 2
5
ASSIGNMENT RIVERGRAM100 students start the course. At each assignment some drop out and enter the ‘exit’ channel. A very few re-enter the ‘progress’ channel having skipped the previous assignment
Assignment 1
Assignment 2
Assignment 3
Progress
Exit
Evidence - what does research say about assessment and retention?
(1) Hattie (2009) most important is ‘self reporting of grades’ – so a student knows how well (s)he is doing
21
‘Self reporting of grades’
Put ‘self-assessment questions’ in text - but how to get students to do them?
1. Tell them
2. Nudging – “most students do the SAQ’s” “research has shown that students who do the SAQ’s do best in the course”
3. Electronic text – make it impossible to move on unless the SAQ is attempted
22
E-teaching- can be provided in two ways
Course texts in print.Teaching / support online
Everything online including course
texts
Reading off a screen can be 30% less effective than reading paper copy http://www.scientificamerican.com/article/reading-paper-screens/
24
E-teaching Advantages of e-teaching – can use many different kinds of media – forums, podcasts, video/audio clips, blogs etc
Disadvantages of e-teaching – can use many different kinds of media….
‘Course Exuberance Syndrome’
‘Learning value’ vs. ‘learning time needed’
Learning value poor
Learning value rich
Learning time needed short
Learning time long
Paper text
Online text
Computer forums
blogs
podcasts
Videoclips
Wikis
‘Use line’
Secondlife
‘Learning time
needed’
‘Learning value’
pencasts
Audioclips
ebooks
26
6. Course evaluations
1. Asking the students
2. Data analysis
27
Asking the students
OU Courses survey
‘Crash testing’
28
29
Module title Students started
Students passed
Predictedpass
Actual pass
Z-score
‘Creative writing’ 1,995 1,615 77.2% 81.0% + 4.38
‘Introducing religions’
355 254 62.9% 71.5% + 3.71
‘Medicine and society in Europe
1500-1930’357 227 69.6% 63.6% - 2.80
‘Exploring the classical world’
605 414 68.9% 68.4% - 0.29
‘Exploring philosophy’
699 500 65.3% 71.5% + 3.84
‘Inside music’ 413 255 74.0% 61.7% - 6.34
Z-scores for a sample of OU modules
Z-scores outside the range +4 or -4 are considered significant
Variations in course retention
T302
T331
MU120
W300
K224
60
65
70
75
80
85
90
95
100
70
% getting to exam
% p
assi
ng
exa
m
Sifters
Heavy goings
Fair
Knock Backers
40 50 60 80 90 100
30
Barriers to ‘retention-friendly’ text?
31
Material focused
ExclusiveHighly
challenging Serious
Student focusedInclusive
Aptly challengingEnjoyable
Weeding out the unfit –
‘Darwinistas’
Sink or swim - ‘Fatalistas’
Help students be the best they can
be- Retentioneer
33
4. Motivational and ungraded feedback
(i) Motivational feedback (Wigfield et al, 2009):
Learning motivation = (assumed possibility of accomplishing task)
x (perceived value of task)
If either factor is zero then motivation is zero
So feedback tasks should be carefully graded for difficulty
34
Formative assessment – the evidence
• Yorke and Longden (2004)
• Black and Wiliam (1998)
• Kluger (1996)
• Gibbs and Dunbar-Goddet (2007)
35
Without external formative assessment 1
100
62 57 52
38
43
48
7 2
5
ASSIGNMENT RIVERGRAM – FORMATIVE ASSESSMENT100 students start the course. At each assignment some drop out and enter the ‘exit’ channel. A very few re-enter the ‘progress’ channel having skipped the previous assignment
Assignment 1
Assignment 2
Assignment 3
Progress
Exit36
With external formative assessment
100
72 70 65
28
30
35
11 9
5
ASSIGNMENT RIVERGRAM – FORMATIVE ASSESSMENT100 students start the course. At each assignment some drop out and enter the ‘exit’ channel. A very few re-enter the ‘progress’ channel having skipped the previous assignment.
Assignment 1
Assignment 2
Assignment 3
Progress
Exit37
Keller’s A = Attention
1. Getting students attention2. Keeping students attention
38
Keller - Getting Attention
Use:- incongruity – eg humour*- empathy – eg stories, sharing
personal thoughts - authority – displaying expertise
But everything in a course text should be relevant to learning.
*eg see ‘Flipnosis’ – Dutton (2011)
39
Keller - Keeping attention - Reading skills and reading habits
• Researchers used ‘Cloze’ tests (replacing blanks in text) on new students – many would have significant difficulties in understanding their course material.
- 42% new students had lower comprehension than needed for courses - Students’ previous reading was newspapers and magazines
- Datta and Macdonald Ross
(2002)
40
Readability Scales
• Automated Readability Index
• Flesch Reading Ease
• Flesch-Kincaid scores
• Gunning-Fog index
• SMOG index (simplified measure of gobbledygook)
41
Newspapers Flesch Reading Ease score
Sun 62.8 – Plain English
Daily Mail 61.5 – Plain English
Mirror 60.5 – Plain English
Guardian 44.5 – Difficult
Telegraph 48.8 – Difficult
42
OU coursesinitial pages
Flesch Reading Ease score
Arts Foundation 47.1 – difficult
Social science Foundation
55.2 – Fairly difficult
Maths Foundation 39.9 - difficult
Science Foundation
53.7 – Fairly difficult
OU Access Course 58.1 – Fairly difficult
Significant differences in readability levels between tabloid newspapers and UKOU courses - Moore (2004)
43
Fonts
• Times New Roman• Arial • Helvetica• Courier • Comic • Calibri• Cambria• Brush • Kumasi Pioneer
Two groups of students were given exercise advice in Arial and Brush fonts.
Students getting Arial text were more likely to change exercise behaviour than students getting the Brush text
Herbert - Sci. Amer. Mind Feb 2009
44
Type layout - Justifying 1
There appears to have been relatively little work done on course design and retention. Indeed designing a course for better retention rates is a complex concept since course design is intimately bound up with course assessment. In recent years in the UK the annual festival of punditry revolves around the question of whether more students passing the ‘A’ level exams means that ‘standards’ have therefore fallen. Thus the retention debate has tended to avoid questions of the difficulty of content and concepts in the course and concentrated on other possibilities for increasing retention such as course workload, course readability and course design and structure.
45
Justifying 2
There appears to have been relatively little work done on course design and retention. Indeed designing a course for better retention rates is a complex concept since course design is intimately bound up with course assessment. In recent years in the UK an annual festival of punditry revolves around the question of whether more students passing the ‘A’ level exams means that ‘standards’ have therefore fallen. Thus the retention debate has tended to avoid questions of the difficulty of content and concepts in the course and concentrated on other possibilities for increasing retention such as course workload, course readability and course design and structure.
46
Using columns
There appears to have been relatively little work done on course design and retention. Indeed designing a course for better retention rates is a complex concept since course design is intimately bound up with course assessment.
In recent years in the UK an annual festival of punditry revolves around the
question of whether more students passing the ‘A’ level exams means that ‘standards’ have therefore fallen.
Thus the retention debate has tended to avoid questions of the difficulty of content and concepts in the course and concentrated on other possibilities for increasing retention such as course
47
Newspapers
o Different fontso Narrow columnso Short paragraphso Justified – sometimes ragged righto Many graphicso Frequent subheadings and quote
boxeso Variable fontso Colour and shading - all aimed at ‘Attention’
48
R = Relevance
• Avoid redundancy•
Avoid ‘split attention’
• - see ‘Cognitive Load Theory’
49
C = Confidence
• Why are texts largely anonymous?- could we use ‘I’ or ‘we’?
• Why little information about authors?– should we ‘personalise’ texts?
50
S = Satisfaction
Feedback to student - Hattie - most important is ‘self reporting’ – so a student knows how well (s)he is doing’
So use self assessment questions?
• put answers straight after questions
• keep questions short so students can see progress 51
Types of assessment
1. Formative - to enhance learning, not to allocate grades - usually ungraded - feedback for student’s benefit 2. Summative – graded - counts towards final grade
- continuous or final exam
3. Ipsative - iterative feedback
52
‘Meta-cognitive awareness’
(3) Gibbs (2010) – marking exercises - enhanced worked examples
Some indirect evidence for increased retention
53
Gibbs – (1) Marking exercises
Students given specimen student assignment answers and asked to mark them
54
55
Graham Gibbs - Marking exerciseF2F GROUP ACTIVITY 2: What do 'they' want? (25
mins)
Give students copies of a short student assignment.
1. Tell them, “Imagine you are a tutor - read this quickly and mark it. Give it a grade (100 to 0) and helpful comments." (5 mins)
2. In pairs, “Compare your comments and grading together. What have you picked out as important in your comments?“ (10 mins)
3. In plenary, “Right let's now try to draw out some good and bad points from this to help us in our own writing". (10 mins)
Gibbs - (2) Enhanced worked examples
“Last week I found 50 New Zealand dollars in my suitcase.
“So I rushed down to the Post Office where there was one of those illuminated signs in the window. Against NZ$ it said ‘We buy at NZ$2.72 to the £’.
So how much did the man behind the counter give me?”
56
Enhanced worked example - answer
“The statement ‘We buy at NZ$2.72 to the £’ means that for every NZ$2.72 I give the man he’ll give me £1.
I've got NZ$50 so we need to know how many times 2.72 goes into 50 to see how many £’s I'll get.
Now 50/2.72 = 18.51851852 on my calculator. So the man gave me £18.51 as the post office doesn’t deal in less than a penny of course.”
57
Enhanced worked example - answer
“The statement ‘We buy at NZ$2.72 to the £’ means that for every NZ$2.72 I give the man he’ll give me £1. (It helps to write out what the statement ‘We buy at NZ$2.72 to the £’ means in practice).
I've got NZ$50 so we need to know how many times 2.72 goes into 50 to see how many £’s I'll get. (remember to explain each step rather than just write down the calculation like 'NZ$ = £50/2.17' The more you explain what you're doing, the better).
Now 50/2.72 = 18.51851852 on my calculator. So the man gave me £18.51 as the post office doesn’t deal in less than a penny of course.” (Be sure your answer makes sense – it's very easy when using the calculator to write down whatever it tells you!) 58
With external formative assessment
100
72 70 65
28
30
35
11 9
5
ASSIGNMENT RIVERGRAM – FORMATIVE ASSESSMENT100 students start the course. At each assignment some drop out and enter the ‘exit’ channel. A very few re-enter the ‘progress’ channel having skipped the previous assignment.
Assignment 1
Assignment 2
Assignment 3
Progress
Exit59
‘Ipsative assessment’ - a new approach?
• Assessment or feedback which compares a student’s achievement not with an absolute standard, but with their previous performance (Hughes, 2011)
• For example students can repeat an assessment with feedback at each attempt until they reach the required standard
• No retention results as yet
60