Date post: | 15-Jan-2015 |
Category: |
Education |
Upload: | leonie-sloman |
View: | 398 times |
Download: | 3 times |
@leonie_learning
Contents
1. Background: project scope, aims, previous studies & data sources.
2. Proportion of medical students using recordings
3. How many recordings medical students used: reliability compared to self-report; validity of accesses representing usage
4. Usage by different student groups: by medical program; by course
5.Access patterns: all medical students; individual patterns.
6.Reasons for use: given in pop-up polls vs student surveys
7. How used: amount viewed; reported behaviour while viewing; advice to other students
8.Technical preferences: device; online vs download; personal devices
@leonie_learning
Background
@leonie_learning
Project scope
•Research undertaken as part of King’s College London,
Technology -Enhanced Learning funded project
•Focused on recordings of lectures for 1281 students:o 425 1st year Biomedical Science students o 469 1st year Medicine students (MBBS 1)o 387 2nd year Medicine students (MBBS 2)
@leonie_learning
Project aims
•To understand how Year 1 & 2 medical students and
Year 1 biomedical science students are using recorded
lectures to support their studying and revision.
•To understand how lecture capture affects lecturers’
teaching practice and experience
@leonie_learning
Previous studies
Many studies that rely on student self-report
Particularly recommended:
•Gosper et al. (2008): 815 students surveyed & 14 interviewed across four Australian universities (Macquarie, Murdoch, Flinders, Newcastle). Includes a case study on 31 multimedia students’ usage behaviour.
•Echo360 (2011): surveyed 1566 students who use lecture capture in 17 UK/US institutions (65% health subjects)
•Gorissen, van Bruggen & Jochems (2012a): surveyed 517 students at two Dutch universities (Eindhoven, Fontys)
@leonie_learning
Studies using server-log data
Particularly recommended: •Gorrissen, van Bruggen & Jochems (2013): compared server log data and self-reported usage of recordings by 307 students at Eindoven University of Technology
•Phillips et al. (2010) & (2011): studied access patterns of 435 students from several Australian universities [part of Gosper project]
•Craig et al. (2009): study of 1350 medical and dental students’ use of recordings across 2 years, University of Sydney
•Bacro, Gebregzuabher & Fitzharris (2010): compared 168 medical students’ use of recordings on 3 basic sciences courses at the Medical University, South Carolina
@leonie_learning
Data sources (2012-13)
1 week pop up survey
Nov Dec Jan Feb Mar Apr May
Lecturer self-report
mini poll
focus groups
survey
Student logs (medicine)
Student self-report
survey 1
focus groups
survey 2
@leonie_learning
Do students use the recordings?
@leonie_learning
At least 94% of medical students
accessed the recordings.
* 806 of 856 MBBS 1 & 2 students accessed between 19 Nov 2012 - 27 May 2013
@leonie_learning
How much do students use recordings?
@leonie_learning
Less than 40
40 to 80
80 to 120
120 to 160
Over 160
Approx 160 lectures recorded for each year
How many lectures did medical students access?
(< 25%)
(25-50%)
(50-75%)
(75-100%)
(>100%) 8%6%
11%20%
55%
@leonie_learning
0 1 to 10
11 to 20
21 to 30
31 to 40
41 to 50
51 to 60
61 to 70
71 to 80
81 to 90
91 to
100
101 to
110
111 to
120
121 to
130
131 to
140
141 to
150
151 to
200
201 to
250
251 to
300
020406080
100120140160180200
number of accesses (Nov to May) of approx 160 recordings
num
ber o
f stu
dent
s
How many lectures did medical students access?
@leonie_learning
Reliability: reported vs recorded accesses by medical students
0%
5%
10%
15%
20%
25%
30%
35%
Reported Recorded
* Self-report by 165 post-exam survey respondents vs server logs for 855 students
@leonie_learning
Validity: accesses vs usage
None Selected (<25%)
Some (25-50%)
Many (50-70%)
Most/all (>75%)
0%
10%
20%
30%
40%
* Based on 155 responses in post-exam survey, Jun 2013
What proportion of downloaded lectures did you actually use?*
@leonie_learning
Similar to previous studies: large proportions use recordings, but most use them selectively
• Brooks et al. (2011) found overall access varied from 60% to 90% across 12 different courses at the University of Saskatchewan
• Craig et al (2009) & Bacro et al. (2010) found similarly skewed distribution patterns for the number of recordings accessed by medical (& dentistry) students inSydney & South Carolina.
• Gorissen et al. (2012b) found 90% of 280 engineering students at Eindhoven Technical University used the recordings, but only 4% watched all 34 lectures recorded
• McNulty & Hoyt (2011) found 10-15% of 438 medical students in Chicago used >70% of available lectures; while 64% used <10%
@leonie_learning
Which students use the most recordings?
@leonie_learning
Male vs female?
1st year vs 2nd year?
Program type?
Confidence? (grade expectations)
Are there differences in how much students access recordings?
@leonie_learning
Usage by medicine programFewer graduate/professional students used the recordings
(71% vs 95% on general /extended programs) and they accessed fewer recordings
Students on the extended program accessed more recordings
Standard program (n= 678)
Extended program (n=150)
Graduate/professional condensed program
(n=28)
Number of accesses per student
@leonie_learning
Usage by course
0%
5%
10%
15%
20%
25%
30%
Medics Biomed
More biomedical science students self-reported using large numbers of recordings – but difference not statistically significant
@leonie_learning
When do students use the recordings?
@leonie_learning
19/1
1/12
26/1
1/12
3/12
/12
10/1
2/12
17/1
2/12
24/1
2/12
31/1
2/12
2013
/1/7
2013
/1/1
4
2013
/1/2
1
2013
/1/2
8
2013
/2/4
2013
/2/1
1
2013
/2/1
8
2013
/2/2
5
2013
/3/4
2013
/3/1
1
2013
/3/1
8
2013
/3/2
5
2013
/4/1
2013
/4/8
2013
/4/1
5
2013
/4/2
2
2013
/4/2
9
2013
/5/6
2013
/5/1
3
2013
/5/2
0
2013
/5/2
70
100
200
300
400
500
600
700
800
18 Dec
25 Dec
15 Apr
Mid-sessional exam
Exams
Recordings accessed per day by 856 medical students
@leonie_learning
0
5
10
15
20
0
5
10
15
20
Individual access patterns
Two examples of regular access increasing at the end of term
@leonie_learning
0
20
40
60
80
100
120
An example of bulk downloading
Individual access patterns
@leonie_learning
Similar to previous study: individual access patterns vary among students
Phillips et al. (2010) developed the following labels to categorise students’ access patterns.• Conscientious: access most lectures in the first week available, incl.
high achievers who also revisit later• Good-intentioned: initially regular access then reduces• Repentant: initially little activity, then active in latter half of term• Binger incl. free-timers who binge during holidays and crammers who
leave it all to last 2 weeks before exams• Random• One-hit wonder• Disengaged: not used at all
@leonie_learning
Why do students use the recordings?
@leonie_learning
Pop-up poll for medics
@leonie_learning
Pop-up poll active periods
0
100
200
300
400
500
600
700
800
Mid-sessional exam (Jan) Ex-
ams
Poll 1(Nov)
Poll 2(Jan)
Poll 3(Mar)
Poll 4(May)
628 (73%) medical students responded for 5156 accesses (excluding SKIPs and repeated accesses of the same lecture on one day)
@leonie_learning
Reasons medical students selected for accessing recordings
Re-watching for revision
Difficult topic
Note comple-
tion
Lost con-centration
Missed lecture
Late for lecture
English is second
language
Other reason0%
10%
20%
30%
40%
50%
60%
70%
80%
Nov Jan Mar May
by 628 (73%) medical students accessing 5156 recordings
@leonie_learning
Reasons given in student survey
• To complete notes
• To review a difficult topic
• Re-watching at own pace
• Back up if absent
Please explain what you like about having recorded lectures*
* 259 respondents(25% of MBBS 1 & 2 & Yr 1 Biomedical Sciences, Feb 2013)
@leonie_learning
Lecture behaviour
During a typical lecture, how much
(a) do you understand?(b) can you concentrate on?
* 312 respondents (Feb 2013)little about
halfmost all
0%
10%
20%
30%
40%
50%
60% Able to concentrate
Able to understand
@leonie_learning
Similar to previous studies: popular reasons for use•Review lectures /revision typically most popular reason, e.g. 75-95% using for this reason (Bramble & Singh 2011; Buchanan et al. 2010; Holbrook & Dupont 2009; Reader, et al. 2012); 60-65% (Fernandes et al. 2008; Settle et al. 2011)
•Preparation for exams 70-80% (Bramble & Singh 2011; Buchanan et al. 2010, Copley 2007, Gosper et al. 2008)
•Difficult topics 75-80% (Bramble & Singh 2011; Buchanan et al. 2010, Gosper et al. 2008; Reader et al. 2012) vs 24% Fernandes et al. 2008
•Note-taking at own pace 74% Gosper et al. 2008 vs 30-40% Copley 2007
•Catching up with missed lectures very varied: 83% Gosper et al. 2008; 72% Holbrook & Dupont 2009; 60% Settle, 48% Reader et al. 2012, 30-40% Copley 2007; 24% Fernandes et al. 2008
•NB Substitute for live lectures rare reason: 5-10% (Bramble & Singh 2011; 4% Fernandes et al. 2008
@leonie_learning
However, previous study shows potential discrepancies between reported and actual behaviour
Gorissen et al. (2013) 280 engineering students’ survey responses on why lecture capture was important to their recorded behaviour:• 96% said it’s important for catching up with missed lectures, but
only 27% watched the full length of 1 or more lectures.
• 93% said it’s important for preparing for exams, and accesses were much higher before test, assignment and exam.
• 54% said it’s important for reviewing material after a lecture, but <10% accesses occurred within 1 week of the lecture
• Self-report of proportion of lectures watched seemed accurate
• 70% believe they usually watched over 75% of a recording, but only 7% watched over 50%
@leonie_learning
How do students use the recordings?
@leonie_learning
Amount of recording used
How often did you
(a) watch/listen to the whole lecture?
(b) watch/listen to the short segments?
February (297 respondents)
May-June (243 respondents)
0%
10%
20%
30%
40%
50%
Whole lectureShort segments
% respondents who answered often or always
@leonie_learning
Comparison to previous studies: varied viewing habits
• Gosper et al. (2008) found 71% of 815 students surveyed preferred using whole lecture vs 33% more selective
• Soong et al. (2006) found 48% of 1160 students surveyed preferred using selected clips vs 29% whole lecture
NB! Gorissen et al. (2013) found 70% students believe they usually watched >75% of a recording, but only 7% actually watched >50%
@leonie_learning
• 94% Pause to think/make notes
• 86% Write notes
• 84% Watch some segments repeatedly
• 51% Consult other sources of information on the topic
• 55% Eat or drink
• 25% Browse mail / Facebook / other websites
• 15% Travel i.e. on bus/train/walking
Behaviour while using recordings
*Based on responses to 2 multiple selection questions in February 2013 survey
@leonie_learning
Comparison to previous study: different access patterns among students
Brotherton & Abowd (2004) analysed behaviour logs of students at Georgia Tech and categorised them as• Straight through: without pauses (most common)• Start-stop: with pauses, but no jumps• Skip-ahead: with jumps forward• Re-listen: with jumps backward• Non-sequential: with jumps forward and backward
@leonie_learning
Students’ advice to others
• Use selectively for difficult material
• Or re-watch whole for consolidation
• Use as a supplement after lectures
• Control the pace: pause, increase speed
What advice would you offer a new student on how to use the recordings most effectively? *
* 150 respondents, post-exam survey
@leonie_learning
Use recordings selectively
“You're not here to memorise 200 hours of
lectures.”
“Read over the lecture slides first and then watch the
parts of the lecture that you are unclear about. Saves
time.”
“Only listen if you don't understand and
need clarification that can't be
gotten from the Internet or textbook.”
@leonie_learning
Use for consolidation
“Don't skip any parts, because sometimes
you might be able to hear details that you feel like
you've just heard for the first time. Those little things help.”
“Review the lectures in the comfort of your own home,
and take advantage of the ability to pause and rewind in order to make detailed notes and look things up mid lecture.”
@leonie_learning
“A supplement not a replacement”
“Go to the lecture! It's not a substitute and you can't
ask the video questions – the video is good for those hard concepts you can't seem to wrap your head around or for
those days when your concentration was not at its
best."
“Mark on the lecture handout with a star or
something the sections you will want to listen to again, and
the rough time, to help you find it later on.”
@leonie_learning
When to watch?
“Try to watch the recordings if you need to as soon as possible after attending the lecture, so you can clear up anything you don't understand and make any
relevant notes while the subject content is still fresh in your mind.”
“Watch again a few weeks later – basic
concepts are clearer by then, allowing you to re-
watch and pick up finer, more detailed points of the lecture.”
@leonie_learning
Control the pace
“You can stop and
rewind, re-listen to
sections you don't
understand and skip the
parts you already know.”
“Listen to them at
double speed,
then slow it down for anything that isn't clear, or that needs to be thought through more attentively."
@leonie_learning
Learn how to improve concentration in both live lectures and when watching recordings, to use time efficiently
Develop good note-taking skills, identifying the key points, structuring the information and noting when to refer to recordings or textbooks
Use recordings selectively, just for the parts found difficult or not caught properly, and look at different resources if still stuck.
Take control: pause to refer to other sources, replay or slow down the speed of difficult bits; speed up or skip the easy bits
Experiment with using both Echoplayer and mp4s
Make notes or diagrams that are concise enough to revise from; a complete record of everything the lecturer says will probably not be helpful
Recommendations for study habits
@leonie_learning
What are students’ technical preferences?
@leonie_learning
Devices used
* 293 respondents (Feb 2013)
Own laptop or desk compter
College computer
Tablet or ipad Smartphone MP3 player or ipod
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
@leonie_learning
Online viewing vs downloading
* Based on 300 respondents (Term-time survey, Feb 2013)
Never Rarely Sometimes Often Always0%
10%
20%
30%
40%
50%
OnlineDownload
@leonie_learning
Personal audio devices
* Based on 233 respondents (post-exam survey, June 2013)
Not at all Occasionally (<10 lectures)
Regularly Most/all lectures (>75%)
-10%
0%
10%
20%
30%
40%
50%
60%
70%
32% still using own devices despite lecture capture system
@leonie_learning
Continue allowing students to use own devices until lecture capture system is more reliable and recordings are available faster
Usage should be subject to the same agreement as use of university-provided lectures, ie personal use, no external sharing. Explain to that the lecturers own copyright of any recordings students makes as their voices.
If the university lecture capture system fails, ask students to upload their own audio recordings (system needs to avoid duplicate uploads)
Recommendations for use of personal devices