Date post: | 14-Dec-2015 |
Category: |
Documents |
Upload: | edith-whiten |
View: | 214 times |
Download: | 0 times |
Streaming Video Tutorials for Complex Streaming Video Tutorials for Complex Laboratory Techniques: Laboratory Techniques:
Lessons Learned in Sophomore and Junior Lessons Learned in Sophomore and Junior Biomechanics CoursesBiomechanics Courses
Kristen L. Billiar, Ph.D.Kristen L. Billiar, Ph.D.
Dept of Biomedical Engineering, Worcester Polytechnic InstituteDept of Biomedical Engineering, Worcester Polytechnic Institute
Dept of Surgery, University of Massachusetts Medical SchoolDept of Surgery, University of Massachusetts Medical School
Department of Biomedical Engineering
UMMSUMMS
GoalsGoals
Laboratories in all BME coursesLaboratories in all BME courses Making and interpreting measurementsMaking and interpreting measurements Challenge-based, self-guided, self-pacedChallenge-based, self-guided, self-paced
Particular challenge Particular challenge Rapid growth of BME (40-50 per course)Rapid growth of BME (40-50 per course) Limited resourcesLimited resources
Instructor time (and TA)Instructor time (and TA)Volume of sophisticated equipment Volume of sophisticated equipment Laboratory space (scheduling time)Laboratory space (scheduling time)Funds Funds
Practical vs. theoreticalPractical vs. theoretical Expanding access, not assessing educational Expanding access, not assessing educational
theories or comparing methodstheories or comparing methods
Share approach to problem and assessmentShare approach to problem and assessment Discuss time and effort involved Discuss time and effort involved Describe assessment methods and findingsDescribe assessment methods and findings Initiate dialog – chance to get your feedbackInitiate dialog – chance to get your feedback
Department of Biomedical Engineering
UMMSUMMS
Approach and goal of studyApproach and goal of study
Approach - teaching technologyApproach - teaching technology Web-based streaming video tutorialsWeb-based streaming video tutorials Provide the “toolbox” Provide the “toolbox” Economical, sustainable Economical, sustainable
Allow students to perform labs Allow students to perform labs Self-paced, flexible schedule Self-paced, flexible schedule Without constant instructorWithout constant instructor or TA supervisionor TA supervision
Goal - formative study Goal - formative study Assess student learning from the tutorialsAssess student learning from the tutorials Gather student feedback on their usefulness and quality as a Gather student feedback on their usefulness and quality as a
guide for future improvementsguide for future improvements
Department of Biomedical Engineering
UMMSUMMS
Tutorials – web-basedTutorials – web-based
TutorialsTutorials How?How? rather than rather than WhatWhat and and WhyWhy Large number (>20), short (< 5 minutes)Large number (>20), short (< 5 minutes) Easy to create and modifyEasy to create and modify11
FormatFormatMS Powerpoint, Camtasia, videoMS Powerpoint, Camtasia, videoSlides, video, audio Slides, video, audio
Design teamDesign team Two professorsTwo professors
Biomedical and Mechanical EngineeringBiomedical and Mechanical Engineering Academic Technology Center StaffAcademic Technology Center Staff
Video, editing, streaming serversVideo, editing, streaming servers Two sophomore-level studentsTwo sophomore-level students
20 hrs/wk x 8wks x 2 = 320 hrs20 hrs/wk x 8wks x 2 = 320 hrs
FundingFunding Two small internal grants, paid for student time and provided accessTwo small internal grants, paid for student time and provided access
1Billiar and Shonat, PEBEL II, 2004
Department of Biomedical Engineering
UMMSUMMS
Example tutorial - safetyExample tutorial - safety
Posted on myWPI (Blackboard)Posted on myWPI (Blackboard)
PPT file complete PPT file complete ((linklink)) 23 MB (w/o video clip)23 MB (w/o video clip)
PPS filePPS file 1.2 MB1.2 MB Streaming audioStreaming audio Streaming videoStreaming video Requires fast internet connectionRequires fast internet connection
X
Department of Biomedical Engineering
UMMSUMMS
AssessmentAssessmentSophomore-level “Foundations of Biomechanics”Sophomore-level “Foundations of Biomechanics”
large number (n=47), limited use, single laboratorylarge number (n=47), limited use, single laboratory
Junior-level “Experimental Biomechanics”Junior-level “Experimental Biomechanics” small number (n=17), extensive use as they were meant to be, challenge-basedsmall number (n=17), extensive use as they were meant to be, challenge-based
Questionnaires (sophomore-level only)Questionnaires (sophomore-level only) Pre-tutorialPre-tutorial Post-tutorial – with answers and explanationPost-tutorial – with answers and explanation Controlled environment – 2 one-hour sessionsControlled environment – 2 one-hour sessions Automation with BlackboardAutomation with Blackboard
Safety practicumSafety practicum 10-20 minutes/group, Password given10-20 minutes/group, Password given
Grade comparison (sophomore-level only) Grade comparison (sophomore-level only) Quiz assessing knowledge of concept of stress-strain analysis Quiz assessing knowledge of concept of stress-strain analysis
Feedback – usefulness and qualityFeedback – usefulness and quality Likert-typeLikert-type Written responsesWritten responses
Department of Biomedical Engineering
UMMSUMMS
Example questionnaireExample questionnaire
Department of Biomedical Engineering
UMMSUMMS
Assessment of tutorialsAssessment of tutorials
QuestionnairesQuestionnaires Pre-tutorial scorePre-tutorial score 34% 34% ± 9% ± 9% Post-tutorial scorePost-tutorial score 71% 71% ± 12%± 12%
p < 0.05 paired t-test in all groups (except M5)p < 0.05 paired t-test in all groups (except M5)
0%
20%
40%
60%
80%
100%
M1 M2 M3 M4 M5 M6 S1 S3 S4 S5 S6 S7
Tutorial questionnaire
% correct answers
pre post
Department of Biomedical Engineering
UMMSUMMS
Assessment of tutorialsAssessment of tutorials
Compliance with questionnairesCompliance with questionnaires Completed 100%: n = 27 of 47 (57% compliance)Completed 100%: n = 27 of 47 (57% compliance) Completed >90%: n = 36 of 47 (77% compliance)Completed >90%: n = 36 of 47 (77% compliance)
R2 = 0.84
75%
80%
85%
90%
95%
100%
1 2 3 4 5 6 7 8 9 10 11 12
Questionnaire #
Department of Biomedical Engineering
UMMSUMMS
Feedback surveyFeedback survey
Please rate the usefulness of each tutorial in helping you to successfu lly complete the lab by checking the one cell for each tutorial that most closely
matches your opinion .
Please rate the general quality of each tutorial by checking the one cell
for each tutorial that most closely matches your opinion .
TUTORIAL
Tutorial M1 Tutorial M2 Tutorial M3 Tutorial M4 Tutorial M5 Tutorial M6
Tutorial S1 Tutorial S3 Tutorial S4 Tutorial S5 Tutorial S6 Tutorial S7
Department of Biomedical Engineering
UMMSUMMS
Feedback surveyFeedback survey
What about the Tutorials would you change?What did you especially like about the Tutorials and your experience using them?Please identify aspects of particular Tutorials that were especially problematic.
In each shaded section below, please check the one box that most closely matches your opinion.
In attemptin g to understand how to do the lab, I found these tutorial s, in general and as a whole , were
_ not at all useful. _ something that made me very confused . _ not very useful. _ something that made me a little confused . _ neither not useful nor useful. _ something that neither confused nor clarified. _ somewhat useful. _ something that mad e the material a little clearer. _ very useful. _ something that made the material much clearer. _ much less helpful than written material s. _ much less helpful than one -on-one time with the prof. _ less helpful than written material s. _ less helpful than one -on-one time with the prof. _ as helpful as written material s. _ as help ful as one -on-one time with the prof. _ more helpful than written material s. _ more helpful than one -on-one time with the prof. _ much more helpful than written material s. _ much more helpful than one -on-one time with the prof. _ very difficult to use. _ very inconvenient to use. _ difficult to use . _ inconvenient to use. _ neither difficult nor easy to use . _ neither inconvenient nor convenient to use. _ easy to use . _ convenient to use. _ very easy to use. _ very convenient to use.
Department of Biomedical Engineering
UMMSUMMS
Feedback – Likert scaleFeedback – Likert scale
SophomoreSophomore JuniorJunior
completion of surveycompletion of survey 74% (35/47)74% (35/47) 94% (16/17)94% (16/17)
general quality either “good,” “very good,” general quality either “good,” “very good,” or “excellent.”or “excellent.”
70% to 90%70% to 90% 80% to 93%80% to 93%
““easy” or “very easy” to useeasy” or “very easy” to use 60%60% 100%100%
““quite” or “extremely” useful (all but one)quite” or “extremely” useful (all but one) 54% to 72%54% to 72% 73% to 100%73% to 100%
““as helpful,” “more helpful,” or “much more as helpful,” “more helpful,” or “much more helpful” than written materialshelpful” than written materials
80%80% 100%100%
tutorials alone were adequate (or better) in tutorials alone were adequate (or better) in preparing them for their labpreparing them for their lab
60%60% 75%75%
““as helpful” or “more helpful” than one-on-as helpful” or “more helpful” than one-on-one time with the professorone time with the professor
30%30% 75%75%
Department of Biomedical Engineering
UMMSUMMS
Quiz grade comparisonQuiz grade comparison
Uniaxi al test data quiz Final course grades 2003
(no lab or demo)
2004 (demo only)
2005 (limited
lab)
2003 (no lab or
demo)
2004 (demo only)
2005 (limited
lab) # students 41 45 46* 41 45 47 Average 74 74 85 85 82 86 SD 18 17 13 13 12 8 CV% 24% 23% 15% 16% 15% 10% For quiz: P<0.001 by ANOVA with 2005 different than 2003 and 2004 (P < 0.05, Dunn’s) For final course grades: P = 0.175 by ANOVA (not significant). *One student did not take this particular quiz
Uniaxial test on leather specimen
0
200
400
600
800
1000
0 5 10 15
Extension (mm)
Load (N)
1st cycle
10th cycle
Department of Biomedical Engineering
UMMSUMMS
DiscussionDiscussion
MotivationMotivation Provide labs to students with resource savings Provide labs to students with resource savings
in the long termin the long term Available tutorials (MTS, Microsoft) not well Available tutorials (MTS, Microsoft) not well
suited, much longersuited, much longer General enough for multiple types of users at General enough for multiple types of users at
WPI and beyondWPI and beyond
AssessmentAssessment Effectiveness of tutorials and aspects to Effectiveness of tutorials and aspects to
improveimprove NotNot video vs. written or professor video vs. written or professor NotNot lab vs. no lab lab vs. no lab Controls for summative evaluations not Controls for summative evaluations not
practicalpractical
Resource requirementResource requirement High initial investment (team, time)High initial investment (team, time) Written materials require effort alsoWritten materials require effort also Advantage - technical staff and studentsAdvantage - technical staff and students
Department of Biomedical Engineering
UMMSUMMS
DiscussionDiscussion
Overall success – short termOverall success – short term Students passed practicum, completed Students passed practicum, completed
laboratories laboratories Sophomore, junior, and grad as wellSophomore, junior, and grad as well Instructor time in lab minimalInstructor time in lab minimal
QuestionnairesQuestionnaires Controlled viewing, but lower scores than desiredControlled viewing, but lower scores than desired Too many (12 x 2), too much time commitment Too many (12 x 2), too much time commitment
FeedbackFeedback Too broad for one lab in sophomore-level courseToo broad for one lab in sophomore-level course Well received when used as designedWell received when used as designed
Increased use of equipment - Increased use of equipment - time logged time logged Sophomore-level course > 50 hours; ~ 1hr/student Sophomore-level course > 50 hours; ~ 1hr/student Junior-level course >20 hours/student, relatively Junior-level course >20 hours/student, relatively
few scheduling conflictsfew scheduling conflicts
Facilitated challenge-based laboratoriesFacilitated challenge-based laboratories
True time and resource savings - futureTrue time and resource savings - future
Department of Biomedical Engineering
UMMSUMMS
AcknowledgementsAcknowledgements
Allen HoffmanAllen Hoffman tutorial contenttutorial content
Megan Murphy and Dan Filipe Megan Murphy and Dan Filipe sophomoressophomores tutorial creationtutorial creation
Paula QuinnPaula Quinn assessment consultantassessment consultant
Adam Jacobi and David Adam Jacobi and David Taranto (WPI ATC)Taranto (WPI ATC)
production supportproduction support
Elias Wilson (TA)Elias Wilson (TA) questionnaire and practicum questionnaire and practicum
administrationadministration
Ross Shonat (NIH)Ross Shonat (NIH) introduction to formal CBIintroduction to formal CBI
FundingFunding WPI Educational Development WPI Educational Development
Council GrantCouncil Grant WPI Teaching Technology WPI Teaching Technology
FellowshipFellowship
Department of Biomedical Engineering
UMMSUMMS
Questions?Questions?
Department of Biomedical Engineering
UMMSUMMS
Educational frameworkEducational framework
Challenge-based learning (self-guided) Challenge-based learning (self-guided) Aid in the development of Aid in the development of
critical thinking – authentic problemscritical thinking – authentic problems
Building on HPL frameworkBuilding on HPL framework11
Learner-, community-, knowledge-, and Learner-, community-, knowledge-, and assessment-centeredassessment-centered
Self-pacedSelf-paced Allow for differencesAllow for differences Allow greater number of student-hours in Allow greater number of student-hours in
lab lab
The “Challenge” & problem
identification
Generate ideas
Literature & interview
Testing & analysis
Communication of findings
LearningCycle
Research & revise
1Bransford, Brown, and Cocking, 2000