Post on 22-Feb-2016
description
transcript
EVALUATION REPORTDerek R. Lane, Ph.D.Department of CommunicationUniversity of Kentucky
Evaluation Questions• RQ1: What is the reliability of existing scales for evaluating the faculty
(n=11) and student (n=198) dimensions of TBL Team Science implementation?
• RQ2: What are the differences in student perceptions with respect to first-time and more experienced TBL teachers?
• RQ3: How do student perceptions of TBL different with respect to lecture and lab classes?
• RQ4: How do student perceptions of TBL Team Science differ as a function of course section?
• RQ5: How has the curricular development impacted student achievement scores (withdrawal, unsatisfactory scores)?
Methods
• Cross-sectional Post-test Only Survey Research Design
• Eleven Dependent Measures
• Scales where appropriate
• Single item measures
• Three open-ended questions
• Multiple Analysis of Variance Analysis
• Eleven Dependent Measures
Measures• Positive Attitudes about Learning
• Student Motivation
• TBL Attitudes
• Value of Teams
• Self Efficacy
• Self-Reported Learning
• Motivation to Excel in College
• Overall Quality of Group Experiences During Semester
• Teacher Caring
• Teacher Classroom Management
• Teacher Immediacy
Sample Demographics• Gender (54% Male; 46% Female)• Class Rank (75% Sophomore)• Ethnicity (70% Caucasian)
Sample Demographics• GPA (68% 3.0-3.5)• Full-time (85%)• Major (~32% Pre-professional)
Student Measures - Scales• Positive Attitudes about
Learning• n=7, α=.913, range = 1-7• mean = 5.64, s.d. = 1.02
• Student Motivation • n=8, α=.885, range = 1-7• mean = 5.07, s.d. = 1.09
• TBL Attitudes • n=11, α=.940, 1-5• mean = 3.65, s.d. = .84
• Value of Teams • n=12, α=.902, range = 1-5• mean = 3.74, s.d. = .68
• Self Efficacy • n=5, α=.929, range = 1-5• mean = 4.16, s.d. = .76
• Self-Reported Learning • n=9, α=.782, range = 1-5• mean = 3.81, s.d. = .59
Student Measures – Single Items• Overall Motivation to Excel in College• range = 1 – 7• mean = 5.99, s.d. = 1.05
• Overall Quality of Group Experiences this Semester • range = 0 - 100• mean = 66.04, s.d. = 26.53
Teacher Measures - Scales• Teacher Caring
• n=6, α=.80, range = 1-7• mean = 5.69, s.d. = 1.09
• Teacher Classroom Management • n=7, α=.938, range = 1-5• mean = 4.16, s.d. = .80
• Teacher Immediacy • n=8, α=.749, range = 1-5• mean = 3.96, s.d. = .60
RQ2: First Time TBL Teachers vs. Experienced TBL Teachers (n=198)
RQ3: Lab Classes vs. Lecture Classes (n=198)
[F(14,183) = 1.92, p = .026]
[F(14,183) = 1.92, p = .026]
[F(14,183) = 3.95, p = .0001]
[F(14,183) = 3.95, p = .0001]
[F(14,183) = 1.75, p = .050]
[F(14,183) = 1.75, p = .050]
Student Outcomes in Selected Chemistry and Biology Courses 2008-2009 Academic Year
Table 4. Student Outcomes for Courses Being Transitioned to TBL Format Spring 2011
* Two courses –one taught by an experienced faculty member (22% TUA) and one by a part-time untrained but mentored faculty member (46% TUA).
Evaluation Results• RQ1: What is the reliability of existing scales for evaluating the faculty (n=11) and student (n=198)
dimensions of TBL Team Science implementation? • All scales operated at acceptable to excellent levels except the problems associated with GOAL STRUCTURES, and
TEACHER IMMEDIACY measures.
• RQ2: What are the differences in student perceptions with respect to first-time and more experienced TBL teachers?
• EXPECTED! Higher scores for experienced teachers on motivation, TBL attitudes, group experiences, and perceived learning. Significant differences for first time classes on positive learning attitudes.
• RQ3: How do student perceptions of TBL different with respect to lecture and lab classes?• GOOD NEWS! No significant differences; though lab means were higher for all measures except classroom
management, immediacy, and learning).
• RQ4: How do student perceptions of TBL Team Science differ as a function of course section?• Courses were significantly different for only 3 of the 11 measures: overall group experiences, TBL attitudes, and the
perceived value of teams)—explained by experience teaching TBL courses.
• RQ5: How has the curricular development impacted student unsatisfactory achievement scores (withdrawal, unsatisfactory scores)? Improvements between 7 – 16%; mean=12.2%.
• BIO 150 improved 9- 12% from 44% to 32% to 35%• BIO 152 improved 17% from 32% to 15%• CHE 170 improved 7% from 48% to 63% to 41%• CHE 180 improved 16% from 45% to 29%
Qualitative Results
• Strengths of TBL Experiences• Challenging/Engaging Hands-on Applications• Collaboration and Peer Support• Outstanding Instruction
• Suggestions for Improving TBL Experiences• Improve structural issues related to . . .
• Time management (4s, no need for outside meetings)• Individual accountability (peer evaluation)• Balance between individual and group expectations• Specificity of RATs and Assignments (i.e., experiments, group appeals)• Logistical Issues
• Timely return of graded assignments • Improve online organization of assignments• More realistic experiments
Qualitative Results
• Additional Comments• Identify TBL Sections• Improve Teacher Training• Student Preparation• Teacher Clarity
Discussion
• Implications• Limitations
• Post-test only• Sample – representative?
• Future Directions – this semester• Pretest on attitudes• Pretest on cognition• Other measures to include
• Teacher Credibility?• Student Engagement?
• Follow-up interviews with students who drop or fail to perform to standards?
Suggested Resources
Suggested Resources
EVALUATION REPORTDerek R. Lane, Ph.D.Department of CommunicationUniversity of Kentucky