Date post: | 07-Aug-2015 |
Category: |
Documents |
Upload: | dr-julia-cronin-gilmore |
View: | 27 times |
Download: | 0 times |
Measuring Program Objectives Via Online Assessment
Julia Cronin-Gilmore, Ph.DTracy Gies
Bellevue University
International Assembly for Collegiate Business Education
About your Presenters…Julia
About your Presenters…Tracy
We are from…
Some think of it as…
Some think of it as…
Gates - $81.6 Billion, Buffett $74.8 Billion
Some even think of it as…
Background• New Degree, M.S., Marketing• 12 Courses
Create Program Objectives• Demonstrate problem solving and critical
thinking skills in order to be successful in the business world
• effective communication skills and professionalism when working with others in a business setting.
• Demonstrate effective interpersonal skills and an appreciation of business ethics.
• Demonstrate the ability to apply content knowledge to create value as business leaders in an organization.
Two Direct & Two Indirect Measures
Direct measures:Entrance & Exit Exams: 80% of students will score at least 73% or greater, which is at a level of proficiency.
Two Direct & Two Indirect Measures
Indirect Measures:Mid-term and End of Program Surveys: At least 80% of the students' will note the ability to apply program learning outcomes as noted by their responses of "Agree" or "Strongly Agree".
Measure Program ObjectivesTwo Direct Measures• In other programs, we used a
business plan and a group of professors would evaluate their specialty area (marketing, finance,…).
• Found to be a time consuming process.
• Needed something manageable for one person
• Decided to try analyzing a case study
Case AnalysisFour Cases Total• All four are equal in length – about ¾
page.• Each has multiple things going on
Three Central Questions• What are the relevant facts of the case?• What are the ethical issues?• What are the ethical alternatives?
Exam #1 ResultsStudents didn’t score so well…
12.21 16.59 23.65
30 Possible Points
Test ResultsTest Results were Skewed…• Students are used to answering once
on a multiple choice exam.• There were several answers that were
correct.• If students answered correctly, they
received a point• If students missed a correct answer, it
deducted points from ones they earned.
Going Forward• Questions reflect how many correct
answers are possible.• Students will earn one point per
correct answer• Students will still loose points earned
for incorrect answers (couldn’t fix in the system).
Test ResultsTest Results were Skewed…• Exam #1 was extra credit• Eliminated the issue with it being a
major portion of their grade or• Something I had to prepare them for.• Most were encouraged to take• Not every student completed the
exam.
Measuring Program Objectives Via Online Assessment
Part IISetting It Up in the LMS
Agenda
• Why use cases online?• Mapping• Creating the tests in the LMS
Why Use Cases
• Authenticity (Suskie, 2009)
• Effectiveness and student acceptance of CBL (Barnes et al, 1994; Srinivasan et al, 2007)
• Program curriculum favors CBL learning
• CBL in assessment good tie-in
Issues
• Grading responses to cases can be very time consuming
• There can be a wide range of correct answers
• Setting up auto graded tests that capture a wide range of correct answers can be difficult
Our Approach
• Use Multiple Answer questions• Try to capture as broad of a range as
possible of correct answers in terms of ethical considerations, and resolutions
What We’ve Learned So Far
• Generally well received by students• Refine instructions
Mapping
• Provides a common document to prompt discussion for all parties involved in course design, development and review
• Keeps the two main components of learning in a program (program outcomes and learning objectives) a the forefront of design and development discussions
• Helps ensure alignment of program outcomes and course objectives
How to use the Excel Mapping Tool
• Some cells are linked so you can enter info on the Data Entry worksheet, and it will fill the appropriate cells on the other worksheets.
• Demo
Setting up the Test
• Blackboard 9.1.201404.160205• Fairly standard Multiple Answer set
up• Partial Credit• Group cases and MC questions
Set Up in Blackboard
Questions?