AAC&U General Education and Assessment February 2014 Martha
VanCleave Professor of Mathematics, Interim Director of Division of
Continuing Education Using ePortfolios for General Education
Assessment Linfield College
Slide 2
Private, comprehensive, undergraduate institution in the
Pacific Northwest 47 majors through three programs: McMinnville -
traditional liberal arts and pre-professional programs (1700
students) A Portland Campus - Linfield-Good Samaritan School of
Nursing (450 students) The Adult Degree Program - a pioneer in
adult education primarily online (805, 410 FTE students). Linfield
College
Slide 3
The Linfield Curriculum(LC) Linfields general education
curriculum was revised in 2007
Slide 4
The Linfield Curriculum Structure Four major components:
Inquiry Seminar(INQS) writing intensive first year course Six Modes
of Inquiry Diversity Studies Major Writing-Intensive (MWI).
Linfield College
Slide 5
The Six Modes of Inquiry Creative Studies (CS) Individuals,
Systems, and Societies (IS) Natural World (NW) Quantitative
Reasoning (QR) Ultimate Questions (UQ) Vital Past (VP) Linfield
College
Slide 6
Upper-Division Mode of Inquiry An upper-division course (300 or
above) In one of the six modes of inquiry CS, IS, NW, QR, UQ or VP
From outside the major department Linfield College
Slide 7
Diversity Studies Global Pluralisms (GP) U.S. Pluralisms (US)
Linfield College
Slide 8
Linfield Curriculum Course Requirements - LCs Students are
required to complete: Inquiry Seminar (INQS) 1 course during first
or second year Modes of Inquiry 6 courses,1 from each area
Diversity Studies - 2 courses,1 from each area Upper-Division Mode
of Inquiry - 1 course Major Writing Intensive Course - 1 course The
INQS is 4 semester credits, all other courses are a minimum of 3
credits Total of 11 courses, minimum of 34 credits
Slide 9
Learning Outcomes U nique for each Mode of Inquiry and
Diversity Studies area Developed by a separate group of faculty
(working group) for each Mode of Inquiry and Diversity Studies area
Consistent with LEAP Essential Learning Outcomes/VALUE rubrics
Requests for courses to fulfill requirements reviewed and approved
by working group Le arning outcomes utilized when reviewing courses
Learning Outcomes must be included in syllabus Syllabus must
describe how learning outcomes will be met
Slide 10
Linfield Curriculum Assessment Plan (LCAT) adopted 2010
Slide 11
LCAT Students must demonstrate meeting the learning objectives
of each mode of inquiry and area of diversity studies through:
Direct evidence Choosing an exemplar or exemplars Posting the work
in an online repository Indirect Evidence Supporting the choice of
these exemplars with a reflection on how they meet the learning
outcomes Linfield College
Slide 12
LCAT Faculty Working group for each LC designation Develop
rubric for assessing evidence of student learning outcomes Assess
random sample of student submissions Linfield College
Slide 13
LCAT Assessment of LC submissions on a rotating schedule 2
designations per year First assessment summer 2011 - QR and VP 2012
US and NW 2013 GP and UQ 2014 CS and IS Cycle repeats Linfield
College
Slide 14
First Round of Assessment QR and VP assessed
Slide 15
QR and VP assessment Exemplars submitted in LMS Limited ability
to provide instructions for students No tool for collecting
demographic data Submissions downloaded for evaluation Record of
evaluation compiled manually Linfield College
Slide 16
Assessing QR
Slide 17
ExemplarySatisfactoryUnsatisfactoryNA 1. Pose questions
involving quantitative relationships... 35 (39.8%)24 (27.3%)25
(28.4%)4 (4.5%) 2. Analyze problems by discussing models 17
(19.3%)29 (33.0%)38 (43.2%)4 (4.5%) 3. Understand uses and
constraints 7 (8%)17 (19.3%)59 (67.0%)5 (5.7%) 4. Communicate and
critique quantitative arguments. 10 (11.4%)20 (22.7%)52 (59.1%)6
(6.8%) Assessing QR Calibrate use of assessment rubric Random
selection of submissions scored by 2 readers, then consensus
Results Linfield College
Slide 18
Assessing QR Observations and recommendations Rated
satisfactory or above #1 67.1%#2 52.3%#3 27.3%#4 34.1% Difficulty
submitting materials from online courses Students need guidance
Revision of learning outcomes Need benchmarks Linfield College
Slide 19
Lessons Learned Students need direction and structure Faculty
should make connections between coursework and LC learning outcomes
Linfield College
Slide 20
Implementing an ePortfolio system
Slide 21
Selecting the ePortfolio Platform Working group Faculty Dean of
Student Affairs Registrar Associate Dean of Faculty Chief
Technology Officer Director of Educational Technologies Linfield
College
Slide 22
Selecting the ePortfolio Platform Criteria Tiered access
Matrices Evaluation Areas for student reflection Ability to
aggregate data Aesthetics and usability Support from provider
Linfield College
Slide 23
TaskStream and Linfield Create a Directed Response Folio (DRF)
Work done by team of faculty Supported by TaskStream Mentoring
Services Linfield College
Slide 24
The DRF Linfield College.
Slide 25
The DRF form https://www.taskstream.com/Main/main_frame.asp
https://www.taskstream.com/Main/main_frame.asp Form for each area
Course Location Course Subject Code Course Number Course Name
Course Delivery (option that best describes how this course was
taught) Year Course Was Taken Class Standing Term Course Was Taken
Linfield College
Slide 26
The DRF form LC Outcomes Please select all of the QR learning
outcomes covered in this course. For courses taken at Linfield,
this information can be found in your course syllabus. For transfer
courses, check all that apply. Note that for courses taken at
Linfield, all four learning outcomes should be selected. In
addition, if this is a transfer course, please select "Transfer
Course." Description of Assignment(s) for Submission Supporting
Description of Quantitative Reasoning Exemplars Linfield
College
Slide 27
TaskStream and Linfield Enroll faculty and students Create
single sign-on using CatNET Username and Password
https://www.linfield.edu/taskstream/index.php Batch process
enrollment via secure FTP site Linfield College
Slide 28
Assessment Using Taskstream
Slide 29
Developing Rubrics with Taskstream
Slide 30
Beneficial Features of TaskStream DRF provides structure for
students Form developed to provide explicit instructions Online
evaluation using the rubric Data storage Data reporting Linfield
College
Slide 31
Assessing NW
Slide 32
ExemplarySatisfactoryUnsatisfactoryAve. Score 1. Use scientific
method... 13 (17%)6 (8%)56 (75%)0.43 2. Think critically 27 (36%)29
(39%)19 (25%)1.11 3. Historical development of ideas 12 (16%)5
(7%)58 (77%)0.39 4. Appreciate how knowledge affects... 12 (16%)26
(35%)37 (49%)0.67 Assessing NW Calibrate use of assessment rubric
Random selection of submissions scored by readers Results Linfield
College
Slide 33
Assessing NW Observations and recommendations Rated
satisfactory or above #1 25%#2 75%#3 23%#4 51% Only 6 submission
(8%) received at least a satisfactory for all four learning
outcomes 13 submissions (17%) received no satisfactory scores Most
students submitted a single document Lack of understanding of what
constitutes a goo exemplar Revision of learning outcomes Linfield
College
Slide 34
New NW Learning Outcomes Courses with the NW designation are
intended to help students understand the scientific method. In
particular, students must demonstrate 1) an understanding of the
theoretical and/or experimental background of a particular topic or
model, sufficient to form a hypothesis 2) an ability to critically
analyze results of scientific inquiry in light of assumptions, 3)
an understanding of how scientific results can be extended to more
general situations in contemporary society Courses with NW
designation address all of the above learning outcomes.
Slide 35
Lessons Learned More attention needs to be paid to developing
studentsunderstanding of what makes a good exemplar Linfield
College
Slide 36
Advantages of Using Taskstream Streamlines mechanics of
evaluation process Additional reports available (summary
statistics) Provides for more meaningful faculty discussions
Between pairs of faculty evaluators Among members of working group
(4 faculty) With faculty teaching courses in each area through
sharing of assessment results
Slide 37
Challenges Student understanding of the LC assessment
Individual student data is not assessed Problematic to ensure
students take the task seriously Need to further develop reflective
habit of mind Faculty development Use of ePortfolio Consistent
articulation of relationship between LC learning outcomes and
course content Linfield College
Slide 38
Thanks Linfield College Charles Dunn - Curriculum Committee
Chair, director of LC Assessment, co-author of DRF Susan
Agre-Kippenhan Vice President of Academic Affairs, Dean of Faculty
TaskStream Ben Coulter Manager, Systems Implementation Jose Romero
Mentoring Services Lauren McCarty Associate Director of Marketing
Dara Wexler Education Solutions Specialist Trudy Milburn -
Education Solutions Specialist Linfield College
Slide 39
Contact Questions contact Martha VanCleave
[email protected] Linfield College