49
Using Taskstream for Student Assessment Dr. Chris Boosalis – Teacher Education May 13, 2011
Using Taskstream for Student Assessment
Dr. Chris Boosalis – Teacher Education
May 13, 2011
Purpose
Demonstrate how Taskstream can be
used for student assessment and
accreditation
• Teacher Performance Assessment
• Signature Assignment Evaluation
• Student Teaching Supervision
• Reporting on Student Learning
Outcomes
SLOs in Teacher Ed
A. Making Subject Matter
Comprehensible
B. Assessing Student Learning
C. Engaging Students
D. Planning Instruction
E. Creating Environments for
Learning
F. Developing as an Educator
Performance Assessment
Signature Assignment
Student Teaching
Accreditation Reporting
Lessons Learned
• Taskstream is useful for archiving
student assessments
• Taskstream is useful for rating
students on SLOs
• Taskstream is useful for displaying
SLO reports to accreditors
Thank You!
Q & A
First Annual
Assessment Spotlight
Friday, May 13, 2011
MSR 130
Welcome
External Review: Nuts & Bolts Jennifer Helzer, Geography
May 13, 2011
Purpose
• Purpose:
To relay experiences and lessons
learned with external review as part
the Academic Program Review
(APR) process.
• Outline:
– Process
– Selecting an external reviewer
– Results
GEOGRAPHY
APR
Process
1. Learn about the process…reach out
and research.
2. Share your knowledge with program
faculty.
3. Address questions and concerns.
4. Select a style of review that works
best for the program
– Campus visit
– No campus visit
– Number of reviewers
5. Develop a timeline that meshes with
the APR calendar.
Process – Selecting an External
Reviewer
1. Identify a pool of candidates
– Previous experience
– Professional conferences/affiliations
– Send out call (professional list-serves, professional
organizations, journals)
– Consider candidates from programs you aspire to
2. More communication with program faculty, Dean.
3. Arrange campus visit:
– Candidates may request to meet with particular
support units and administrators
– Consider including: staff, other departments, IR,
OSRP, OSL, PAC and of course students.
– Develop itinerary
– Send draft APR to reviewer; other relevant
materials.
End Result
21 page report with executive summary: Data Sources
• This external review is based on information from the following sources:
• Examination of the “Self-Study -- Geography Program (2009-2010)” document, including its substantial
• collection of supportive and supplementary appendices
• Visits to the following web pages:
• • Geography Program web pages:
• ◦ http://www.csustan.edu/geography/
• ◦ http://geoimages.csustan.edu/Geowebsite.htm
• ◦ http://www.csustan.edu/academics/CHSS/Geography.html
• ◦ http://www.extendeded.com/GIS/
• • Geography Club's web sites and Facebook pages:
• ◦ http://csustanislausgeographyclub.webstarts.com/index.html
• ◦ http://www.facebook.com/pages/CSU-Stanislaus-Geography-Club/157930007711
• • College of Humanities and Social Sciences web page:
• ◦ http://www.csustan.edu/CHSS/
• • CSU Stanislaus web page:
• ◦ http://www.csustan.edu
• • CSU Stanislaus online catalogues from 2005-06 through 2010-11:
• ◦ http://catalog.csustan.edu/index.php?catoid=3
• • The 2010 WASC report
• ◦ http://www.csustan.edu/wasc/
• • Outside ranking sites:
• ◦ Princeton Review: http://www.princetonreview.com
• ◦ State University: http://www.stateuniversity.com
• • Searches for information on the Geography Program at CSU Stanislaus in local newspaper web
• sites:
• ◦ Modesto Bee: http;//www.modbee.com
• ◦ Turlock Journal: http://www.turlockjournal.com/
End Result
21 page report and
executive summary:
• Mission and Goals
• Assessment
• Curriculum Improvement and
Modernization
• Faculty and Staff
• Faculty Research
• Service to Students
• Institutional Support and
Resources
• Tables and graphs
• Conclusions and
Recommendations
Assessment Spotlight 2011
Direct and Indirect Assessment of Biology Majors Matt Cover, Jim Youngblom, Marina Gerson Department of Biological Sciences
May 13, 2011
Purpose
Existing approach: senior surveys – indirect assessment of learning goals – career goals
Desired approach: senior AND new student surveys – direct assessment of student learning – career goals, expectations,
preparation, motivation, activities
Challenges • Large number of bio majors (~650) •Direct assessment: what test or assignment?
–Should reflect the entire discipline –Year-to-year consistency
• No shared senior-level class •Logistics of administering surveys
Process Direct Assessment
– Campbell and Reece’s Biology: ~4,000 question test bank
– Reviewed by bio faculty, reduced to ~1,400 questions
– Random sample of 20 questions – Knowledge/comprehension – Qualtrics online survey
(I.R.: Angel, Lisa, Kelli)
Process New Student Survey
– Background, motivation, expectations Senior Survey
– Indirect assessment of learning objectives (“Please evaluate how your degree program…”)
• Skill in communication, analysis, research, teamwork…
– Plans, career goals, student experience, activities
# Correct / 200 5 10 15 20
05
101520253035
Results New Student Survey
• 4th week of lab, 1st semester of General Biology (BIOL1050), <1 hour
• Direct assessment (n = 164) – Mean: 6.6 / 20 – SD: 2.5
Results New Student Survey (BIOL1050) 180 responses Bio Major?
– Yes (67%) Why bio?
– Preparation for career (49%) – Familial advice (28%)
Preparation – AP Bio (38%) – Honors Bio (21%)
Results New Student Survey (BIOL1050) Background
– English language learner (11%) – 1st Language: Spanish (32%), Hmong (26%)
Work – None (59%) – 11-20 hrs/week (17%) – >20 hrs/week (19%)
Results New Student Survey (BIOL1050) Planned highest degree
– B.A./B.S. (14%) – M.S. (26%) – PhD (18%) – Professional health degree (41%)
Future career – Medical Doctor (26%) – Pharmacist (8%) – Nurse (8%) – Dentist (6%)
Results Senior Survey • Email students who have applied to
graduate (mid-May) • Outside of class, online survey (honor
system) • Entice with party and prize
Lessons Learned 1. I.R. is a tremendous resource for
departmental assessment 2. Multiple choice makes large-scale direct
and indirect assessment feasible 3. Appreciation for new students'
backgrounds, expectations, motivations 4. System in place for:
– direct assessment of student learning (knowledge)
– changes in career goals, motivation, work… – long-term trends
Library Assessment of Research
Skills Instruction Tim Held Library
May 13, 2011
Purpose
• The library is a support unit that provides
instruction.
• Information literacy forms the basis for lifelong learning. An
information literate individual is able to:
– Determine the extent of information needed.
– Access the needed information effectively and efficiently.
– Evaluate information and its sources critically.
– Incorporate selected information into one’s knowledge base.
– Use information effectively to accomplish a specific purpose.
– Understand the economic, legal, and social issues
surrounding the use of information.
From Association of College & Research Libraries Information
Literacy Standards for Higher Education
Purpose
• Support units need to demonstrate their
contribution to student success.
• WASC says:
“The Library’s Support Unit Review notes that the Library needs
to collect data that will assess student learning as a result of
their instructional programs, and the team encourages the
Library to do so. Based on the documents made available, the
Library has not developed the capacity to assess the library’s
role in student engagement and learning (CFR 4.3).”
Report of the WASC Visiting Team Capacity and Preparatory Review,
Western Association of Schools and Colleges, 2008, p. 11.
Issues
• Assessing information literacy is challenging
because:
– Librarians only see students for 50 minutes.
Issues
• Assessing information literacy is challenging
because:
– Librarians only see students for 50 minutes.
– Evidence of learning goals is in their research papers.
Issues
• Assessing information literacy is challenging
because:
– Librarians only see students for 50 minutes.
– Evidence of learning goals is in their research papers.
– Few instructional programs assume responsibility for
information literacy as a learning goal.
Issues
• Assessing information literacy is challenging
because:
– Librarians only see students for 50 minutes.
– Evidence of learning goals is in their research papers.
– Few instructional programs assume responsibility for
information literacy as a learning goal.
– The library does not have the ability to reach all
students or track their progress from start to finish.
Build Assessment
• Library Instruction Assessment Plan
– Based on academic program assessment plan
– Information literacy learning goals mapped to library
curriculum.
– Each year target one or two goals to assess.
Build Assessment
• Library Instruction Assessment Plan
– Based on academic program assessment plan
– Information literacy learning goals mapped to library
curriculum.
– Each year target one or two goals to assess.
• Assess Research Skills Instruction Sessions
– Typical session
– Typical learning goals
– Survey faculty
• http://library.csustan.edu/
Results of Faculty Survey
• Sessions working well.
Results of Faculty Survey
• Sessions working well.
• Issues with citations as a goal of the session.
Solution includes addressing it in other forms of
instruction.
Results of Faculty Survey
• Sessions working well.
• Issues with citations as a goal of the session.
Solution includes addressing it in other forms of
instruction.
• Previous pilot survey also garnered qualitative
comments from faculty that indicated
satisfaction, though lecture is boring.
Assess the Assessment
• Goals of survey are not flexible, though session
goals are flexible.
• Direct assessment, but aggregated.
• Easy to administer.
Future Improvement
• New expectations from WASC Substantive
Change Proposal to assess library instruction.
• Automate the survey.
• Increase sample size.
• Customize your survey.
• IR is key.
• Integration of assessment with curriculum is key.
• Instructional faculty buy-in is key.
• Library faculty buy-in is key.
Critical Transitions: Using Literacy
Assessment Data to Guide Retention
Efforts for At-Risk Students John Wittman and Alex Janney, English
May 13, 2011
Purpose
• To investigate student matriculation from
ENGL1000: Introduction to Composition to
ENGL1001: First-Year Composition
(skills, perceptions, pedagogy)
• Do skills acquired in ENGL1000 adequately
prepare students for ENGL1001 and writing in
subsequent courses? (transference)
Process/Methods
• Student interviews of ENGL 1000 and ENGL
1001 experiences--coded with MaxQDA
(indirect)
• Rhetorical analysis of student writing (direct)
• ENGL1000/1001 cohort survey (indirect)
• Rhetorical analysis of ENGL 1000 portfolios
(direct)
• Who and When: 24 ENGL 1000 students over
the course of the Spring 2011 semester
Recurring Themes
• Pedagogy: Peer Review, Rubrics, Instructor
Feedback
• Student Experience: Small Class Size, Student-
teacher interaction, Writing Center
• Skills: Reader Response, Writing across Genres
End Result
• Direct impact-curriculum development, staff
training
• Direct impact-assessment protocol developed
for writing program
• Indirect impact-research and assessment
protocol established for future research (i.e. a
longitudinal study to track student matriculation
to graduation--especially the WPST and WP
courses)
Lessons Learned/Advice
Lessons and Advice
• Take advantage of university resources
– (EAP, Office of Assessment, IR, GEAC, ORSP)
• Have both short-term and long-term goals
• Double dip (or make it count as service and
research-IRB process)
To improve
• Continue to refine methodology (grounded
theory)
• Translate qualitative research into administrative
value
