STUDENT LEARNING IN AND OUTSIDE THE CLASSROOM USING A HELIX MODEL
OF ASSESSMENT
Lori Knapp
Margaret F. BoorsteinLong Island University
Presented at
AALHE June 6, [email protected]
Copyright Lori Knapp and Margaret F. Boorstein
Phases of Development
• Traditional: Initiation of OA structure
• Importance of Faculty involvement and leadership
• Necessity of Administration expertise and support
• Some re-routing of resources
• Formalized and sustained structure
– Examining learning in the major and general education led to beginning of reflection and sharing of results
• Integrative approach
– Faculty cooperation in the classroom
– Integration of faculty and campus life inside and outside classroom FYE, FY Seminar
• Helix
– Keeping all these pieces intact and well-oiled. Including all of the above with curricular innovation and new perspectives of looking at, and evaluating, student learning.
Incorporating additional modes of assessment
Inquiry-based
Data Elements for Phases of DevelopmentProgression of measures employed
• Traditional: Initiation of OA structure• MAPP; AP; Senior Student Survey and Senior Exit Interview
• Formalized and sustained structure examining learning in the major and general education
• Major and Program SLOs; General Ed; Student Artifact Studies; NSSE; Development of rubrics
• Integrative approach• First-yr retention, GPA; IR Graduate Survey; AACU Integrative learning rubric; Early-warning system
• Helix: – Campus Symposia;
• New modes of assessment: Inquiry-based– Investigation of the literature in the discipline combined with what faculty know are weaknesses and
strengths of our students
Implement: More holistic perspective
Outcome: Enhanced student success.
Student-centered with faculty and administration in the backgroundMore self-responsibility and taking of chances beyond the traditional studies to reap benefits
Faculty-Driven Process
Co-chairs
Outcomes Assessment
Steering Committee
Campus Outcomes Assessment Committee
Academic
Department Chairs, Faculty
and Deans
Support Services:
Associate Provosts and Area Directors
Assessment of Student Learning:
Faculty-ledAssessment of Support Services
Strengths and Weaknesses
+ -• Nascent culture of
assessment
• Formalized some administrative support
• Built representation of stakeholders
• Department-centric approaches and measures
• Campus approaches and measures: AP and MAPP
• Lack of clarity about terms or jargon• Lack of recognition of value• Misunderstanding of purpose• Lack of systematic approach to SL at
the program level• AP and MAPP: Limited usefulness• Campus support: Program
assessment over student learning• Limited faculty participation in OASC
and Campus-wide committee• Confusion about roles of academic
deans• Persistence of silos• Still missing: ways to use assessment
data to improve the whole-student experience
Campus –wide
Outcomes Assessment
Steering
Committee
Campus Support Services Branch
OASC
Student Learning Objectives
Institutional Effectiveness
Faculty Branch
OASCUniversity Director of
Assessment
Student Learning Objectives: Major, Program,
General Education
University Director of Assessment
AY10-11
Phase I – due November 15, 2010 Phase II –due May 31, 2011
Program Learning Goal:
Students will…
Student Learning
Objectives
Students will…
Direct and Indirect
Measures *
Findings (Learning
Outcomes)
Interpretation Actions Planned
Goal 2: Students should
understand the role of
water in sculpting surface
landforms.
Learning Objective 2.1
Students will explain how
streams shape the earth’s
surface
Measure 1:• Students will identify the erosional and
depositional features associated with a
meandering stream on multiple choice lecture exam questions.
sample lecture exam question
1) Point bars are deposited:
a) Where the river flows fastest around a meander bend
b) On the inside of a meander bend
c) On the outside of a meander bend
d) Next to a cut bank
Measure 1:
•
Measure 1:
•
Measure 1:
•
LONG-TIME AGO…CALLING IN OUTSIDE INFLUENCES:AAC&U…
• Standardized tests: • Academic Profile
transformed into MAPP
• Limited usefulness
Department Reports
AEC
• Liberal Arts faculty analyzed their department learning goals compared to Campus-wide Core goals
• Student Artifact Study• Cooperation of faculty:
Over summer and with actual student work
• Campus-wide symposium: Faculty and Campus Support Staff with invited speaker
• Followed by integrated break-out sessions
• Break-out sessions to delve more deeply into the themes of speaker
• and apply to Post’s particulars,
• e.g., what it means to be an educated graduate from Post, which is related to student preparedness, core, retention,…
• Have continued for the past five years.
• Fall and Spring: Open to all CSS and faculty
Assessment Flash: Monthly e-mail update designed to inform and alert faculty and staff about what is happening and new possible paths of assessment
Assessment Fair
• Opportunity for Staff and Faculty to share results (and get a free lunch):
• Posters; PowerPoint presentations; Three-dimensional displays (Chemistry)
• Most recently: Three-minute PowerPoint summaries, then discussion
Welcome to the April 2012 Assessment Flash, a monthly e-message update from the LIU Post OASC. The Flash includes brief progress of current projects and keeps you informed about assessment-related events. Roundtable Discussions Two roundtable discussions held this semester resulted in a proposed version of the AY 2012/13 Template which will be distributed to Department Chairs and Program Directors by the end of May so that faculty have access to it and the AY 2012/13 requirements before leaving campus for the summer. Departments will have the choice to use the new template or the original template during the upcoming 3-year assessment cycle. As always, the first reporting for the new cycle will be due on November 15
th, 2012.
Assessment Fair LIU Post’s first annual Assessment Fair was held in the Tilles Atrium on Thursday, April 12th. Since the goal of the Fair was to get people talking about assessment and learning from each other, it was a great success! Those attending had an opportunity to learn from the assessment work being done by many programs on campus. Thanks to all the participants who translated the sometimes dense process of assessment into colorful and interesting presentations that made it all so much more understandable. Special thanks to the Information Technology Resource Center (ITRC) staff, the Provost’s Office and the Office of Academic Affairs for their support of the event. Fall 2012 Curriculum Mapping Workshop Early in Fall 2012, the Office of Academic Affairs will sponsor a Curriculum Mapping workshop. Curriculum Mapping is a process that addresses alignment between program goals/objectives and the program’s curriculum. Program faculty who collaboratively complete the curriculum mapping process often identify gaps in curriculum when they analyze the alignment b etween their curriculum and the program learning objectives, leading them to make curricular changes before they even begin to collect data on their objectives. With this in mind, the Outcomes Assessment Team decided to support Faculty in completing their Curriculum Mapping with this workshop.
Reminder: Phase II OA Reports are due on May 31. Directions have been sent via Academic Deans. Please contact your Dean’s office if you have not received these directions. OA Support Personnel
Kathleen Morley University Director of Assessment Margaret F. Boorstein, John Lutz, John McLoughlin, Elizabeth Mezick Co-Chairs for LIU Post OASC Michele Dornisch LIU Post Assessment Specialist Kathleen Keefe-Cooperman (CEIS), Dulcie Stoepker (CLAS), Nancy Frye (CLAS), David Jalajas (COM), Suzanne McGuirk (SHPN), Donna Tuman (SVPA) LIU Post Assessment Fellows Measurement Tool Resources
The OA website provides resources for faculty including the Rubrics Brief Overview, Designing Rubrics PowerPoint, Examples of Student Learning, and Suggestions Regarding Measurement Tools (see http://liu.edu/Academic-Affairs/Outcomes-Assessment/Assessment-Resources.aspx) Campus Labs Remember that Campus Labs, formerly StudentVoice, provides webinars for the Baseline products related to assessment. More information is available at: http://www.campuslabs.com/support/training-instructions/
For more information about Outcomes Assessment at Long Island University, visit http://www.liu.edu/Academic-Affairs/Outcomes-Assessment.aspx.
Institutional maturity: well-established framework for assessment and
documentation
Redistribution of Resources:
Board of Trustees Grant: Faculty Development, including:
FYE-sponsored AACU high-impact institutes
Widening pool of faculty fellows: including addressing general education, beyond traditional school/college boundaries
Continue gen-ed symposia
No longer solely driven by external forces, including regional
accreditor, rather program assessment findings
Expansion of understanding of assessing the “whole student.”
Campus OutcomesAssessment Committee
ProvostFaculty Co-Chairs:OASC
Clerical Support
Faculty-Administratorinteraction
Campus OutcomesAssessment Committee
Campus and Academic Leadership
Faculty Co-Chairs:OASC Facultyand CSS Branches
Expertise And resources
Faculty-Administratorinteraction
Whole
student
Partnership
And respect
Further exploration of diversity of approaches:
Assessment of student learning
Pedagogy
Further integration of student support services in assessment model
Formalized faculty governance driving and recognizing new modes of assessment and inquiry
Fine-tuning of redistribution of institutional resources which is one of the main objectives of OA. Based on logic: Coincides with Institutional Effectiveness efforts.
Helix: the shape formed by a line that curves around and along a central line (axis); tangent drawn at any point is at a constant angle with the axis. Our model incorporates a spiraling of
new approaches while remaining true in using data-driven evidence. Our campus faculty and staff built or revised interlocking support mechanisms, including transparency of data; directed financial education; targeted academic guidance; and teaching students how to integrate their learning with their campus and off-campus lives.
Interlocking support mechanisms:
Academic-Advising partnerships:FYE evolved into FY seminars:Curriculum Change by faculty
Campus-wide Faculty governanceIn contrast to school-college
Outside normal structure: curving around helix with constant target of improving student learning.
Novelty and old-school at the same time
LIU promise
• Advisory board of faculty integrated with advising
• Formal integrated individualized and dedicated advisement across 4+ years of study.
– Financial aid
– Resume building
– Academic program progression
Integration of classroom learning and campus and off-campus lives
• Greek life
• Time management
• Development of first-year projects
• Common Read: Process and themes for Post foundations
• Athletes: Service projects
• LIU Cares
• Experiential learning
• Student-run businesses
Learning Communities
Integrative learning
Energizing old and new faculty
Advisors excited about new courses
Enthusiasm spreads to new and existing students
Sophomore Learning Communities
• Happiness, Science and Humanity
• What is happiness and how do we achieve it? Should happiness be our sole goal in life? Recent research within psychology has revealed possible answers to these and other related questions that are often surprising and counter-intuitive. But philosophy often poses challenges to precisely psychology’s “answers” and approach to understanding human nature. The goal of these classes will be to address such questions regarding human happiness, reviewing what both psychology and philosophy have to say about the topic, as well as how we might apply these ideas to our lives.
• Thieves, Vandals and Forgers: A Look at Art and Crime Since the Beginning of Time
• Socialization and the Media
AACU Integrative Learning Rubric
• Connections to Discipline– Sees (makes) connections across disciplines, perspectives
• Transfer– Adapts and applies skills, abilities, theories, or methodologies gained
in one situation to new situations
• Integrated Communication
• Reflection and Self-Assessment– Demonstrates a developing sense of self as a learner, building on prior
experiences to respond to new and challenging contexts (may be evident in self-assessment, reflective, or creative work)
AACU rubric
• Audience experiences
• Integration with inside and outside classroom connections, not only for students, or staff, but faculty and administration.
• Integrative learning leads to ownership which leads to more growth and cooperation.
Measures of Student Success
• First-year retention:
– Increase of 4 percentage points from 2012 -16
• Graduation rates:
– Increase of 8 percentage points from 2012-16
• Faculty use of Early Warning System
– Increase of 13 percentage points from 2014-2015
Resistance to Resignation, Acceptance, Enthusiasm
• Terminology, defining, translating
• AP and MAPP
• Department Assessment reports
“How to”
• Intellectual reflection and inquiry at the academic program and department
• Assessment template
Silo• Intellectual reflection and
inquiry at the campus level
• Sharing of actual results
• Assessment Fair and Assessment Flash
• FYE
Nascent Campus Community
•Recognizing the importance and integration of learning in and outside classroom
Campus community Maturing
•Learning Communities
•First-year seminars
Curricular innovation
embedding •Formalized re-evaluation of assessment results, literature, advances in academic disciplines
New modes of assessment inquiry-
based
Helix Model
Identify the inquiry question and the student learning outcome(s) related to it.
Describe the overall approach and timeline for the entire inquiry.
Describe the data collection methods for the inquiry. These should align with the best practices for, and at least one of the data sources must involve a directexamination of student learning.
For each data source, include a description of -
The tool being used (assignment description with rubric, test blueprint, survey, interview protocol).
When and where the tool will be employed (semester(s), number of administrations, course number, type of course)
The people with whom each tool will be employed (seniors in the major, all students in the course, program graduates, program faculty).
What challenges do you anticipate in implementing this inquiry study?
Any proposed assessment inquiry model must be built upon best practices for assessment by:
• centering on meaningful, continuous improvement of student learning
• focusing on knowledge and skills needed upon graduation
• using multiple measures, including direct measurement of student learning
• having annual data collection & some level of annual reporting
• proposing program-level changes (across the program, as appropriate)
• reporting on implemented changes & eventual re-assessment over time
What is your inquiry question?
What literature might help inform the inquiry?
What potential data/evidence might help answer the inquiry question?
Data source When is that data available?
• Writing Across the Curriculum (WAC): Discipline-based
• Learning Communities: Linked Interdisciplinary courses: Common themes in different disciplines, aka, Integrative Learning
• FY seminar: Core course linked to Introduction to College Life
• Quantitative concepts in different disciplines: Graphing Across the Curriculum (GAC)
• What obstacles have you faced in integrating support staff and faculty• Diverging goals? Resource scarcity? Others?
• What strengths have your campuses built on?• Sharing of success stories? Successful use of standardized
tests?
• What projects have induced enthusiasm?
• What types of data or methods of collection that work for both classroom and out-of-classroom assessment
• How does your campus share outcomes results?