A New Web-Based Tool for Assessing the Student Experience in Learning Communities

PNAIRP 2009

Portland, OR

Presenters:

Maureen PettittSkagit Valley College

Gillies MalnarichWashington Center for Improving the Quality of Undergraduate Education

Session Outline Why this survey? Development

Survey items Feedback/revision

Fall 2009 Survey Priorities Future Priorities An Invitation

Thinking about learning

Thinking about learning

What behaviors and/or activities create more than mere rote learning?

Survey Development

The survey was created as a result of input from colleges and universities involved in the Assessing Learning in Learning Communities project who expressed the need for a survey that examined students’ experiences in learning communities, especially with regard to integrative learning.

Washington Center approached Skagit Valley to help design and administer the online survey.

Goals

1. Collect national data about the “what” and “how” of substantive learning in learning communities

Use that data to guide the creation of online resources and tools that help faculty to enhance students’ -- and their own -- experiences in the learning communities

Definitions Learning community (LC)

A common cohort of students who attend two or more classes together in which they experience an explicitly designed opportunity for integrative learning.

Integrative learning the intentional integration of ideas and/or

practices drawn from two or more disciplines (or areas of expertise) which invite new questions, understanding, explanations, and applications.

Development Process

Review of literature and surveys about student learning and integrative learning

Draft multiple versions (lots and lots of revisions!) based on: Input from colleagues across the country Recommendations made by students in a Skagit

learning community – instructors used a “think aloud” protocol to identify and revise questions which lent themselves to different interpretations

Survey Elements

1. Students’ engagement in classroom activities

(You just responded to some of those questions!!)

2. Instructors’ activities that support learning

3. Students’ perceptions of gains made in their own understanding and abilities

4. Students’ perceptions of the mental activities they engage in in the LC versus other courses

Students’ Experiences of Learning in Learning CommunitiesFall 2009 - National Pilot Establish liaison with learning community

leads and institutional research offices Administer ~ 250 surveys per participating

college Trouble shoot technical aspects and get

feedback on logistics and survey questions (“revise again!?”)

A few quotes about revising... “There are always too many words at first.”

Jacques Barzun

“Forced to weigh your words, you find out which are the styrofoam and which are the heavy gold.” Ursula K. Le Guin

“All first drafts are shit.” Ernest Hemingway

Issues Thus Far….

Individualizing survey – for example, additional questions

Institutional Review Board – colleges will deal with this at the local level

Limiting the pilot to 250 surveys per college – ok we bent a little on that, but don’t tell

Limitations of Remark software – especially the requests for college-specific questions

Students’ Experiences of Learning in Learning CommunitiesWinter 2010 Compile and send data results to fall 2009

pilot schools Summarize and respond to feedback Revise online survey based on pilot feedback Post new survey for use at minimal cost (e.g.

$50 for every 250 surveys) with goal of making survey available for mid-term use

Students’ Experiences of Learning in Learning CommunitiesSpring 2010 Analyze data from all participating campuses

(i.e. broad sweep of classroom practice) Adapt “small group instructional diagnostic” to

supplement mid-term survey use Draft a prototype resource list and faculty

development module based on survey results Report early findings to national learning

communities community via listserv

Students’ Experiences of Learning in Learning CommunitiesSummer 2010 Begin work on faculty development section of

Washington Center website Build network of IR specialists and LC leads

interested in deepening research on students’ experiences of learning in learning communities

Seek funding for further development of this work

Faculty Development Modules (in design) Sample Question

“Work on an assignments that require integrating ideas, strategies, or skills from classes (or disciplines) included in this learning community”

Faculty development module could include: References about designing integrated learning from

Washington Center’s National Project on Assessing Learning in Learning Communities

Teaching practices drawn from Classroom Assessment Techniques (CATS) and collaborative learning techniques

CCSSE Annotated Bibliographies

The CCSSE annotated bibliography can be accessed from the “About the Survey” webpage:

http://www.ccsse.org/aboutsurvey/biblio/page1.cfm

An Invitation If your college is engaged in learning

communities and you are looking for ways to assess the learning that takes place in the LC, feel free to email Gillies to discuss the possibility of conducting the survey at your college: malnarig@evergreen.edu

If you have questions about survey development, data collection or analysis, feel free to email Maureen: mpettitt@skagit.edu