FEVATOOLS: A TOOLKIT TO JUMP-START FORMATIVE

ASSESSMENT OF STUDENT LEARNING AND COURSE

DESIGN

Jim JuliusSDSU Course Design Institute

May 26, 2010

Guiding Questions What does it mean to bring an inquiry

mindset to course design? How should one decide what kind of

feedback to seek? What tools are available to collect

feedback? What do I do with the data?

Summative

Formative

What (and why) are you measuring?

Outcomes: tell you what you got, not how or why

Inputs Processes

Seeking continuous improvement

Also formative

What (and why) are you measuring?

Outcomes

Inputs, Processes

Refining type and quality of learning materials &

activities

Cycle 1 Cycle 2 Cycle 3 Cycle 4

Optimized Hybrid or

Fully OnlineClasses

ConventionalFace-to-Face

Classes

Narrowing Your Inquiry + Selecting Useful Tools

http://fevatools.wikispaces.com

Data from M. Laumakis pICT fellow in 2005 Began teaching parallel 500-student

sections of PSYCH 101 in 2006, one traditional and one hybrid

First fully online PSYCH 101, Summer 2008

Grade data, attendance data, IDEA survey, clicker survey, observational protocols all part of data gathering

SALG How much did the following aspects of

the class help your learning? Rated from 1 (no help) to 5 (great help)

Summer 2008 Fully Online: SALG Data

Question Summer 2008 Online

Taking the test online 4.27

Discussion Forums 3.00

Introduction e-mail that explained the basics of the course

4.50

SALG Data over time & format

Question Fall 2007

Blended

Fall 2007F2F

Spring 2008

Blended

Spring 2008F2F

Summer 2008

Online

Summer 2009

Online

Questions, answers, and discussions in class

3.96 4.04 4.10 4.01 4.36 4.30

Live online class sessions

3.39 4.20 4.15 4.40

Archives of live online class sessions

4.15 4.50 4.44 4.70

Quality of contact with the teacher

3.41 3.48 3.94 3.90 4.26 4.40

Working with peers outside of class/online

3.12 3.22 3.31 3.39 3.82 3.40

Community of Inquiry Survey Statements rated from 1 (strongly

disagree) to 5 (strongly agree) Based on the Community of Inquiry

framework’s three elements:1. Social Presence2. Cognitive Presence3. Teaching Presence

Community of Inquiry SurveyCoI Dimension Summer 2008

RatingsSummer 2009

RatingsSocial Presence 3.94 4.16

Affective Expression 3.56 3.95Open Communication 4.29 4.42Group Cohesion 3.97 4.12

Cognitive Presence 3.96 4.22Triggering Event 3.91 4.15Exploration 3.73 4.08Integration 4.09 4.33Resolution 4.10 4.33

Teaching Presence 4.38 4.52Design and Organization 4.50 4.67Facilitation 4.38 4.52Direct Instruction 4.23 4.33

What would you like to further explore?

Image sources http://www.flickr.com/photos/brockzilla/271302584/ http://www.flickr.com/photos/littledebbie11/3195586863/ http://www.flickr.com/photos/aaronjacobs/2346932630/ http://www.flickr.com/photos/shareski/3044437847/