Assessing Essential Learning Outcomes with the “VALUE”

METARUBRICs

Using Rubrics for Teaching, Learning, &

Assessment

Introductions

Office of Academic Assessment 101 Hillcrest Hall (0157)– Ray Van Dyke, 231-6003, rvandyke@vt.edu– Steve Culver, 231-4581, sculver@vt.edu– Kate Drezek, 231-7534, kmdrezek@vt.edu

Others here today

Today’s agenda

Rubrics as an assessment tool– Definition and Overview of Rubrics– Using Rubrics for Program Assessment– Pros/Challenges

VALUE METARUBRICS – Assessment toolkit– Background– Purpose– Application at Virginia Tech

Overview: What is Assessment of Learning Outcomes?

“Assessment of student learning is the systematic gathering of information about student learning, using the time, resources, and expertise available, in order to improve the learning.” – Walvoord

A student learning outcome states a specific skill/ability, knowledge, or belief/attitude students are expected to achieve through a course, program, or college experience.

Example: Upon completion of a B.A. degree in English, a student will be able to read critically and compose an effective analysis of a literary text.

What is The Process for Assessing Student Learning Outcomes?

2. Gather and AnalyzeInformation About

StudentAchievementOf Outcomes

3. Use InformationGathered

To ImproveStudent Learning

1. IdentifyAnd Articulate

StudentLearning

Outcomes

Rubric as an Assessment Tool

A rubric is an explicit summary of the criteria for assessing a particular piece of student work, plus levels of potential achievement for each criterion. 

Not only can rubrics can be used to assess an individual student’s performance for a course grade, rubrics can be used to assess the achievement of course, program, or college/ institutional-level student learning Outcomes

Rubric as an assessment tool

To the extent that a given student assessment measures or relates to a programmatic outcome, it can potentially be aggregated and used for program assessment (course-embedded assessment)

The same student product (e.g., essay, capstone project) is used twice: once for individual student assessment and again for program

Rubric as an assessment tool

Pros– Course-embedded assessment helps

to overcome student motivation problem

– Inherent efficiencies – “two birds, one stone”

– Authenticity of student learning “artifact”

Rubric as an assessment tool

Challenge – the rubric!– Think-pair-share

Take a moment and jot down as many challenges associated with rubrics as you can

Pair up with a neighbor and share your list

Rubric as an assessment tool

Challenge – the rubric! Needs to be well-designed – Time – to create, to test, to

implement, to assess– Reliability/validity – needs to be well-

designed

Well-designed rubrics increase an assessment’s reliability by setting criteria that raters can apply consistently and objectively.

Well-designed rubrics increase an assessment’s construct and content validity by aligning evaluation criteria to standards, curriculum, and instruction tasks.

Rubric as an assessment tool

If, in theory, rubrics can be a useful assessment tool…

AAC&U’s VALUE METARUBRICS ARE A AAC&U’s VALUE METARUBRICS ARE A TANGIBLE “TOOL KIT” TO TANGIBLE “TOOL KIT” TO

FACILITATE THE APPROPRIATE AND FACILITATE THE APPROPRIATE AND MEANINGFUL ASSESSMENT OF THE MEANINGFUL ASSESSMENT OF THE ESSENTIAL LEARNING OUTCOMES ESSENTIAL LEARNING OUTCOMES

THAT ARE THE HALLMARK OF A 21THAT ARE THE HALLMARK OF A 21stst CENTURY LIBERAL EDUCATIONCENTURY LIBERAL EDUCATION

VALUE METARUBRICS: ASSESSMENT TOOLKIT

Assessing essential learning outcomes

AAC&U– Organization– role in assessment, accountability – “LEAP” project – Liberal Education &

America’s Promise

VALUE METARUBRICS: Assessment toolkit

Background

VALUE METARUBRICS: Assessment toolkit

Empowers individuals and prepares them to deal with complexity, diversity, and change.

Provides students with broad knowledge of the wider world (e.g. science, culture, and society) as well as in-depth study in a specific area of interest.

Helps students develop a sense of social responsibility, as well

as strong and transferable intellectual and practical skills such as communication, analytical and problem-solving skills, and a demonstrated ability to apply knowledge and skills in real-world settings

usually includes a general education curriculum that provides broad learning in multiple disciplines and ways of knowing, along with more in-depth study in a major.

Liberal Education is an approach to learning that:

http://www.aacu.org/leap/What_is_liberal_education.cfm

VALUE METARUBRICS: Assessment toolkit

http://www.aacu.org/leap/What_is_liberal_education.cfm

AAC&U's LEAP Campaign has defined a robust set of "Essential Learning Outcomes" that students develop during an excellent contemporary liberal education. Beginning in school, and continuing at successively higher levels across their college studies, students should prepare for twenty-first-century challenges by gaining all these outcomes

VALUE METARUBRICS: Assessment toolkit

•Valid Assessment of Learning in Undergraduate Education

•seeks to contribute to the national dialogue on assessment of college student learning.

• builds on a philosophy of learning assessment that privileges multiple expert judgments of the quality of student work over reliance on standardized tests administered to samples of students outside of their required courses.

•The assessment approaches that VALUE advances are based on the shared understanding of faculty and academic professionals on campuses from across the country.

VALUE project: PURPOSE

http://www.aacu.org/value/

VALUE METARUBRICS: Assessment toolkit

•to achieve a high-quality education for all students, valid assessment data are needed to guide planning, teaching, and improvement;

•colleges and universities seek to foster and assess numerous essential learning outcomes beyond those addressed by currently available standardized tests;

•learning develops over time and should become more complex and sophisticated as students move through their curricular and co-curricular educational pathways toward a degree;

VALUE project: ASSUMPTIONS

http://www.aacu.org/value/

VALUE METARUBRICS: Assessment toolkit

•good practice in assessment requires multiple assessments, over time;

•well-planned electronic portfolios provide opportunities to collect data from multiple assessments across a broad range of learning outcomes while guiding student learning and building self-assessment capabilities;

•e-portfolios and assessment of work in them can inform programs and institutions on progress in achieving expected goals.

VALUE project: ASSUMPTIONS

http://www.aacu.org/leap/What_is_liberal_education.cfm

VALUE METARUBRICS: Assessment toolkit

VALUE project: Connections to VT

http://www.aacu.org/value/

Essential Learning Outcomes– Incorporated into purpose statement for

cle: http://www.cle.prov.vt.edu/purpose.html

– Charge to QEP from Provost– Alignment with SCHEV Core Competencies– VT “Partner Institution” in VALUE project

VALUE METARUBRICS: Assessment toolkit

VALUE project: Connections to VT

http://www.aacu.org/value/

VT “Partner Institution” for Value Project– Opportunity to use, provide feedback,

shape rubrics– Join Google Group with access to rubrics as

they are revised– Final rubrics – fall 2009

VALUE METARUBRICS: Assessment toolkit

VALUE project: Why META?????

http://www.aacu.org/value/

Best thinking of higher ed faculty professionals

Specific enough to be useful, broad enough to be useful, meaningful in multiple contexts, at multiple levels for assessment

To the METARUBRICS!

1. Civic engagement2. Creative thinking3. Critical thinking4. Ethical reasoning5. Foundations and skills

for lifelong learning6. Information literacy7. Inquiry and analysis

8. Integrative learning9. Intercultural knowledge

and competence10. Oral communication11. Problem solving12. Quantitative reasoning13. Teamwork14. Written communication

To the METARUBRICS!

Identify learning outcome(s) of interest to program?

Which essential learning outcome(s) are in alignment with program goals/outcomes?

Which rubric(s)/elements of the rubric(s) can be incorporated into assessment efforts?

The Big Message

Avoids Re-Inventing the wheel, staring at a blank page

Potential connections to other campuses, best practices for using metarubrics

More sophisticated, authentic approach to capturing information on our students’ learning

Rubrics help us to Reflect Before we React to our work

“We are being pummeled by a deluge of data and unless we create time and spaces in which to reflect, we will be

left with only our reactions.” – Rebecca Blood

Questions/Comments?

(Please take a Few minutes to complete the session feedback form. Thanks!)