Building Assessments to Meet Course Objectives

Standards

Authentic Tasks

Criteria

Rubric

Score or Benchmark

Student Feedback

[How well your students perform] ALLOWS

An Overview of 4 Step Process

1) Identify the goals2) Select an authentic task3) Identify the criteria for the task4) Create the rubric

Departmental Mission

Course Standards

Course GoalsObjectives

Identify the Goals

• Goals/objectives drive assessment– What knowledge and

skills do you want the student to know/do?

• Observable and measurable

• Typically one sentence– What the student WILL

do by the end of the module or course.

How to Write Goals

1) Reflect– Former students– Departmental standards– Complaints about students

2) Review– Look at other sources– Talk to colleagues– Look for items that you didn’t list before. Do they

need to be included?

3) Write– Look at the verb phrase– Avoid words like “understand, develop, or appreciate”– Not too broad—can be reasonably assessed with one or

two assessments; covers at least half of the subject matter of a course

– Doesn’t refer to a specific task that the students will do

Introduction to the GroupsTLT 419 TLT 321 Group Member 1 Group Member 1 Group Member 2 Group Member 2 Group Member 3 Group Member 3 Group Member 4 Group Member 4

TLT 234 TLT 112 Group Member 1 Group Member 1 Group Member 2 Group Member 2 Group Member 3 Group Member 3 Group Member 4 Group Member 4

GROUP WORK

1. On the DETC Assessment—Final Project blog page, find the groups and the links to your fake courses in the content section

2. Read your group’s course description and goals.

3. Decide whether the course goals have been written correctly. If not, rewrite them in correct terms.

Select an Authentic Task

• What indicates students have met these standards?

• Easiest way to work from scratch? Do exactly what the standard says!

• Ask “where/how would the student use this in the real world?”

• How do you know if your assessment makes sense?

GRASP IT!

G-- Goal– Statement of the task– Establish the problem, challenge, or obstacle

RASP

IT

GR – Role

– Define the role of the student– State the jobs of the group members

ASP

IT

GRA – Audience

– Identify the target audience within the context of the scenario– Client, committee, someone with no knowledge of material

SP

IT

GRAS – Situation

– Set the context for the scenario– Explain the situation and setting events for the project

P

IT

GRASP – Product and Performance

– Clarify what the students will create and why– Set the Parameters

IT

GRASP

I – Indicators– Provide students with a clear picture of success– Identify with the standards/goals/objectives– Issue rubrics to or develop them with the students

T

GRASP

IT– Test it

– Does it measure what you want it to measure?– Plug in fake student scores to check for accuracy

G– Goal R– Role A– Audience S– Situation P– Product or Performance

I– Indicators T– Test it

EXAMPLE STANDARDS

• Measure quantities using appropriate units, instruments, and methods

• Setup and solve proportions• Develop scale models• Estimate amounts and determine levels of

accuracy needed• Organize materials• Explain their thought process

EXAMPLE ASSESSMENTRearrange the Room

You are designing a space for a client. To help persuade your clients to rearrange the furniture you are going to make a two dimensional scale model of what the room would ultimately look like. Procedure:

1) You first need to measure the dimensions of the floor space in the room you want to rearrange, including the location and dimensions of all doors and windows. You also need to measure the amount of floor space occupied by each item of furniture in the room. These dimensions should all be explicitly listed.

2) Then use the given proportion to find the scale dimensions of the room and all the items. 3) Next you will make a scale blueprint of the room labeling where all windows and doors are

on poster paper.4) You will also make scale drawings of each piece of furniture on a cardboard sheet of paper, and

these models need to be cut out.5) Then you will arrange the model furniture where you want it on your blueprint, and tape them

down. 6) You will finally write a brief explanation of why you believe the furniture should be arranged

the way it is in your model.

Your models and explanations will be posted in the room and the class will vote on which setup is the best.

GROUP WORK

• Given the course goals for your fake courses, choose an authentic task to measure at least 2 of the goals.– You do not need to outline the full task

Identify the Criteria

• What does mastery performance look like?• How will I know if they’ve done well?• Task analysis

What do Good Criterion Look Like?

• Clearly stated– Written in language students understand.

• Distinct and brief• Observable– Statement of behavior

• Shared before the students begin the task

Criteria Basics

• Limit the number of criteria

• You don’t have to assess everything in the task– Does it make sense to

assess this?

• Do your colleagues give similar assignments?– Ask colleagues who are

unfamiliar with subject matter for their opinion

“You may have noticed in the second example that some of the standards and some of the criteria sounded quite similar. For example, one standard said students will be able to develop scale models, and two of the criteria were accuracy of measurements on the scale model and labels on the scale model. Is this redundant? No, it means that your criteria are aligned with your standards. You are actually measuring on the task what you said you valued in your standards.”

~J. Mueller

GROUP WORK

• As a group, identify the criteria that are most important to you to assess within the task you have chosen.– Choose between 2-4 criteria

Standards

Authentic Tasks

Criteria

Rubric

Score or Benchmark

Student Feedback

[How well your students perform] ALLOWS

Rubrics Basics and Building Blocks

• Rubric: A scoring scale used to assess student performance along a task-specific set of criteria

• Criterion Referenced Measures: a student's aptitude on a task is determined by matching the student's performance against a set of criteria to determine the degree to which the student's performance meets the criteria for the task

Components of Rubrics

Criteria

Descriptors

Levels of Performance

Point Values

Create the rubric

• Begin with the criteria• Analytic or holistic?

Analytic Rubric

• Performance is judged separately for each criterion

• How fine of a distinction does it need to be?– 2 options (did or did not)– 5 options (Likert scale)

Criteria Limited Acceptable Proficient

made good observations observations are absent or vague most observations are clear and detailed all observations are clear and detailed

made good predictions predictions are absent or irrelevant most predictions are reasonable all predictions are reasonable

appropriate conclusion conclusion is absent or inconsistent with observations

conclusion is consistent with most observations

conclusion is consistent with observations

Holistic Rubrics

• Considers all criteria together• Potential problem: performance does not

always fall into straight categories• Best for– Making a quick judgment that carries little weight

in evaluation– Evaluating performance in which the criteria

cannot be easily separated.

Oral Presentation Rubric

Mastery

usually makes eye contact

volume is always appropriate

enthusiasm present throughout presentation

summary is completely accurate

Proficiency

usually makes eye contact

volume is usually appropriate

enthusiasm is present in most of presentation

only one or two errors in summary

Developing

sometimes makes eye contact

volume is sometimes appropriate

occasional enthusiasm in presentation

some errors in summary

Inadequate

never or rarely makes eye contact

volume is inappropriate

rarely shows enthusiasm in presentation

many errors in summary

Analytic Vs. Holistic

Analytic• Assess each criterion

separately• Better for large number of

criteria• Handles weighting of

material better

Holistic• Quick judgment needs to be

made• Minor assessments

Levels of Performance

• Clearer Expectations– Leads to a more consistent and objective

assessment• Better feedback for students – Less effort in individual grading

Descriptors• Language used to describe the

behavior associated with a quantified score.

• Do not have to have the same amount of performance levels for each criteria

• More detail in the descriptors, the better a student can approach the task

• “When a rubric is applied more consistently and objectively it will lead to greater reliability and validity in the results.”

Assigning point values

• Flexibility based on professor discretion• Points should be distributed across the levels

of a rubric to best capture the value you assign to each level of performance

Criteria never sometimes usually

makes eye contact 0 2 4

volume is appropriate 0 4

enthusiasm is evident 0 4

summary is accurate 0 4 8

Checking Your Rubric

• Let a colleague review it.• Let your students review it -- is it clear to

them?• Check if it aligns or matches up with your

standards.

Checking Your Rubric

• Create a fake student score– Is it representative of what you think it should be?– Adjust by adding weight to certain categories

• Universal—can someone else use it to grade an assignment who is not versed in the subject matter

• Monitor and adjust– Rubrics are works in progress