Leveraging Blackboard Solutions for Program AssessmentFour Perspectives

Slides are available at facdev.niu.edu/bbworld17assessment

Answer our

survey

questions in

the app!

Stephanie RichterAssistant Director, Faculty Development

and Instructional Design Center

Northern Illinois University

srichter@niu.edu

@slrichter

Jenna DulakDirector of Web Services

and Online Technology

Hilbert College

jdulak@hilbert.edu

@WNYBUG

Corrie BergeronInstructional Designer,

Blackboard System Administrator

Lakeland Community College

cbergeron@lakelandcc.edu

@skydaddy

Terry PattersonCurriculum, Assessment,

and Technology Specialist

Pulaski Technical College

tpatterson@pulaskitech.edu

@terrypatterson

Introductions

Northern Illinois University

Stephanie RichterAssistant Director, Faculty Development and Instructional Design Center

Northern Illinois University Overview

• Somewhat large rural public university

– Founded 1895 as a Normal School

– 750-acre campus in DeKalb, IL

– 19,000 students, 1,100 faculty

– 60 minutes west of Chicago

• Using Blackboard since 2000

– 95% of students, 90% of faculty

– Automated Bb administrative processes

Program Assessment at NIU

Managed by the Office of Accreditation, Assessment, and Evaluation

Overseen by University Assessment Panel and General Education Committee

Highly distributed process - programs define their own assessment methods, measures, and policies

More programs being held accountable for achievement data by outside accreditors

Blackboard Tools Used for Program Assessment at NIU

Goals and Alignments

Goal Performance Export

Rubrics Portfolios

Why not Outcomes?

Price - budget crisis in Illinois

Primary vs. secondary assessment strategies

Data collection limited to Assignments

Program Assessment with Goals, Alignments, and Performance Export

Champions in each program given Goal Manager role on staging serverBuild Goals using the GUI

Sys Admin exports Goal Set from staging server to ProdUsing free B2 available on Oscelot

Champions and faculty align courses, often starting with past coursesOften requires standardizing assessments and rubrics within the program

Sys Admin uses Goal Performance Export to generate report

Blackboard Portfolios for Program Assessment

● Introduced in 2015 (with upgrade to April 2014)

● Using Blackboard Communities for student submissions

● Complex assessment strategies involving

─ Portfolio Assignments─ Rubrics─ Delegated Grading─ Anonymous Grading

14Templates

9 Programs

General Education Assessment with Rubrics

● Uses standardized rubrics for each baccalaureate outcome (SLOs)○ Rubrics are “No Points”

● Automated process places rubrics in courses○ Based on SLO mapping in SIS ○ Essentially a course copy○ Built into course request process

● DoIT developed a query to extract frequency data from rubrics

Challenges

Goals

• Insufficient staff to support properly

• Rubrics don’t copy or export with alignments *except when copied with an assessment

Portfolios

• Users struggle with the “snapshot” concept

• Complex assessment practices

Rubrics

• “Too many steps”

• Difficult to aggregate data from No Points rubrics

• Lack of central administration for rubrics

Lesson: You can do a lot with built-in tools!

Hilbert College

Jenna DulakDirector of Web Services and Online Technology

Hilbert College Overview

• Small four-year, liberal arts college

– Founded 1957

– 1,100 students

– Suburban Hamburg, NY

– 10 minutes south of Buffalo

• Using Blackboard since 2012

– Migrated from ANGEL

– Migrated to SaaS in summer 2016

Blackboard Portfolios for Institutional Assessment

Portfolios Usage:

• Graduate student requirement

• Course level and program level assessment

• Required for all students for institutional assessment

Graduate Assessment

Required for all grad students

• templates are available for each program

• students have access to a Bb organization that has step by step videos and a portfolio assignment

• students can share their portfolio beyond graduate school

Course Level and Program Level

Institutional Level Assessment

• Why do institutional level assessment?

• How do we assess our blue print?

• Who will participate?

• How will assess the portfolios?

• Where will they be collected?

Institutional Level Assessment

Additional Portfolio Usage

• Faculty portfolios for teaching award submissions

• Leadership program requirement

• Honors program requirements

Future Ideas

Wish List for Portfolio enhancements:

• Add achievement certificates and badges

• Student self assessment rubrics

• Aggregated portfolio reports

Obstacles:

• Faculte usage (i.e. faculty using Bb assignment, rubrics, grades, etc.)

• Oversight and ownership

• Template creation privileges

Lakeland Community College

Corrie BergeronInstructional Designer & Blackboard System Administrator

Video presentation available at

facdev.niu.edu/corrie

Lakeland Community College Overview

• Medium-sized suburban community college

– Founded 1967

– 400-acre wooded campus

– 7,000 students, 450 faculty

– 20 minutes east of Cleveland, Ohio

• Using Blackboard since 2001

– Nearly all students, 75% of faculty

– 30% of class sections are online or hybrid

– Automated Bb administrative processes

Outcomes Assessment Implementation Process

Spring 2015 – Initial meetings and system training

Summer 2015 – Ongoing consultation, small-scale testing

Fall 2015 – Pilot with Library, Nursing, English, Business

Spring 2016 – Library, English, Business

Fall 2016 – Spring 2017 – Library, English, Business (collecting from entire year)

How were we able to implement Outcomes so successfully?

1. We already had widespread Blackboard adoption. Faculty were used to it and comfortable using it.

2. We already had an existing culture of institutional assessment and already-tested rubrics and processes.

3. We had strong support from administration and academic leadership.

4. We had a terrific consulting team - Ruth Newberry and Laura Little – to guide us through the process.

Things that could be improved

• Adding evaluators is clunky & time-consuming

– Have to search individually

• Editing an existing Evaluation Session is limited

– Cannot change name or rubrics

• Cannot filter students by credit hours

– This is important for all Gen Ed departments who want to assess only students who are ready to graduate

– Not an issue for technical programs with a Capstone course from which they can easily gather artifacts from graduating students

University of Arkansas Pulaski Technical College

Terry PattersonCurriculum, Assessment, andTechnology Specialist

Video available at

blackboardguru.com/bbworld2017

Who We Are…

University of Arkansas – Pulaski Technical College

- Joined the U of A System February 2017

- 90+ Programs

- 6,576 Enrollment (Fall 2016)

- Transfer Students

- AA, AS, AAS Degrees

- Technical Certificates

- Certificates of Proficiency

What We Faced…

The UA-PTC Process

• Tedious paper process

–Hours for Instructors

–Hours for Assessment Coordinator

• Some Instructors saw no benefit based on the amount of work.

Establish student learning outcomes

Provide learning opportunities

Assess student learning

Use the results to improve instructional

delivery

Assessment By The Numbers

Paper Based Assessment

4 hours – Collecting, Encoding and Compiling Assessment Data (per instructor)

3 hours – Additional time for a lead instructor to produce reports

Example

6 instructors4 different courses

4 x 6 = 24 hours for instructor to compile data3 x 4 = 12 hours for lead instructor to compile data

Additional 36 hours of time spent pushing papers

Measuring Student Assessment

Rubrics

Used to assess

• Assignments• Test Essays• Online Discussions• Presentations• Speeches• Foreign Language

Interviews

Checklists

Used to assess

• Microscope Skills• Welding Proficiencies• Video Projects• Knife Proficiencies• Automotive Mechanical

Skills• Processes

Tests / Surveys

Used to assess

• Quantitative Reasoning• Critical Thinking• Conceptual Knowledge• Problem Solving• Student Perceptions• Institutional Climate

UA-PTC Rubric Example

UA-PTC Rubric Example

Institutional Learning Outcomes

Critical Thinking

Program Learning Outcomes

Currently none because this is a General Education course.

Course Learning Outcomes

Demonstrate an ability to think critically and creatively by writing well-argued, well organized essays that articulate a solid thesis…

UA-PTC Checklist Example

UA-PTC Checklist Example

Institutional Learning Outcomes

Technology Literacy

Program Learning Outcomes

The AP 1 & 2 student will demonstrate intermediate microscopy skills in order to locate and identify designated tissues/cell structures.

Course Learning Outcomes

The student will demonstrate intermediate microscopy skills in order to locate and identify designated tissues/cell structures.

UA-PTC Test Example

UA-PTC Test Example

Institutional Learning Outcomes

Quantitative Reasoning

Program Learning Outcomes

Currently none because this is a General Education course.

Course Learning Outcomes

The AP 1 & 2 student will demonstrate intermediate microscopy skills in order to locate and identify designated tissues/cell structures.

The EAC Benefits

Data Collection

Collect Rubric / Checklist Data

• Across Sections• Comparing Course

Deliveries• Across Multiple Courses

for Program Assessment

Advanced Goals Alignment

Add / Remove Multiple Goal Alignments

• Rubric Rows• Test Questions

Advanced Test Statistics

Advanced Item Analysis• Download Table of Student

Answers• View Correct and Incorrect

Answers in Table Format

Using The Data

Using The Data

Anatomy & Physiology I & II Same Student Averages

82.49%

82.49%

89.21%

76.09%

84.07%

84.07%

82.21%

86.14%

70% 75% 80% 85% 90%

BIOL.AA.CLO.01.00

BIOL.AS.CLO.01.00

ILO.02.00

ILO.05.00

Spring 2017 Fall 2016

Q & A

Corrie Bergeron (Remote)

Stephanie RichterAssistant Director, Faculty Development

and Instructional Design Center

Northern Illinois University

srichter@niu.edu

@slrichter

Jenna DulakDirector of Web Services

and Online Technology

Hilbert College

jdulak@hilbert.edu

@WNYBUG

Corrie BergeronInstructional Designer,

Blackboard System Administrator

Lakeland Community College

cbergeron@lakelandcc.edu

@skydaddy

Terry PattersonCurriculum, Assessment,

and Technology Specialist

Pulaski Technical College

tpatterson@pulaskitech.edu

@terrypatterson

Contact Info

Don’t forget

to rate this

session in the

BbWorld app.