+ All Categories
Home > Documents > Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment:...

Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment:...

Date post: 05-Apr-2020
Category:
Upload: others
View: 6 times
Download: 0 times
Share this document with a friend
79
Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement For Presentation at the Nassau Community College 2014 Assessment Symposium Presenter: Michael A. Heel Assistant Director of Curriculum and Assessment, Monroe Community College, SUNY President, Assessment Network of New York (ANNY)
Transcript
Page 1: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Curriculum Mapping and Assessment:

Utilizing Effective Curriculum Design

Principles for Improvement

For Presentation at the Nassau Community College

2014 Assessment Symposium

Presenter: Michael A. Heel

Assistant Director of Curriculum and Assessment, Monroe Community College, SUNY

President, Assessment Network of New York (ANNY)

Page 2: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

•SECTION ONE

Page 3: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Brief Overview

•Assessment & Program Evaluation:

Where We Are Today.

Page 4: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Assessment:

Everyone’s Favorite Task

• What started out as a paradigm of research in education is

now focused on accountability

• Specialized accreditors are changing their approaches and

standards

• Middle States (MSCHE) is “upping the ante”

• The Feds are coming (parents, too)!

• As budgets tighten, “assessment for accountability”

becomes an even bigger deal…

Page 5: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

What Middle States says

• Assessment processes are not about the faculty

(assessment results cannot reasonably be used for hiring,

promotion, or tenure decisions)

• Assessment is not about the numbers (accrediting bodies

generally don’t care about the statistics; they want to

know how we are using that information to make

strategic choices)

• Assessment should not be the top priority of a faculty

(“What are faculty members not doing while they are

engaging in excessive assessment activities?” – Suskie)

Page 6: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Question: Why do “we” conduct

academic assessment projects?

• A) Because the Middle States and other specialized

accrediting bodies (ABET, AACSB, etc.) require it;

• B) Because other external stakeholders (like donors,

trustees, grant applications) want us to;

• C) Because the federal government wants institutions of

higher education to be more mindful of how college students

are educated;

• D) Because we want to avoid…..THIS:

Page 7: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at
Page 8: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

The Brighter Side: Utilizing

Assessment for Improvement

• Ideally, assessment processes are NOT all about the

numbers and statistics, but about the content and

context of the educational product (learning)

• Assessment can help department faculties organize

systematically some of the business they are already

conducting, and provide a sensible framework for those

endeavors

• Even accrediting bodies are now focusing on the “back

end” of assessment, rather than the “front end…”

Page 9: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

TIME-OUT: The Agony and Ecstasy

of Assessment

• What have been your experiences with assessment,

good and bad?

• In your experience, what has been the relationship

of assessment work to all the other responsibilities

you carry in your professional position?

Page 10: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

•SECTION TWO

Page 11: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

How does assessment relate to the

curriculum?

• Assessment processes, designed and executed carefully

and purposefully, offer the faculty important insights

• Inform the faculty about student learning: patterns and

trends, strengths and weaknesses

• Engage the faculty in discussions about teaching and

learning

• Inspire faculty members and departments to make

curricular improvements to programs and courses

• Assessment is not about the numbers and data, but about

information and what we do with that information

Page 12: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Curriculum design as foundation

for assessment

• Assessment as a process cannot be useful without a curriculum that is well-designed, with

• Program-level mission, goals, objectives, outcomes

• Complete sets of course learning outcomes (CLOs)

• Well-written course learning outcomes (CLOs)

• Why is course and program design important to assessment?

• Integrity of assessment relies upon a curriculum with a sound foundational design

• When design is flawed, assessment will automatically be rendered less relevant

Page 13: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

What constitutes a “well-

designed curriculum?”

• As one might expect, the best curriculum design is one

that is NOT taking a cookie-cutter approach

• Specific curriculum designs need to express the intrinsic

qualities and characteristics of their respective disciplines

• Some programs truly require rigidity in curriculum design

• Most programs can be designed with a curriculum that

possesses appropriately unique character, but that also

meet common institutional standards

Page 14: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Criteria for a “well-designed

curriculum”

• Design of program and/or course is:

• Consistent

• Clear to all stakeholders

• Reflective of what is taught and learned

• Responsive to the educational needs of students

• Result of professional collaboration

• Process-driven

Page 15: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Necessary Elements of Program

Design, Associate’s & Bachelor’s

Degrees

• Mission Statement

• Program-level goals and outcomes

• Course teaching objectives and learning outcomes

• An “aligned curriculum”

• (The same is essentially true for general education)

Page 16: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Relationship of Curriculum Elements

(broad to narrow)

Mission Statement

Program-level Goals and Outcomes

Course teaching objectives and

learning outcomes

Page 17: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

DETOUR: The Language of

Curriculum & Assessment

• The language of curriculum & assessment is still not

uniform

• Differentiating “goals” from “objectives” and “outcomes”

can be difficult and irritating

• What about “mission” versus “vision,” “values,” or

“principles?”

• Find a sensible internal language standard, and develop your

own internally consistent approach

Page 18: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Terms for this presentation (1):

• Program-level goals

• defined as the faculty’s aspirations, broadly expressed,

for the program’s operation; linked and supporting the

program mission statement; not assessed

• Program Learning Outcomes (PLOs)

• Statements summarizing specific summative learning

targets that students are expected to meet cumulatively

over the course of their educational experience

Page 19: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Terms for this presentation (2):

• Course Learning Outcomes (CLOs)

• Statements summarizing specific summative learning targets

that students are expected to meet cumulatively upon the

completion of a particular course

• Student Learning Outcomes (SLOs)

• Statements summarizing specific summative learning targets

that students are expected to meet – a generic, umbrella term

that may apply to a single course, several courses, or a

student’s entire institutional experience

• Course Teaching Objectives

• Statements that frame the instructor’s aspirations of learning

for the student, and that express the learning environment

and/or the learning experience the student will have in a

particular course

Page 20: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Example: PLOs: Biotechnology A.S.

Degree Program

A graduate of the Biotechnology Program will be able to:

• 1. Demonstrate effective written communication skills.

• 2. Demonstrate effective oral communication skills.

• 3. Solve problems related to biological topics.

• 4. Apply computer skills to routine laboratory applications.

• 5. Describe foundation concepts in the discipline of molecular biology.

• 6. Discuss foundation concepts in the discipline of biochemistry.

• 7. Utilize core molecular biology techniques commonly employed in a research/industry laboratory.

• 8. Conduct appropriate experimental protocols.

• 9. Analyze and interpret experimental data.

• 10. Operate laboratory equipment commonly used in a research/industry setting.

Page 21: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Example: Field Studies in Biology

(Teaching Objectives)• 1. This course will provide students the opportunity to experience conditions, equipment and methods used by scientists to study biology in the natural environment.

• 2. The instructor of the course will promote the development of skills necessary to construct a scientific field notebook detailing observations, data, and experiments conducted in the field.

• 3. In this course, students will have the opportunity to conduct and report results of experiments conducted in the field.

• 4. This course and the instructor will promote the development of skills required to draw inferences based on observations made in the field.

Page 22: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Example: Field Studies in Biology (CLOs)

• Upon the successful completion of this course, students will be able to:

• 1. Describe the physical, biological, or chemical factors that help shape a

particular environment studied.

• 2. Make and record observations of natural biology, geology, chemistry, or

atmospheric features of a particular habitat or environment.

• 3. Develop inferences and explanations for observations made in the field.

• 4. Demonstrate the proper use of scientific equipment used for the collection

of field data.

• 5. Construct a scientific field notebook that fully documents observed field

phenomenon.

• 6. Execute a field-based scientific experiment.

• 7. Report the results of a field-based scientific experiment.

• 8. Identify and describe significant organisms common to the environment

studied.

• 9. Discuss the distribution and the relative abundance of organisms observed

in the environment studied.

Page 23: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Course Learning Outcomes – “Where

the Rubber Hits the Road”

• What is the purpose of having CLOs?

• Convey to external and internal stakeholders an honest representation of the common learning that will occur in each course (and each section), regardless of who teaches it

• Middle States calls this our contract with students, parents, and the community served

• Common content should be agreed-upon by participating faculty

• CLOs as presented on syllabi should serve as the foundation for course organization

Page 24: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Section Summary

• Mission Statements and Program-level Learning

Outcomes (PLOs) frame the broad structure, aspirations,

and expectations of degree programs (and general

education)

• Course Learning Outcomes (CLOs) provide the basis for

specifying specific expectations for student learning

• CLOs should support and link to PLOs in meaningful

ways.

Page 25: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

•SECTION THREE

Page 26: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Now What?

• The basis of a sound program curriculum includes both

design and assessment

• Curriculum mapping and alignment provide the gateway

to meaningful assessment processes

Page 27: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Curriculum Alignment as a process

gives us the chance to:

• Connect what we do in the classroom with what’s going

on the real world

• Demonstrate that students are learning what we intend for

them to learn

• Understand whether or not the courses we are teaching

substantively support the curriculum we have designed

• Pause and consider whether the education we are

providing supports the needs of our constituencies

Page 28: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Sounds good, but why bother?

• Busy program and department faculties run the risk of

operating their courses and programs on “auto-pilot.”

• “It ain’t broke!”

• “The assessment results show we’re doing okay…”

• “I’m doing assessment, I’m doing program review,

what more do you want from me?”

• Even the most invested faculty members may seem

reluctant to add (what are perceived to be) extra steps in

the assessment and program review process

Page 29: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

When skeptic becomes problem-

solver…

• No one likes doing extra work

• BUT, what if that work, once undertaken, winds up being

valuable, and offers up results that are immediate and

worth the time spent?

• The first-time investment requires the greatest effort

• The value of doing academic assessment is not always

apparent – curriculum alignment often clarifies that value

(remember – “useful” and “truthful”)

Page 30: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

“A well-designed curriculum map is one of the most

effective tools you can create for documenting how your

program-level outcomes are supported by your

curriculum.”

-- Michael Middaugh

Retired Chair, MSCHE Executive Committee

A Basic Toolbox for Assessing Institutional Effectiveness

Cranberry Township, PA, August 2010

Page 31: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Initiating the Mapping Process

• Distribute a numbered/lettered list of Program Learning Outcomes among the relevant faculty members

• Challenge each faculty member:

• For each class he/she teaches in the program, determine which of the PLOs is covered in the class as a MAJOR part of the course content; indicate also MINOR coverage (separately)

• This exercise benefits from input from multiple faculty members who teach different sections of the same course, but who respond independently

• Collect and display the information in grid format

Page 32: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Clear, right?

How about an example?

Page 33: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Case Study

Beekeeping

at Cranmore Community College

Page 34: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Example: Program Learning Outcomes

A graduate of the A.A.S. degree program in Apicultural Science will be able to:

a. Construct and maintain beehives of appropriate design relative to the beekeeper’s purpose (breeding, honey production, supporting agriculture, etc.) and the breed of bee;

b. Create appropriate breeding environments and apply professionally-recommended techniques in promoting healthy fertility and growth of bee populations;

c. Apply and interpret basic genetic tests of bee samples;

d. Conduct appropriate hybridization techniques;

e. Identify, recognize, and respond to insect behaviors, including those of bee, competing insect, and predator insect species;

f. Promote health and wellness among bee populations;

g. Cultivate plant species supportive of the healthy maintenance of bee populations;

h. Recognize and manage responses to threats to bee populations, such as changes in the seasons, extreme weather conditions, or the presence of pesticides and other toxins in the environment;

i. Apply basic business management principles to the management of the costs and expenses of beekeeping;

j. Discuss the ethical principles underpinning beekeeping as both a hobby and a profession;

k. Describe the role of bee species in the maintenance of a healthy and sustainable agricultural business;

l. Teach and train apprentice beekeepers in the rudimentary aspects of the practice of beekeeping;

m. Capably communicate the basic tenets of good beekeeping practices to individuals unfamiliar with bees and the practice of beekeeping.

Page 35: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

What Are We Looking For?

• The primary reason we construct a curriculum map is to be

able to formulate a picture of program design that helps us

discern patterns of coverage

• Look for gaps – are PLOs all covered?

• Look for reinforced learning – are PLOs covered

“sufficiently?”

• Look for redundancy – are some PLOs getting too much

attention, at the cost of other curriculum content?

• Look for course purposefulness – are particular courses

seemingly adding very little to the curriculum?

• Look for course content overload – sometimes, less really

is more!

Page 36: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

PLO

APS

101

ENV

111

APS

112

ENV

172

APS

122

ENV

231

APS

201

BIO

205

APS

244

APS

260

APS

252

a

b

c

d

e

f

g

h

i

j

k

l

m

Page 37: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

PLO

APS

101

ENV

111

APS

112

ENV

172

APS

122

ENV

231

APS

201

BIO

205

APS

244

APS

260

APS

252

a M m M M

b m M m m M M

c m m

d M m m m

e M m

f m m m M m m m M

g m M M m

h m m m M m m M M

i m M m m M M

j m M m

k m M m m

l m

m

Page 38: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

TIME-OUT: Group Assessment

•WHAT DO YOU THINK?

Page 39: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Preliminary Review Indicated:

• Program Learning Outcome “m” is not covered

• Faculty discussion revealed that virtually every faculty

member thought others were covering the outcome in

their respective courses

• Most faculty members did not feel qualified to teach or

grade students on their communication skills

• ENV 111 and APS 112 seemed of little value to the

curriculum

• PLO’s “c,” “e,” and “l” are under-covered

Page 40: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

But wait, there’s MORE

• The curriculum mapping exercise ALSO picked up some

areas where there were redundancies, and where courses

seemed over-packed

• Outcomes “f” and “h” were covered in almost every

course, and Outcomes “b” and “g” seemed also to be

overly emphasized given their relative importance to the

whole curriculum

• Four courses (APS 244 and APS 260) seemed too densely

packed with redundant content

Page 41: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Follow-up on the initial mapping

• The program faculty is now both empowered and

challenged to “fix” these curriculum issues

• Benefit of this simple procedure is that the pictorial

representation gives participants a quick and easy

understanding of an otherwise complex system

Page 42: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Changes to Curriculum

• Outcome “m” now covered in an APS course each semester

of a student’s plan of study

• Outcomes “c” and “d” were dependent upon BIO 205, so

courses scheduled in the second year had to address these

outcomes

• Significant course redesigns for APS 112, APS 244, APS

260, and APS 252 to accommodate and address both gaps

and redundancies in curriculum, particularly Outcome “l”

• ENV 112 retained, to fulfill Natural Science requirement,

and because it is a pre-req for ENV 172

• Moving forward, the faculty must now document student

learning

Page 43: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

PLO

APS

101

ENV

111

APS

112

ENV

172

APS

122

ENV

231

APS

201

BIO

205

APS

244

APS

260

APS

252

a m M m M m

b m M m m M M

c m M M

d M M m M m

e M m M

f m m m M m m m M

g m M M m

h m m M M m m M M

i m M m m M M

j m M m M

k m m M m m

l M M m

m M M M M

Page 44: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Ascertaining an Assessment Strategy

for Courses

• Above all else, assessment should be useful and efficient, so

think strategically!

• Not all CLOs for all courses need to be assessed (although at

some point, the program faculty might consider assessing

more broadly with purpose)

• Some courses are stronger in some areas of interest than

others

• Although multiple courses may cover the same outcome, not

every course provides good assessment opportunities

• Some courses offer breadth over depth

• Some PLO’s can only be covered (and measured) in specific

courses

Page 45: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

PLO

APS

101

ENV

111

APS

112

ENV

172

APS

122

ENV

231

APS

201

BIO

205

APS

244

APS

260

APS

252

a m M M m

b m M m m M M

c m M M

d M M m M m

e M m M

f m m m M m m M

g m M M m

h m m M M m m M

i m M m m M M

j m M m M

k m m M m m

l M M m

m M M M M

Page 46: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Final “Linkage” Documented

• The faculty has to determine which Course Learning

Outcomes best suit the matching PLO

• The assumption is that individual courses are taught

according to their respective CLOs

• If CLOs are supported by graded (and thus, embedded)

student assignments, then that documentation will directly

demonstrate support for the PLO (MSCHE wants direct

evidence where possible)

• Common sense rules for assessment still apply (as do the

broad MSCHE assessment principles)

Page 47: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

PLO

APS

101

ENV

111

APS

112

ENV

172

APS

122

ENV

231

APS

201

BIO

205

APS

244

APS

260

APS

252

a clo #2 clo #1

b clo #6 clo #1

c clo #5 clo #4

d clo #4 clo #3

e clo #1

f clo #1 clo #2

g clo #8 clo #2

h clo #3 clo #2

i clo #5 clo #3

j clo #1 clo #7

k clo #3 clo #4 clo #2

l clo #6 clo #4

m clo #7 clo #9 clo #11

Page 48: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Bringing everything together

• Note that all APS courses are participating in the

assessment of at least one Course Learning Outcome

• All PLO’s are covered by at least one assessed course

• Remember that MSCHE cares only that the Department

faculty demonstrate that PLO’s have been assessed; they

aren’t concerned about the minutiae

• This process allows the faculty to show that, by fulfilling

linked CLOs, they are likewise showing that PLOs have

been measured and analyzed

Page 49: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Inputting and Interpreting Assessment

Results

• Program faculty set a benchmark standard of learning of

80% for each outcome

• Based on historic rates of success in the program

• Faculty agreed that:

• Highest result would count, unless later results showed

a backward trend of learning

• If result < 80%, they would discuss

• If result < 60%, then this outcome would need

immediate and special attention

• If result > 95%, check for integrity & rigor

Page 50: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

PLO

APS

101

ENV

111

APS

112

ENV

172

APS

122

ENV

231

APS

201

BIO

205

APS

244

APS

260

APS

252

a 88% 95%

b 82% 100%

c 58% 69%

d 55% 73%

e 69%

f 94% 92%

g 40% 68%

h 77% 80%

i 61% 80%

j 49% 65%

k 55% 80% 88%

l 60% 80%

m 38% 52% 78%

Page 51: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

TIME-OUT: Group Assessment

•WHAT DO YOU THINK?

Page 52: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Conclusions from Assessment

• Outcomes a, b, f, are in very good shape

• Outcomes h, i, k, l looking good too

• Outcomes c, d, e, g, j, m all discussed

• What’s up with APS 244?

Page 53: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Unexpected benefits

• As with many assessment activities, once the group

moves beyond the “worker bee” focus (pardon the pun),

the opportunity to consider the bigger picture is usually

gratifying and engaging

• Relationship and roles of individual courses within a

curriculum are greatly clarified

• The benefit extends also to other stakeholders, who can

now be reassured that a program curriculum “delivers” on

what is advertized

• This particular exercise can help build faculty consensus,

teamwork, and identity

Page 54: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

•SECTION FOUR

Page 55: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Case Study

•Religious Studies A.S. Degree at

Franklin Community College

Page 56: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Another Example--

• Consider the following application of curriculum

mapping (based on an actual experience; the discipline

has been changed, and the institution remains

anonymous):

• Department of Philosophy and Religion at a 2-year

institution offers a major, but serves far more students

who take courses to satisfy the institution’s general

education humanities requirement

• What should that faculty’s approach be to assessment?

• How can the process of curriculum mapping help the

faculty to figure this out?

Page 57: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Their Mission Statement

• The Mission of the Religious Studies degree program is

to provide students with a complete educational

experience. Students majoring in Religious Studies will

grow intellectually, civically, emotionally and spiritually.

Students graduating with a degree in Religious Studies

will be well-rounded individuals capable of completing a

B.A. degree, will have a tolerant world view, and will

have knowledge of a variety of cultures and belief

systems, allowing them to capably interact with people of

wide and varying backgrounds.

Page 58: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Program Goals

• To offer students an understanding of the basis for

spiritual thought;

• To develop in students an understanding and respect for

religious thinking, both western and non-western

• To provide students with an understanding of the

historical developments of world religions

• To expose students to a variety of ethics models

• To help students become better citizens

• To promote strong emotional and spiritual health among

students

Page 59: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Program Learning Outcomes

Graduating students with an Associate’s degree in Religious Studies will

be able to:

• 1) Interpret, analyze and evaluate literature, founded in both religious

faiths and secularism, in terms of the spiritual and ethical standards from

within the society in which that literature is based.

• 2) Interpret, analyze and evaluate public discourse, in all media forms,

in terms of the spiritual and ethical standards from within the society in

which that discourse occurs.

• 3) Discuss the main historical developments of at least four of the

following world religious paradigms: Judaism, Christianity, Hinduism,

Islam, Buddhism.

• 4) Identify cultural differences based on unshared historical religious

experiences.

• 5) Differentiate between philosophical and religious thinking.

• 6) Assess how human spiritual pursuits have an impact on cultural,

economic and political developments of societies.

Page 60: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Curriculum Map for the Religious

Studies Program (initial view)

PLO

REL

101

REL

102

REL

112

HIS

140

PHL

132

REL

201

REL

210

REL

175

REL

220

PHL

211

ANT

138

1 m M M M m M m M M m M

2 m M m m M m M m M m M

3 M m m M M M M m m

4 m M m M M M m M M

5 m m M M M m M M m

6 M m m M M M M M m

Page 61: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

TIME-OUT: Group Assessment

•WHAT DO YOU THINK?

Page 62: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Initial Reactions

• Of the faculty members reviewing the map, most

expressed concern and confusion

• One senior faculty member pronounced the map as an

example of “complete success…we can go home

now!” (and that was his sincere reaction…)

• A colleague lamented: “We seem to be all teaching the

same basic course over and over and over.”

• Most agreed that this map wasn’t especially helpful,

unless it really did portray extended redundancy in the

curriculum, in which case…

• “We need to seriously rethink something…”

Page 63: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

First Response

• Analytically speaking, the tool is the tool – it can only be

flawed if it is either being used improperly, or

information is incorrect

• Problem could be with Program Learning Outcomes

(too few; too many combined learning targets)

• Faculty may be overestimating coverage of learning

outcomes

• Map can only be useful if it portrays accurate picture of

well-stated learning outcomes

• Faculty opted for “new look” at the information

Page 64: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Curriculum Map for the Religious

Studies Program (second view)

PLO

REL

101

REL

102

REL

112

HIS

140

PHL

132

REL

201

REL

210

REL

175

REL

220

PHL

211

ANT

138

1 M M M M M M M

2 M M M M M

3 M M M M M

4 M M M M M M

5 M M M M M

6 M M M M M M

Page 65: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Second View Results

• Faculty considered just the points in the curriculum where

there was “major” coverage of each outcome present in a

course

• Patterns that emerged were useful, interesting, and

revealing

• At first, faculty members considered making revisions to

courses, except that discussion revealed common threads

of concern

• PLOs hard to assess

• How much learning is enough (and is it “right”

learning?)

• Faculty opted for yet another look at the map…

Page 66: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Curriculum Map for the Religious

Studies Program (“IRA” view)

PLO

REL

101

REL

102

REL

112

HIS

140

PHL

132

REL

201

REL

210

REL

175

REL

220

PHL

211

ANT

138

1 iiii IIII IIII IIII iiii RRRR rrrr IIII RRRR rrrr IIII

2 iiii IIII IIII iiii RRRR rrrr RRRR rrrr AAAA rrrr IIII

3 IIII iiii rrrr IIII RRRR RRRR IIII rrrr rrrr

4 iiii IIII RRRR RRRR RRRR RRRR iiii RRRR IIII

5 iiii iiii IIII RRRR RRRR rrrr IIII RRRR IIII

6 IIII iiii iiii IIII IIII AAAA IIII RRRR rrrr

Page 67: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

New Response

• This map did provide the faculty with evidence needed

for change, both in the design of courses, and in the

design of PLOs

• Evidence that PLOs were too complex

• Evidence that student learning was “stuck” at the

introductory level for too long, and not enough

instruction was at the “advanced” level

• Faculty opted to re-visit PLOs

Page 68: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

New PLOs for Religious Studies A.S. Program

• 1. Interpret religion-based literature in terms of the cultural standards of the society from which that literature originates.

• 2. Interpret public discourse on religious or spiritual topics as portrayed in various media forms.

• 3. Analyze or evaluate secular literature written with spiritual or religious themes.

• 4. Discuss the main historical developments of: a) Judaism; b) Christianity; c) Hinduism; d) Islam; e) Buddhism

• 5. Describe the major tenets of: a) Judaism; b) Christianity; c) Hinduism; d) Islam; e) Buddhism

• 6. Identify cultural differences based on unshared historical religious experiences.

• 7. Differentiate between philosophical and religious thinking.

• 8. Explain how human spiritual pursuits have an impact on cultural, economic, or political developments within societies.

• 9. Self-reflect on personal spiritual or religious beliefs and one’s connection to others.

Page 69: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Here’s the new series of curriculum maps

from the Religious Studies A.S. program…

Page 70: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

PLO

REL

101

REL

102

REL

112

HIS

140

PHL

132

REL

201

REL

210

REL

175

REL

220

PHL

211

ANT

138

1 m M M M m M m M M m M

2 m M m m M m M m M m M

3 M M m m m m M M

4a m M m M m

4b m M m M m m

4c m M M

4d m m m M M M

4e m M M

5a M M m m m

5b M M m m M M m

5c M m M m

5d M m m M M m

5e M m M m

6 m M M M M M M M M

7 m m M M M m M M m

8 M m M M M M M m

9 M M M m

Page 71: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

PLO

REL

101

REL

102

REL

112

HIS

140

PHL

132

REL

201

REL

210

REL

175

REL

220

PHL

211

ANT

138

1 M M M M M M M

2 M M M M M

3 M M M M

4a M M

4b M M

4c M M

4d M M M

4e M M

5a M M

5b M M M M

5c M M

5d M M M

5e M M

6 M M M M M M M M

7 M M M M M

8 M M M M M M

9 M M M

Page 72: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

PLO

REL

101

REL

102

REL

112

HIS

140

PHL

132

REL

201

REL

210

REL

175

REL

220

PHL

211

ANT

138

1 i I I I i R r I R r I

2 i I i i R r R r A r I

3 I I i i i r R I

4a i I i R r

4b i I i R r r

4c i I R

4d i i i I R A

4e i I R

5a I I i i i

5b I I i i R A I

5c I i R i

5d I i i R A i

5e I i R I

6 i I R R R R I R I

7 i i I R R r I R i

8 I i i I I A I R i

9 I I R i

Page 73: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

PLO

REL

101

REL

102

REL

112

HIS

140

PHL

132

REL

201

REL

210

REL

175

REL

220

PHL

211

ANT

138

1 I I I R I R I

2 I R R A I

3 I I R I

4a I R

4b I R

4c I R

4d I R A

4e I R

5a I I

5b I I R A I

5c I R

5d I R A

5e I R I

6 I R R R R I R I

7 I R R I R

8 I I I A I R

9 I I R

Page 74: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Still Not Done….

• Concerned about the coverage of some key concepts, the

faculty considered their options

• They decided to create an additional course, a capstone,

that would provide the added level of learning that they

thought their students deserved and needed

• The number of elective courses was reduced by one to

accommodate the change without increasing the number

of credits

Page 75: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

REL 250 -- Capstone Course in Religious

Studies

Upon successful completion of this course, students will be able to:

• 1. Discuss in detail the primary tenets of the world’s major religions,

including: a) Judaism, b) Christianity, c) Hinduism, d) Islam, and

e) Buddhism

• 2. Review critically literature of a religious or spiritual nature, including works such as the Bible, the Talmud, the Quran, or others.

• 3. Assess non-fiction works with religious foci for analytical rigor and factual accuracy

• 4. Discuss the impact of religious movements on the political cultures of selected nations

• 5. Engage in oral debate on religious topics

• 6. Express personal spiritual beliefs or disbeliefs in a reflective manner

• 7. Practice reflective listening in discussions related to religious or spiritual beliefs of others

Page 76: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

PLO

REL

101

REL

102

REL

112

HIS

140

PHL

132

REL

201

REL

210

REL

250

REL

175

REL

220

PHL

211

ANT

138

1 I I I R R I R I

2 I R R R A I

3 I I R R I

4a I R

4b I R

4c I R

4d I R A

4e I R

5a I R A

5b I R R A A I

5c I R A

5d I R A A

5e I R A I

6 I R R R R I R I

7 I R R I R

8 I I I A R I R

9 I I R R

Page 77: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Issues with the Religious Studies

Dept. Approach

• Mission and Goals are part of identity, and so, cannot

easily be criticized

• Department seemed to ignore non-majors in their set of

goals and outcomes, however

• Program Learning Outcomes not well-developed initially

– led to confusion and (perhaps) over-confidence

Page 78: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

BUT, the process still helped…

• Faculty recognized that their PLOs needed attention

• More importantly, the faculty recognized that, even with

more well-developed PLOs, their courses were

overlapping significantly

• The program faculty successfully engaged in an

intellectually honest and complete inquiry about their

degree program.

• Curriculum mapping triggered a wholesale

reconsideration of the program’s curriculum and focus

Page 79: Curriculum Mapping and Assessment: Utilizing Effective ......Curriculum Mapping and Assessment: Utilizing Effective Curriculum Design Principles for Improvement ForPresentation at

Concluding Remarks

•Questions/Comments/Criticisms?


Recommended