An experiment in institutional change

1. Motivation2. Goals and brief description3. Some early results and observations4. Underlying ideas/rationale for design

A new model for transforming the teaching in post-secondary education focus on science but likely general

Science (physics) education research results(as branch of physics research)

• Quantifying failures of traditional teaching and assessment practices

•Good validated assessments in widespread use (conceptual mastery)

•Identifying specific student difficulties (“misconceptions”) and how to address

• Quantifiably and reliably better instructional approaches -- everyone, not just future scientists•Effective uses of technology

consistent with principles from learning research

cognitivepsychology

brainresearch

classroomstudies

Major advances past 1-2 decadesConsistent picture Achieving learning

On average learn <30% of concepts did not already know.Lecturer quality, class size, institution,...doesn't matter!Similar data for conceptual learning in other courses.

R. Hake, ”…A six-thousand-student survey…” AJP 66, 64-74 (‘98).

• Force Concept Inventory- Force Concept Inventory- basic concepts of force and motion 1st semester physics

Fraction of unknown basic concepts learned

Average learned/course 16 traditional Lecture courses

Measuring conceptual mastery

Ask at start and end of semester--What % learned? (100’s of courses)

improvedmethods

How to make these more effective research-based teaching practices the norm in every post-secondary classroom?

The goal ofCUSEI 3 yrsCWSEI 2 yrs

1. Detailed goals2. Summary of results3. How being implemented4. Design principles5. Lessons learned

“There is nothing more difficult to take in hand, more perilous to conduct, or more uncertain in its success, than to take the lead in the introduction of a new order of things. For the reformer has enemies in all those who profit by the old order, and only lukewarm defenders in all those who would profit by the new order, this lukewarmness arising partly from fear of their adversaries … and partly from the incredulity of mankind, who do not truly believe in anything new until they have had actual experience of it.”– Machiavelli

SEI finding #1IS EXACTLY CORRECT-- true for faculty members, administration, students, alums, funders, ...

Attempting to change an ancient culture

Changing major research universities set standards for teaching & learning science

Focus on the departmental level decide what and how to teach in scienceunit of change

1. Detailed goals

1st: Decide on learning goals. (what should students be able to do?)

2nd: Rigorously assess student learning on anongoing basis.

3rd: Apply research-based teaching techniques to improve student learning. Measure progress!(technology to improve effectiveness & save time)

Ensure materials, data, assessment tools, software,… saved, reused, improved. (like scientific research)

Department- For each course & program offered

“SEI Holy Trinity”

semi-hidden change

Student learning to become departmental ratherthan individual instructor responsibility

Ultimate goal…SEI model for all universities

Incentives to copy?1) convincing data that improves student learning2) responds to calls for accountability-- shows whatstudents are learning per course and program

How we make it cheaper for others?•online data base of new designed courses.materials, assessment tools, evidence, etc. freely available.

2. Summary of results --what has been accomplished?U. Colorado--3 years into 5-6 year plan.

Int. Physio, Geo. Sci., MCDB-- Over 80% of faculty have some involvement. About 60% have significantly changed how they teach. ~80% of their undergrad credit hours.

Physics and Chemistry & Biochem-- Substantial impacts, many faculty members.

•Establishing learning goals (what students can do)-- better guidance to students & coherence in curriculum•Rigorous assessments: learning & student attitudes •Improved teaching methods-- engagement, interactivity, feedback, higher level thinking•TA training and faculty development•Research on learning: content, concepts, and attitudes

Improved 53 courses, > 10,000 student-courses/year

What accomplished at UBC (2 years into 6 year plan, ramping up incrementally)

After 2 years, dramatic progress in Earth and Ocean Sciences Dept. at UBC. Most faculty, many courses changing.

Several other departments ramping up

Not always so successful!

Some depts. much slower progress

Some science departments not participating or onlydoing small projects

3. How SEIs being implemented

• 5 departments, selected competitively on basisof proposals. Commitment and readiness to pursue “SEI Holy Trinity” & sustainability plans/structure.

• Focused $$$ ($1-2 M) and guidance to Department.Support over 6 yrs-- then self-sustaining (no ongoing extra $$)

CW--“Hands-semi-on” funding agency

one-time investment permanent impact$5 M at Colorado, $12 M Cn at UBC

“Science education specialist” approachnot required, but dominates $$$ and activity.

SES expert in discipline, hired by department, trained in sci. ed., research and assessment (by me).

Work collaboratively with 1-3 faculty to implement changes in teaching and assessment for a course (SEI Trinity).

Provide 1) Expertise2) Labor (design, implement, assess, sustain)3) Push research agenda and methods

4. Design principles of the SEIs

• Many elements common to any organizational change

•Incentives at level of both the individual and the relevant unit for change

•Change requires investment (time, ... =$$$)

What is the unit for change?

The department (in research university).

Decide what gets taught, who teaches it, how it is taught.

If want change at that scale, must focus resources to match. Major research department budget $20-40 M/yr

•Incentives at individual and departmental level.Otherwise, never overcome ability of old (or young) curmudgeon to block change.

How much should it cost?Can’t you change the culture of a major science dept.for $100-200 K, instead of $1-2 M+?

Change costs time=money. Industry-- major shift involves 5-10% or morereallocation of annual budget.

SEI based on 5% annual budget for 5 yrs-- Looks about right. 1) Gets full department attention,2) Roughly (?) covers cost of labor required to transform most courses (= involve most faculty).

Other underlying ideas--

1. Much easier to get people to adapt existing culture/identity than invent new one.(scientist-- teaching as scientific activity withdata and research based principles and practices)

2. Physics Ed. research has impacted instruction.Primary Drivers:• backed by data and publications,• fairly good easy to use standard assessment tools• topic and course specific proven interventions, many easy to incrementally adopt• advocates who know subject, speak same language

3. Evidence is necessary. (although far from sufficient)

4. Competing for funding drives change MUCHbetter than simply making resources available.

5. Require good measurement of learning, but department decides on what should be learned and develops/accepts the assessment instruments.

Acknowledges their expertise and ownership.

5. What learned so far from SEI experiments?

I. Some aspects working well--• SES model works well• Extensive dept-wide conversations on undergrad ed.• Meaningful learning objectives being developed and assessments being rethought (but slooowly)(but 1/2 the battle)•Many long overlooked opportunities for improvement and streamlining emerging•Faculty members enjoy talking to each other aboutteaching in substantive ways--provide forum•Also enjoy approaching teaching as research•Steadily growing number of converted faculty who can be wonderful advocates

II. some major challenges (ideology)1. Tyranny of content-- data only way to respond (?)

2. Threatening to a few faculty in variety of ways

3. “Students are lazier and less well prepared than last generation” (4000+ years and counting)

4. Individual course ownership

5. “Better instruction ≠ more work, more $$!”Efficiencies are possible. Its ok to use other’s material.

6. Incentive systems not connected to better studentlearning or using best practices.

7. Discipline related cultural oddities (from N=2 )

chemistry

intro

biology

?

geology

Computer science

C.S. education???

abcd

xCan’t teach x, until cover a, b, c, d ...t, u, v, and w!

Physics

III. Some neutral lessons

1. Lots of little stuff is important.•Teaching assignments•TA requirements and training, ...

2. Very high degree of similarity and challengesacross the two universities and the various disciplines.

What basic differences there are between universities are smaller than differences between disciplines.

a work in progress-- stay posted

www.cwsei.ubc.ca (extensive set of resources for faculty and students,course archive (sei.ubc.ca) where course materialsassessments, etc. preserved for dissemination)

www.colorado.edu/sei/