Date post: | 24-Dec-2015 |
Category: |
Documents |
Upload: | sharlene-mclaughlin |
View: | 224 times |
Download: | 3 times |
OCRI OCRI SymposiumSymposium
OCRI is a joint effort between the Oregon Association of Colleges of Teacher OCRI is a joint effort between the Oregon Association of Colleges of Teacher Education (OACTE) and the Oregon Association of Teacher Educators (ORATE).Education (OACTE) and the Oregon Association of Teacher Educators (ORATE).
“Investigating the efficacy of teacher work sample methodology in teacher
preparation”
AACTE February 25th, 2007
This Symposium is Dedicated to the This Symposium is Dedicated to the Memory and Legacy of Dr. Del SchalockMemory and Legacy of Dr. Del Schalock
Dr. Del Schalock, Professor, The Teaching Research Institute, Western Oregon University
Lead Developer and Advocate of the Teacher Work Sample Methodology (TWS)
Symposium OverviewSymposium Overview IntroductionIntroduction
Paper 1: Collaborative research design in teacher Paper 1: Collaborative research design in teacher preparation: Blueprints and processes preparation: Blueprints and processes
Paper 2: Evolution of policy implementation of teacher work Paper 2: Evolution of policy implementation of teacher work samples in Oregonsamples in Oregon
Paper 3: Perceptions of teacher educators and candidates Paper 3: Perceptions of teacher educators and candidates on the use of teacher work sampleson the use of teacher work samples
Paper 4: Teacher work sample effects on the learning of K-Paper 4: Teacher work sample effects on the learning of K-12 students 12 students
DiscussionDiscussion
IntroductionIntroduction
Paper 1Paper 1
Collaborative research design in Collaborative research design in teacher preparation: Blueprints and teacher preparation: Blueprints and
processesprocesses
Ronald A. Beghetto (University of Oregon)Ronald A. Beghetto (University of Oregon)
Linda Samek (Corban College)Linda Samek (Corban College)
Imagine…Imagine…
A focused, coherent multi-institutional collaborative A focused, coherent multi-institutional collaborative research initiativeresearch initiative Nearly a dozen institutions (public and private)Nearly a dozen institutions (public and private) Engaged in meaningful and sustainable teacher Engaged in meaningful and sustainable teacher
education research.education research.
This vision is a realityThis vision is a reality With several partnerships across the nationWith several partnerships across the nation And, most recently, a new network in the state of OregonAnd, most recently, a new network in the state of Oregon
The Oregon Collaborative Research Initiative (OCRI).The Oregon Collaborative Research Initiative (OCRI).
Blue Prints and ProcessesBlue Prints and Processes
Under what conditions do such partnerships Under what conditions do such partnerships develop? develop?
How do collaborative groups develop and How do collaborative groups develop and carry-out their research agendas? carry-out their research agendas?
What seems to support such efforts? What seems to support such efforts? What challenges do such efforts face? What challenges do such efforts face?
Historical ContextHistorical Context
The Diversity of OACTE InstitutionsThe Diversity of OACTE Institutions 19 institutions19 institutions
6 public universities6 public universities 3 research and/or urban3 research and/or urban 3 regional3 regional
12 private non-profit colleges and universities12 private non-profit colleges and universities 6 religious6 religious 6 secular6 secular
1 for profit university1 for profit university Small to largeSmall to large
From newly approved with no completers toFrom newly approved with no completers to Hundreds each yearHundreds each year
New to oldNew to old Multnomah Bible College program to begin fall 2007Multnomah Bible College program to begin fall 2007 Western Oregon University, preparing teachers since 1882Western Oregon University, preparing teachers since 1882
Precursors to OCRIPrecursors to OCRI
All subject to rules governing programs from the Oregon All subject to rules governing programs from the Oregon Teacher Standards and Practices CommissionTeacher Standards and Practices Commission
Mid 1990s – Oregon Collaborative for Excellence in Mid 1990s – Oregon Collaborative for Excellence in Preparation of Teachers, a 5-year, $5 million NSF Preparation of Teachers, a 5-year, $5 million NSF collaborative that included public, private, and community collaborative that included public, private, and community collegescolleges
1998-2001 – Oregon Quality Assurance in Teaching, a 4-1998-2001 – Oregon Quality Assurance in Teaching, a 4-year, multi-million dollar grant that involved the 16 four-year year, multi-million dollar grant that involved the 16 four-year institutions with teacher preparation programsinstitutions with teacher preparation programs
Common interests in the education of children through Common interests in the education of children through OACTEOACTE
Perfect Storm that Spawned OCRIPerfect Storm that Spawned OCRI
Leaders from all institutions have common goalsLeaders from all institutions have common goals Oregon TSPC requires evidence of competent Oregon TSPC requires evidence of competent
performance and student learningperformance and student learning All student teachers are required to produce All student teachers are required to produce
Teacher Work SamplesTeacher Work Samples More Oregon institutions interested in NCATE More Oregon institutions interested in NCATE
accreditationaccreditation Decentralized education enterprise leaves Decentralized education enterprise leaves
everyone under-fundedeveryone under-funded OACTE had treasury funds that needed to be usedOACTE had treasury funds that needed to be used
Sowing the Seeds of Sowing the Seeds of CollaborationCollaboration
Building a Flexible BlueprintBuilding a Flexible Blueprint ApproachApproach
Listen, synthesize, and verify Listen, synthesize, and verify WorkflowWorkflow
An iterative, multistage process of planning and activityAn iterative, multistage process of planning and activity
Research Designer
CORT
Project Strand Leaders
Local Participants & Partner Sites
Primary Goals of OCRIPrimary Goals of OCRI
The The Oregon Collaborative Research Oregon Collaborative Research InitiativeInitiative (OCRI) (OCRI) was established with was established with the purpose of:the purpose of: Engaging Oregon's teacher preparation Engaging Oregon's teacher preparation
institutions in the collaborative study and institutions in the collaborative study and dissemination of a dissemination of a meaningfulmeaningful and and sustainablesustainable set of research projects. set of research projects.
MeaningfulMeaningful
MeaningfulMeaningful in that the projects address in that the projects address questions of common interest, are questions of common interest, are aimed at improving teacher aimed at improving teacher preparation, and have real-world preparation, and have real-world implications for teacher development implications for teacher development and PK-12 student learning. and PK-12 student learning.
SustainableSustainable
SustainableSustainable in that the research in that the research projects do not overburden projects do not overburden participants but rather offer feasible participants but rather offer feasible and rewarding opportunities to engage and rewarding opportunities to engage in and disseminate the findings of in and disseminate the findings of collaborative inquiry.collaborative inquiry.
Collaborative FocusCollaborative Focus
Our initial collaborative endeavor has Our initial collaborative endeavor has focused on examining the efficacy of focused on examining the efficacy of teacher work sample methodology in teacher work sample methodology in preparing teachers in the state of preparing teachers in the state of Oregon. Oregon.
TWSM: An Initial ProjectTWSM: An Initial Project
Three interrelated questions:Three interrelated questions: How has the TWSM How has the TWSM evolvedevolved? ? What is the What is the experienceexperience of teacher of teacher
educators and candidates with the TWSM?educators and candidates with the TWSM? What What effecteffect has the TWSM had on PK-12 has the TWSM had on PK-12
students?students?
Example Planning ToolsExample Planning Tools
Evolution StudyEvolution Study
Goal: Goal: Examine how TWSM first became a Examine how TWSM first became a mandate and how implementation has mandate and how implementation has evolved over time.evolved over time.
Brief overview: Brief overview: Using a “Theory of Action” Using a “Theory of Action” framework, juxtapose the enactment of framework, juxtapose the enactment of TWSM with its intended and espoused TWSM with its intended and espoused theory.theory.
Experience StudyExperience Study
Goal: Goal: Document the perceptions of teacher Document the perceptions of teacher educators and candidates regarding TWSM.educators and candidates regarding TWSM.
Brief overview: Brief overview: Participants’ perceptions Participants’ perceptions (representing public and private institutions) (representing public and private institutions) regarding the meaningfulness, usefulness, regarding the meaningfulness, usefulness, and applicability will be examined and and applicability will be examined and reported on.reported on.
Effect StudyEffect Study Goal: Goal: Examine how TWSM has impacted PK-Examine how TWSM has impacted PK-
12 learners.12 learners. Brief overview: Analysis of completed work Brief overview: Analysis of completed work
samples to examine how work samples are samples to examine how work samples are impacting K-12 learning. This effort also impacting K-12 learning. This effort also focuses on the development and refinement focuses on the development and refinement of research/measurement tools most of research/measurement tools most appropriate for analyzing the impact of appropriate for analyzing the impact of TWSM.TWSM.
Supports and Supports and ChallengesChallenges
Key MetaphorsKey Metaphors
““Cyclical scaffolding” and “loose coupling” of Cyclical scaffolding” and “loose coupling” of parallel efforts as metaphors for supporting parallel efforts as metaphors for supporting collaborative research designcollaborative research design Key challenges:Key challenges:
Anticipating and providing “just in time” assistanceAnticipating and providing “just in time” assistance Helping to navigate research strand “autonomy” and Helping to navigate research strand “autonomy” and
“relatedness”“relatedness” Issues and challenges with respect to intellectual Issues and challenges with respect to intellectual
property (e.g., datasets), secondary analyses, and property (e.g., datasets), secondary analyses, and dissemination of findings.dissemination of findings.
Concluding ThoughtsConcluding Thoughts
Even though the Oregon group is still in the early Even though the Oregon group is still in the early stages of this process, stages of this process, the project has already attained no small measure of the project has already attained no small measure of
success. success. Only time will tell what the Oregon group is able to Only time will tell what the Oregon group is able to
accomplish in this ambitious endeavor.accomplish in this ambitious endeavor. W are highly optimistic and, indeed, energized by W are highly optimistic and, indeed, energized by
the idea thatthe idea that multiple institutions can come together multiple institutions can come together
To forge a strong link between the research and practice of To forge a strong link between the research and practice of teacher education, and teacher education, and
Work towards enhancing teacher preparation and, in turn, PK12 Work towards enhancing teacher preparation and, in turn, PK12 student learning. student learning.
Evolution studyEvolution study
Paper 2 – Evolution StudyPaper 2 – Evolution Study
Evolution of Policy and Implementation of Evolution of Policy and Implementation of Teacher Work Sample in OregonTeacher Work Sample in Oregon
Presenter:Presenter:Hilda Rosselli - Western Oregon UniversityHilda Rosselli - Western Oregon University
Research Team:Research Team:Dew Anna Brumley - Warner Pacific UniversityDew Anna Brumley - Warner Pacific University
Maria Cirielllo - University of PortlandMaria Cirielllo - University of PortlandHilda Rosselli - Western Oregon UniversityHilda Rosselli - Western Oregon University
Research QuestionsResearch Questions
1. What led to teacher work samples becoming mandated?
2. How have teacher work samples evolved over time within the state of Oregon?
3. To what extent do dominant patterns of implementation run counter to the major intended premises of the state’s policy?
(espoused theory and theory in use)
Research DesignResearch Design
Phase 1Phase 1 Historical--4 qualitative interviewsHistorical--4 qualitative interviews Key players involved with the early stages of Key players involved with the early stages of
teacher work samples policy developmentteacher work samples policy development Phase 2Phase 2
Survey data collected from 8 institutionsSurvey data collected from 8 institutions Early implementation, evolution of practice, challenges that have Early implementation, evolution of practice, challenges that have
evolvedevolved
Phase 3Phase 3 Document analysisDocument analysis
MethodologyMethodology
Four individuals selected for interviewing Four individuals selected for interviewing representing varying perspectivesrepresenting varying perspectives Del Schalock—Teacher researcher Dave Myton— Teacher education policymaker Kathy Wiper—Teacher on state commission Holly Zanville—Higher education official
MethodologyMethodology
Conducted 4 semi-structured interviewsConducted 4 semi-structured interviews Transcribed interview notesTranscribed interview notes Conducted membership checksConducted membership checks Coded transcripts individuallyCoded transcripts individually Compared coding schemesCompared coding schemes Reviewed and refined themesReviewed and refined themes
Theme 1: Original IntentTheme 1: Original Intent
What were teacher work samples designed What were teacher work samples designed originally to accomplish? What were they a originally to accomplish? What were they a reaction to or remedy for?reaction to or remedy for? An eAn evidence-based process designed to demonstrate
that student teachers can teach and that students learn under the guidance of a student teacher.
Legislators were looking beyond teacher mandated standardized tests for more accountability from teacher education programs about their graduates.
Theme 1: Original IntentTheme 1: Original Intent
What were teacher work samples designed What were teacher work samples designed originally to accomplish? What were TWSs a originally to accomplish? What were TWSs a reaction to or remedy for?reaction to or remedy for? Oregon Teacher Standards and Practices Commission
were looking to output standards for program review purposes
Going beyond what do teachers know and do to producing evidence that indicated student learning.
Theme 1: Original Intent Theme 1: Original Intent
“Document teacher performance beyond knowledge and connected to the standards”
Del Schalock
“…analogy of the driver’s test. Not enough to just take a written driver’s test. Need to demonstrate the skill to drive.”
Holly Zanville
Theme 1: Original Intent Theme 1: Original Intent
[It also] “ arose out of a dissatisfaction with the practice of using tests to demonstrate teacher competence”
Holly Zanville
“At Western it was clearly to go beyond the wall we had hit with competency based teacher ed where the issue is not what teachers know and can do. By 1974 we knew we had to go somewhere else because that mode didn’t help us so we realized we needed to connect teaching and learning.”
Del Schalock
Theme 2: Roots and ContextTheme 2: Roots and Context
What else was happening at the same time that influenced the development and adoption of TWSM? National, regional, state contexts? Intersection of key events
Development of competency based programs Creation of state curriculum standards in K-12 Increased accountability for teacher prep
Theme 2: Roots and ContextTheme 2: Roots and Context
Other factorsOther factors Elementary Secondary Act and engagement with Elementary Secondary Act and engagement with
schools (late 60’s and 70’s)schools (late 60’s and 70’s) Com-Field- a study funded through NWREL offered a Com-Field- a study funded through NWREL offered a
model for competency based, field-centered, and model for competency based, field-centered, and personalized teacher preparation (65 - 74)personalized teacher preparation (65 - 74)
Formation of TSPC, the nation’s first independent Formation of TSPC, the nation’s first independent commission for teacher education (1971-72)commission for teacher education (1971-72)
Commitment to context for research: Teaching Commitment to context for research: Teaching Research Institute and College of Education at WOUResearch Institute and College of Education at WOU
TSPC revision of programs to standards-based (1973)TSPC revision of programs to standards-based (1973)
Theme 2: Roots and ContextTheme 2: Roots and Context
Other factors Other factors Mid-Valley Consortium to assist districts with goal-based Mid-Valley Consortium to assist districts with goal-based
education and integration of Curriculum, Instruction, and education and integration of Curriculum, Instruction, and Assessment (1975 - 1984)Assessment (1975 - 1984)
Publication of Publication of Nation at RiskNation at Risk Report to Joint Boards referencing work samples (1986) Report to Joint Boards referencing work samples (1986) 21st Century Schools Act in Oregon (1991) 21st Century Schools Act in Oregon (1991) Analysis of this act and its implications for teachers.Analysis of this act and its implications for teachers. Recasting of teacher education standards to outcome Recasting of teacher education standards to outcome
based (1997)based (1997)
Theme 2: Roots and ContextTheme 2: Roots and Context
“ The school context in which teacher ed was being fashioned became much more outcome oriented, e.g. what we want our kids to know and be able to do and hold ourselves to responsibility for assessing that. Wasn’t that big a jump to apply this to teacher ed which led to TWSM.”
Del Schalock
Theme 2: Roots and ContextTheme 2: Roots and Context
“ComField was field centered—a graduated set of field experiences—moving from observations, to small group teaching, to full responsibility for teaching. That was a frame of reference that was firmly in the faculty minds so we married that with the goal based curriculum instruction demands for Oregon’s standards based movement.”
Del Schalock
Theme 2: Roots and ContextTheme 2: Roots and Context
“Behind closed doors [they] were able to pinpoint right to the classroom door where students were meeting the expectations….what they were getting at with the student test data that they were beginning to be able to look at effective schools.”
Dave Myton
Theme 2: Roots and ContextTheme 2: Roots and Context
Differing opinion
““I don’t think TWSMs were intended to address the credibility of the institutions but it did allow conversation with legislators in which Colleges could say we are using multiple measures to assess quality.”
Holly Zanville
Theme 3: Instrumental PlayersTheme 3: Instrumental Players
What individuals or agencies or political issues were instrumental in the start up and then evolution of TWSM? Del SchalockDel Schalock Theory and practice context at WOUTheory and practice context at WOU TSPCTSPC
Theme 3: Instrumental PlayersTheme 3: Instrumental Players
Who sat down and crafted the first draft? “I think I did.”
Del Schalock
“…75% probability that the language of TWSM came out of the research team in the mid 70s. The last half of the 70s was where it was being formulated. Tested at WOU in the late 70s or early 80s. Had sufficient testing within WOU before it went to TSPC.”
Del Schalock
Theme 3: Instrumental PlayersTheme 3: Instrumental Players
“They [TRI/WOU] built and still maintain a huge database of student test results…got the superintendents at the table looking at the data and it was all over the place. So their concern was how to ensure that kids are getting not only a good education.
Gerald????so he created a formula. It filled up the whole front of a classroom…but what it was to see if there were correlations with any of the inputs.”
Dave Myton
Theme 3: Instrumental PlayersTheme 3: Instrumental Players
“The 21st Century Schools Act didn’t mention teacher licensing but the Board of Ed and the Board of Higher Ed both indicated that they were going to do something, they were going to take Teacher Ed back, if the Commission didn’t do something pretty drastic.”
Dave Myton
Theme 3: Instrumental PlayersTheme 3: Instrumental Players
“So in the end, it kind of turned out to be the commission taking hold of it and saying we want them [student teachers] to have a work sample, you can waive anything else.”
Dave Myton
Theme 4: Evolution of IntentTheme 4: Evolution of Intent
What other purposes did teacher work What other purposes did teacher work samples attempt to serve over time?samples attempt to serve over time? Stronger clinical supervision modelStronger clinical supervision model Follow-up on teacher candidates performanceFollow-up on teacher candidates performance Quartile learning gains for groups of studentsQuartile learning gains for groups of students Shift of emphasis from learning gains to also Shift of emphasis from learning gains to also
more focus on student teacher performancemore focus on student teacher performance
Theme 4: Evolution of IntentTheme 4: Evolution of Intent
“Maybe unintended but one of the purposes was to create a closer link between student teachers and mentor teachers in the classrooms. Became a tool for tightening the expectations for student teaching and an opportunity to work together with the school personnel. Mentoring a student teacher couldn’t be a time to go sit in the lounge.”
Holly Zanville
Theme 4: Evolution of IntentTheme 4: Evolution of Intent
““The responsibility for the evaluating of work samples never became the CTs [cooperating teachers’] responsibility because of the OEA opposition.”
Kathy Wiper
Theme 4: Evolution of IntentTheme 4: Evolution of Intent
“And people were thinking, ‘now we know we have good people coming in, and we can see them in practice, but what we want to know though, is how are they when they leave? Not only the door of the university, but how are they 1, 2, or 5 years out? And frankly, we never get it, partly because the turnover rate in teaching is so high. And second because nobody had resources to go out and observe them in the field.”
Dave Myton
Theme 4: Evolution of IntentTheme 4: Evolution of Intent
““ In the early iterations of the work sample, we were talking about quartiles of students. If we were to go back and collect the old graphs of the standards…The goal of the work sample was to show that students had learned by quartiles, as opposed to each student.”
Dave Myton
Theme 4: Evolution of IntentTheme 4: Evolution of Intent
“Work sample purposes have evolved from teaching. Looking for student gains first…the meaningful gains that were focused on the CCGs (Common Curriculum Goals), then shifted to looking at the student teaching…demonstrating knowledge and skills.”
Kathy Wiper
Theme 4: Evolution of IntentTheme 4: Evolution of Intent
“The language that went into the TSPC standard was faithful to the idea of TWS but what didn’t go into the implementation were performance demands around learning gains. Became an exercise. Can you do it? TSPC never really put any quality control mechanisms in place. How those standards were interpreted and used? TSPC believes that this is partly handled through the program review.”
Del Schalock
InsightsInsights
1. Del Schlock is the “father” of work sample methodology - he had access to the research based on his position at
Teaching and research - he had funds through the Early Models program - he worked with the mid valley consortium
2. The state and national context in which TWSs first emerged was essential to understanding the full intent of the espoused theory as was the state of affairs in BOTH K-12 schooling and teacher preparation.
InsightsInsights
3. Although TWSM is required in order to complete a teacher education program and receive a license, the specific content and implementation process has been left to the discretion of a particular teacher education program. It has never been exactly prescribed by TSPC. Therefore different kinds of work samples and evolutions have flowed naturally from each institution which tailored the requirements to fit the philosophy and needs of the specific program.
Maria Ciriello
InsightsInsights
4. “In 2002-03, the detractors started coming out and telling the legislators that they were being burdened with this extensive TWS project requirement. And it’s more than any teacher should ever need. They had the opportunity to make teacher education a real profession and a self-governing profession. And by running back to the legislature saying TSPC is going to make it too hard to be a teacher, it undercut a lot of work.”
Dave Myton
InsightsInsights
5. ““I thought one of the changes that went into place that was helpful was when it was mandated that all campuses were to do TWSMs. Then a collaborative approach emerged. There was sharing and that was an important tool around which campuses, both private and public, could collaborate.”
Holly Zanville
InsightsInsights
6. “In a perfect world, we would have spent 6. “In a perfect world, we would have spent the energy required to get solid observations the energy required to get solid observations in place and quality measures of student in place and quality measures of student learning and its interpretation. But that not learning and its interpretation. But that not the case. So, in some respects, it’s all been the case. So, in some respects, it’s all been a little vacuous but faculties in Oregon have a little vacuous but faculties in Oregon have continued to play with that concept over continued to play with that concept over time.”time.”
Del Del SchalockSchalock
Experience studyExperience study
Paper 3 – Experience StudyPaper 3 – Experience StudyPerceptions of teacher educators and Perceptions of teacher educators and candidates on the use of teacher work candidates on the use of teacher work
samplessamples
Presenters:Presenters:Blaine C. Ackley – University of PortlandBlaine C. Ackley – University of Portland
Melina Dyer – Lewis & Clark CollegeMelina Dyer – Lewis & Clark College
Research Team:Research Team:Blaine C. Ackley – University of PortlandBlaine C. Ackley – University of Portland
Roy Bunch – Corban CollegeRoy Bunch – Corban CollegeEmily de la Cruz – Portland State UniversityEmily de la Cruz – Portland State University
Melina Dyer – Lewis & Clark CollegeMelina Dyer – Lewis & Clark CollegeMollie Galloway – Lewis & Clark CollegeMollie Galloway – Lewis & Clark College
Experience Team MembersExperience Team Members Roy BunchRoy Bunch, Corban College, Chair, Corban College, Chair Blaine AckleyBlaine Ackley, University of Portland, University of Portland Janine AllenJanine Allen, Northwest Christian College, Northwest Christian College Emily de la CruzEmily de la Cruz, Portland State University, Portland State University Melina DyerMelina Dyer, Lewis & Clark College, Lewis & Clark College Mollie GallowayMollie Galloway, Lewis & Clark College, Lewis & Clark College Mary JohnsonMary Johnson, George Fox University, George Fox University Ann MatschinerAnn Matschiner, Pacific University, Pacific University Tisha MorrellTisha Morrell, University of Portland, University of Portland Paula PolittePaula Politte, Pacific University, Pacific University Amanda SmithAmanda Smith, Western Oregon University, Western Oregon University
Survey InstrumentsSurvey Instruments Twelve quantitative response questions using a Likert Twelve quantitative response questions using a Likert
scale from 1 (strongly disagree to) to 5 (strongly agree)scale from 1 (strongly disagree to) to 5 (strongly agree) Three questions about the TWS methodologyThree questions about the TWS methodology Nine questions about elements of TWSNine questions about elements of TWS
Two qualitative questions:Two qualitative questions: In what ways is the work sample process and product beneficial in In what ways is the work sample process and product beneficial in
the development of an effective teacher?the development of an effective teacher?
What, if anything, would you change about the work sample What, if anything, would you change about the work sample requirements to make the process and product more beneficial in requirements to make the process and product more beneficial in the development of effective teachers?the development of effective teachers?
Survey Data ResponsesSurvey Data Responses
8 Teacher Education Institutions8 Teacher Education Institutions 33 Faculty Respondents33 Faculty Respondents 242 Candidates who had completed at least 242 Candidates who had completed at least
one Teacher Work Sample (TWS)one Teacher Work Sample (TWS)
Quantitative Survey ResultsQuantitative Survey Results No statistically significantNo statistically significant
differencesdifferences in candidates in candidates perceptions of the work perceptions of the work sample by:sample by: Institution type (public or Institution type (public or
private)private)
Graduate and Graduate and undergraduate teacher undergraduate teacher education programseducation programs
GenderGender
Number of work samples Number of work samples completed (1 vs. more completed (1 vs. more than 1)than 1)
For candidates there was a For candidates there was a statistically significant moderate statistically significant moderate correlationcorrelation between elements of between elements of TWS and the overall work sample TWS and the overall work sample (.627, p<.01)(.627, p<.01)
Faculty rated elements of the TWS Faculty rated elements of the TWS more highly than candidates:more highly than candidates: t t (316)=5.38, p<.001(316)=5.38, p<.001 Faculty M=4.20, SD=.65Faculty M=4.20, SD=.65 Candidate M=3.66, SD=.65Candidate M=3.66, SD=.65
Faculty and candidate felt the TWS Faculty and candidate felt the TWS best aided them in lesson planning best aided them in lesson planning and assessment:and assessment: M=4.07, SD=.93 for planning itemM=4.07, SD=.93 for planning item M=3.88, SD=.77 for assessment itemsM=3.88, SD=.77 for assessment items
Qualitative Analysis of TWS BenefitsQualitative Analysis of TWS Benefits
Faculty find the TWS more beneficial overall than Faculty find the TWS more beneficial overall than candidates.candidates.
Candidates focus on the specific elements of the TWS Candidates focus on the specific elements of the TWS while Faculty focus on the entire process.while Faculty focus on the entire process.
TWS Benefits
0
20
40
60
80
Planning Assessing Reflecting Other Teaching “BigPicture”
%Candidates Faculty
Respondents’ Exemplars: Respondents’ Exemplars: Benefits of TWSBenefits of TWS
TeachingTeaching CandidateCandidate: “… Writing and putting the work sample : “… Writing and putting the work sample
together was a necessary evil, but the experience together was a necessary evil, but the experience teaching is what made the work sample important to teaching is what made the work sample important to me.”me.”
FacultyFaculty: “It helps them understand the intricacies of : “It helps them understand the intricacies of teaching…teaching…
““Big Picture”Big Picture” Candidate:Candidate: “It helped me to think about the large picture “It helped me to think about the large picture
as well as the small details in my teaching.”as well as the small details in my teaching.”
Faculty:Faculty: “The entire process forces the student teacher “The entire process forces the student teacher to plan, teach, and assess a unit of instruction. “to plan, teach, and assess a unit of instruction. “
Qualitative Analysis of TWS ImprovementsQualitative Analysis of TWS Improvements
Faculty and candidates agree improvements Faculty and candidates agree improvements should be made to the logistical implementation of should be made to the logistical implementation of the TWS.the TWS.
Faculty to a greater degree than Candidates Faculty to a greater degree than Candidates indicated that TWS modifications are needed.indicated that TWS modifications are needed.
TWS Improvements
020
4060
80100
120140
Planning Assessing Reflecting Other Teaching “BigPicture”
Logistics Modify theProcess
NewMethod
%
Candidate Faculty
Respondents’ Exemplars: Improvements to TWSRespondents’ Exemplars: Improvements to TWS LogisticsLogistics
Candidate:Candidate: “The handbook that was supposed to guide us “The handbook that was supposed to guide us and provide requirements was scarce, poorly organized, and provide requirements was scarce, poorly organized, with bad examples.”with bad examples.”
Faculty:Faculty: “We seem to be forcing them to concentrate “We seem to be forcing them to concentrate more on the wording than the teaching process, and they more on the wording than the teaching process, and they put a lot of time into duplicating pages of information from put a lot of time into duplicating pages of information from the lesson plans…”the lesson plans…”
Modify the ProcessModify the Process Candidate:Candidate: “The work sample does not allow for “The work sample does not allow for
flexibility. There were some days I had to completely flexibility. There were some days I had to completely change my lesson, and this has no place in the work change my lesson, and this has no place in the work sample other than reflections sample other than reflections
Faculty:Faculty: “The work sample process often becomes one of “The work sample process often becomes one of jumping through hoops for students. Is it pretty enough? jumping through hoops for students. Is it pretty enough? Is their grammar correct? “Is their grammar correct? “
Respondents’ Exemplars: Improvements Respondents’ Exemplars: Improvements to TWS (2)to TWS (2)
New MethodNew Method Candidate:Candidate: “Work sample requirements could “Work sample requirements could
be more flexible, perhaps shorter.”be more flexible, perhaps shorter.”
FacultyFaculty: “The work sample both as a process : “The work sample both as a process and product is valuable, if and only if, the and product is valuable, if and only if, the students get a chance to discuss it with friendly students get a chance to discuss it with friendly cooperating teachers and peers. Otherwise, it cooperating teachers and peers. Otherwise, it tends to remain superficial.”tends to remain superficial.”
Next StepsNext Steps
Spring SurveySpring Survey Most candidates will have completed a second Most candidates will have completed a second
work sample.work sample. We will add a code to distinguish between We will add a code to distinguish between
elementary and secondary candidates.elementary and secondary candidates. We will administer the qualitative questions We will administer the qualitative questions
before the quantitative questions.before the quantitative questions.
Effect studyEffect study
Paper 5 – Effect StudyPaper 5 – Effect Study
Teacher work sample effects on the learning of Teacher work sample effects on the learning of K-12 studentsK-12 students
Presenters:Presenters:Tom Greene, Co-Chair, University of Portland
Mark Girod, Co-Chair, Western Oregon Jan Albrecht, Concordia UniversityDana Barbarick, Cascade College
Karen Buchanan, George Fox UniversityMickey Caskey, Portland State University
Jane Kovisto, Concordia UniversityLinda Samek, Corban College
Jacqueline Waggoner, University of Portland
The Effect GroupThe Effect Group
Consists of self-selected volunteers from participating Consists of self-selected volunteers from participating public and private institutionspublic and private institutionsCo-Chairs, one public, other privateCo-Chairs, one public, other private
Guiding PrincipleGuiding Principle:: What kinds of information do teachers need to support and What kinds of information do teachers need to support and
measure student progress toward learning in standards-measure student progress toward learning in standards-based systems of schooling most effectively?based systems of schooling most effectively?
Judging Instruction and Judging Instruction and Assessment…Assessment…
Candidate level variables - gender, type of program, Candidate level variables - gender, type of program, endorsements, authorization levelendorsements, authorization level
School and classroom level variables - SES, mobility, School and classroom level variables - SES, mobility, race/ethnicity, program eligibility….race/ethnicity, program eligibility….
Instructional methods by lessonInstructional methods by lesson Type of assessment strategies by outcomeType of assessment strategies by outcome
Judging Alignment…Judging Alignment…
Judgments about alignment Judgments about alignment betweenbetween::
State StandardsState Standards District StandardsDistrict Standards GoalsGoals ObjectivesObjectives InstructionInstruction Pre/Formative/Post Pre/Formative/Post
AssessmentsAssessments
Judging Learning Gains…Judging Learning Gains…
By individual studentBy individual student By categorization of By categorization of
student services - ELL, student services - ELL, IEP, TAG, None…IEP, TAG, None…
By Race/ethnicityBy Race/ethnicity By type of learning By type of learning
outcomeoutcome Met, Progress, Not metMet, Progress, Not met
Pilot StudiesPilot Studies
How will TWS data gathering process How will TWS data gathering process function at different institutions?function at different institutions?
Are there instrumentation and procedural Are there instrumentation and procedural issues across institutions that need to be issues across institutions that need to be modified prior to full implementation? modified prior to full implementation?
Two Pilot StudiesTwo Pilot Studies
Pilot StudyPilot Study II IIII
TimeTime Spring, 2006Spring, 2006 Fall, 2006Fall, 2006
Number of Number of TWSTWS
8888 99
Number of Number of P-12 P-12 StudentsStudents
19071907 214214
Results: Pilot StudiesResults: Pilot Studies
TWSs addressed the following subjects: math, language arts, social TWSs addressed the following subjects: math, language arts, social studies, science, world languages, health and PEstudies, science, world languages, health and PE
Teaching methods - Most frequent: Student- centered (62%); Teaching methods - Most frequent: Student- centered (62%); Teacher-centered (27%)Teacher-centered (27%)
Assessment Methods - Most Frequent: Selected Fixed Response at Assessment Methods - Most Frequent: Selected Fixed Response at the knowledge level; Student-constructed response at the the knowledge level; Student-constructed response at the knowledge levelknowledge level
Results: Pilot StudiesResults: Pilot Studies
92% of the P-12 students had learning gains92% of the P-12 students had learning gains
There were no significant difference in the learning gains or There were no significant difference in the learning gains or no learning gains based on race, TAG, SPED, ESL, or in no learning gains based on race, TAG, SPED, ESL, or in schools with poverty designationschools with poverty designation
Greater frequency of learning gains in high school…Greater frequency of learning gains in high school…Function of instrumentation?Function of instrumentation?
What is Next?What is Next?
Use revised measuresUse revised measures Gathering faculty supportGathering faculty support Consider consequences for Consider consequences for
pedagogypedagogy Local point people will clean and Local point people will clean and
organize data and submit to a organize data and submit to a central databasecentral database
Full implementation across Full implementation across seven institutionsseven institutions
Continue cross validationContinue cross validation
ChallengesChallenges
Ownership of data - Institutions, Authors, Associations? IRB process across all institutions Communication - frequent management meetings Institutional integrity - compliance with protocols Congruity of definitions - developing statewide common
terminology Support from stakeholders Time, Money, Resources Theory in action—differences in perspectives
General DiscussionGeneral Discussion