INCORPORATING A COLLABORATIVE ASSESSMENT EXCHANGE PROJECT WITH PRE-SERVICE TEACHERS AND ADMINISTRATORS: GETTING TO KNOW EDUCATOR PEDAGOGY FROM TWO PERSPECTIVES
American Educational Research Association Annual Conference Vancouver, British Columbia, Canada
April 2012
Mary E. Yakimowski and Michael AlfanoUniversity of Connecticut – Neag School of Education
Neag School of Education’s Office of Assessment
PRESENTATION DEDICATED TO MARY’S MOM – SHE HAD ALWAYS WANTED TO BE A GEOGRAPHY TEACHER; SHE ENDED UP AS A GREAT “EDUCATOR” THOUGH SHE DIDN’T HAVE A TEACHING CERTIFICATION; SHE PASSED AWAY AT THE AGE OF 93
To determine how we might use an authentic assessment project to positively influence the knowledge, skills, and/or dispositions required of pre-service teacher and administrator candidates (e.g., Wise, Ehrenberg, Leibbrand, 2008)
Based on theory of adult learning
Collaborative exchange UConn Administrator Preparation Program (UCAPP)
Teacher Certification Program for College Graduates (TCPCG)
PURPOSE
Design and implement an exchange project with between two programs across 3 departments
Assess and study the effects of this project on such things as teacher pedagogy and teacher supervision knowledge, skills and dispositions
Interpret the findings and present their implications for both programs
OBJECTIVES
Review of LiteratureMethodology
ResultsImplications of Results
Learning's / Future Avenues
REVIEW OF LITERATURE: A SAMPLING
Teacher preparation programs Kagan, 1992; Tang, 2004; Darling-Hammond, Hammerness, Grossman, Rust, & Shulman, 2005; Zeichner, 2005)
Early clinical experiences Varrati, Lavine, & Turner, 2009
Integrating administrators Blasé & Blasé, 2004; Brock & Grady, 1998; Vann, 1986; Varrati,
Lavine, & Turner, 2009; Cawelti & Protheroe, 2001; Schmoeker, 2001; Sergiovanni, 2001; Varrati, Lavine, & Turner, 2009; Wenger, McDermott & Snyder, 2002
Authentic Moore, 2003
Assessment McTighe & O’Connor, 2005; Wiggins & McTighe, 2008). Stiggins, 2009
Adult learning theory Mindful practice includes developing an awareness of one’s
mental processing, engaging in active listening, exercising flexibility, identifying bias and judgments, and acting compassionately (Epstein, 1999; McGarvey, 2010).
METHODOLOGY: SUBJECTS
TCPCG – 78 candidates UCAPP – 48 candidates
Both affiliated with the Neag School of Education at the University of Connecticut
Action research - Mixed methods
Overall research question Does this activity enhance candidates’ knowledge, skills, and/or dispositions?
Multi-angulation of data; Qual/Quant components
Including observations, document analysis, Likert-scale responses,
METHODOLOGY: RESEARCH DESIGN
Action research - Mixed methods
Overall research question Does this activity enhance candidates’ knowledge, skills, and/or dispositions?
Multi-angulation of data; Qual/Quant components
Including observations, document analysis, Likert-scale responses,
METHODOLOGY: RESEARCH DESIGN
Lesson Plan (TCPCG) Strategic School Profiles, Lesson Plan, Video of Lesson, Reflections,
Exchange Form – Part A Document analysisAssessment of Lesson Plan (UCAPP)
Exchange Form – Part B e.g., Quantitative analyses of rubricExchange Project Meeting (TCPCG/UCAPP) ObservationsImmediate Assessment of Project (TCPCG/UCAPP) Exchange Form – Parts C & D Quantitative descriptive stats; paired t-testsFollow-up Assessment of Project (TCPCG only) Semi-structured focus group
METHODOLOGY:
DATA SOURCES/ANALYSIS
RESULTS - LESSON PLANSDocument Analysis
We found that while some candidates listed content as themes or topics, such as “motion’s laws,” others offered a more thorough description, placing the lesson in the context of previous learning and providing a rationale for the timing of the lesson with associated activities.
Prompt: “The content that came earlier in the unit was:”
“At this point, students have been working on the Personal Narrative unit. We are specifically focusing on Memoirs. Students read a variety of short memoirs to understand exactly what the genre is and is not. They have engaged in the prewriting and writing processes, and they have presented in class with their rough draft. Students were asked to have the entire content of their memoirs in class this day.”
Prompt: “Set the stage” for viewing the lesson. Responses often exhibit thoughtfulness.
“Force is a difficult concept and as you can see from the many definitions, the students have an idea of what they think force is, however it has a very specific meaning in physics and the student’s initial definitions do not completely line up.”
RESULTS – ASSESSING LESSON PLANS
Unacceptable Acceptable Target Points Awarded
N (%) N (%) N (%) M SD
Assessing Appropriate & Effective Instruction
1 (2.0) 18 (36.7) 21 (42.9) 15.18 3.65
Assessing Differentiated Instruction
7 (14.3) 14 (28.6) 7 (14.3) 13.15 5.10
Assessing Classroom Management
0 (0) 16 (32.7) 24 (49.0) 16.05 3.39
Assessing Student Progress
0 (0) 23 (46.9) 17 (34.7) 14.90 3.66
TOTAL 59.28 13.33
RESULTS – THE EXCHANGE MEETING
Observation
First face-to-face meeting
Director had each UCAPP student stand to introduce the names of the one or two teachers
Parties introduced themselves to each other
Dinner line together / Informal discussions
Formal meetings convened
Culminating activity
Teachers In preparing for a classroom observation by an administrator, ensure that
the lesson is “authentic”” Share what you would like the administrator to focus on during the
classroom observation (e.g., As part of the project, differentiation and student assessment was the focus. You could state something about how you plan to incorporate these in your lesson and seek feedback.)
Be open-minded…open to suggestions and willing to accept new ideas Be self-reflective (e.g., Reanalyze the lesson yourself and share your
thoughts on the strengths and areas needing improvement.) Be willing to seek clarification of what the administrator says in the
conference Do not take “criticism” personally Have a conversation with the administrator Assume the administrator has the best interest in mind (e.g., helping you
- the teacher - and helping to facilitate learning by the students) Be respectful of the administrator (e.g., Watch tone of voice; folded arms) Try to summarize the major points shared by the administrator at the end
of the conference
Administrators Have a dialogue before the classroom observation to lay out the expectations for
the observation Have clear expectations of the focus of the lesson (e.g., As part of the project, it
was differentiation and student assessment) Be sure to watch the students, as well as the teacher, during the classroom
observation As an administrator, be confident in your observation and subsequent analyses Be respectful of the teacher (e.g., Watch tone of voice and be aware of body
language throughout the session) Do not let the ego of being an administrator creep up Understand that teachers are people who have feelings, too Be honest Keep the meeting a mutual conversation – not a monologue Do not use the “w” word (weakness vs. area for improvement) Practice “PBIS” Be specific and help to be a part of the solution (e.g., “What does that look like in
the classroom?”) Use data to back up the areas for improvement and the resulting suggestions Wrap up the session with clear communication (e.g., Let’s summarize …) End on a positive note
RESULTS – IMMEDIATE ASSESSMENT
TCPCG UCAPPPaired t
M SD M SD
Positive experience 4.47 0.82 4.61 0.61 -1.16
Enjoyable experience 4.18 1.05 4.57 0.65 -2.91**
Clear communication generally about the project
3.69 1.31 3.37 1.17 1.63
Clear expectations about the project
3.55 1.29 3.35 1.09 1.09
Focus on differentiation 3.90 1.12 3.92 0.81 -.12
Focus on pupil assessment 4.02 1.01 3.80 0.91 1.21
Clear recommendations offered 4.73 0.64 4.31 0.74 3.46**
Evidence-based feedback given 4.49 0.92 4.23 0.75 1.70
RESULTS – FOLLOW-UP ASSESSMENT
Semi-structured Focus Group
1. Tell us what you know about UCAPP and initial reaction?2. Briefly, tell me how you assessed the students in your class
during your lesson?3. How did you embed differentiation?4. How (in what ways) did the feedback provided by the
TCPCG/UCAPP Exchange project improve your teaching?5. How did the TCPCG/UCAPP Exchange project prepare you for
future interactions with school administrators?6. Please share anything else regarding with Exchange project
that we have not addressed.
RESULTS – FOLLOW-UP ASSESSMENT
How (in what ways) did the feedback provided by the TCPCG/UCAPP Exchange project improve your teaching?
Positive responses: Levels of comfort, detailed feedback, provision of insight/resources for future use, impartial reviewer, more tailored feedback
Negative responses: Issues with the timeline, lack of feedback
OVERALL – FINDINGS GIVE HOPE
Participants from both programs generally felt it was a positive, enjoyable experience with clear and evidence-based feedback/recommendations offered
The last activity of the dinner meeting was found particularly helpful to many of the participants as items to remember in the future when facing a teacher-principal evaluative interaction
PARAMETERSAll data collection needed to be done within the framework of designated courses and timeframe
Participants were from one institution
Did not include either all TCPCG students, or the other teacher education program at that institution Researchers needed to respect the regular course load of these students, particularly in TCPCG as participants are in an intensive one-year program
LEARNINGS/FUTURE AVENUES
Clearer explanations of the project More immediacy of feedback Expand to the other TCPCG cohort and
perhaps to the other teacher education component
Examine the standardized assessment results that UCAPP students take
Schedule alumni survey, embed some questions on the long-time potential benefits
CONCLUSIONS Much time for all stakeholders - Take it slowly and cautiously As higher education faculty, adult learners are mindful
practitioners Learning process did provide authentic learning
opportunities (i.e., delivering a lesson, evaluation and giving feedback to a lesson) to support students’ self-esteem, self–efficacy, self-directedness
Self-directed learning allow students to be involved in what and how they learned (e.g., reflecting on the experience and dialoguing with others promoted collegiality across participants from the two programs in a positive way)
Scholarly significance - A collaborative assessment could foster in future teachers and administrators that learning is a continuous process that will further evolve and develop
