Teacher Action ResearchWorkshop 4:
Data Analysis, Interpretation & Publication
EARCOS Teachers Conference28-31 March 2012
Donna Kalmbach Phillips, Ph.D.Pacific University, OR USA
“Action research…..requires teachers to be acutely aware of a sense of process, and to refine their perceptions to account for that
process…action research raises to a conscious level much of what is already being done by
good teachers on an intuitive level. It enables teachers to identify and come to grips with their
practice in a human way that is at once supportive and critical.”
McNiff, 2008
Action Research…•Involves a systematic or organized approach to problem-solving•Requires active engagement & interaction•Insist upon reflection, critical analysis & revolving assessment•Analyzes systems of power•Deconstructs taken-for-granted assumptions•Results in action as a practical outcome•Results in transformation•Relies upon democratic ethical principles•Focuses on a single place of inquiry
Criteria of Trustworthy Action Research
Teacher Action Research: Process Workshops
Framing the Study
Discover an Area of Focus
Develop a critical Question
Research Design
Literature Review
Trustworthy Action Research
DesignData Analysis, Interpretation
Research Design
Triangulation
Criteria for Trustworthiness
Data Analysis Fundamentals
Ongoing Analysis:Cycle & Strategies
Final Data Interpretation
Going Public
………………………………………………………Critical Questions……………………………………………………………
Researcher Dispositions
“I’ve got the data, now what?”
“Interpretation is revelation based upon information.
But they are entirely different things. However, all interpretation includes information.
The chief aim of interpretation is not instruction, but provocation.”
Tilden (1957)
Data Analysis & Interpretationrests upon
triangulated, multi-layered “thick” data
Principles of Data Analysis & Interpretation
Analysis
…..to take apart, to break down or dissect. The act of separating data for the purpose of study; a narrowing of the gaze.
Principles of Data Analysis & Interpretation
Analysis
1. What seems to be happening in this data? What is not happening in this data?
2. What is repeated in this data (words, behaviors, attitudes, occurrences)?
3. What is surprising, perplexing, disturbing in this data?
4. What information seems to be missing from the data?
Principles of Data Analysis & Interpretation
Synthesize
…..the act of putting back together again, of integrating pieces; seeing the data set as a whole; changing the lens of inquiry to wide angle.
Principles of Data Analysis & Interpretation
Synthesize1. What patterns emerge across the data? (What are
the repeating themes?)2. What are the contradictions, dilemmas in the
data? What doesn’t seem to fit?3. What are the emotional & intellectual reactions to
this data?4. How does or doesn’t this data answer the CQ?5. How are systems functioning in this data?6. What material factors are at play?
Principles of Data Analysis & InterpretationDeconstruction
…..to check assumptions, consider personal & social context; to put the lens of inquiry down, climb back up the hill to see what we’ve been missing.
Principles of Data Analysis & Interpretation: Deconstruction
1. Study categories of data: what other ways might these be configured and reconfigured?
2. What are the limitations?3. What personal assumptions, values, beliefs are
present in the synthesis?4. What would other voices say about the synthesis?5. How are systems functioning in this data?6. What material factors are at play?7. What is useful & dangerous?
Principles of Data Analysis & Interpretation: Contextualization
…..the cultural, social, physical, political context of the data
Principles of Data Analysis & Interpretation: Contextualization
1. How are larger issues, values, political views, systems & structures influencing data interpretation? (Or how should they influence interpretation?)
2. How is power at work in the study?3. Are conclusions mirroring, representative,
in conflict with mainstream discourse? What does this mean?
Criteria of Trustworthy Action Research
Categories of Data Analysis & InterpretationCategory of Analysis &
InterpretationWhat it looks like What it is
On the Ground Running: Daily
Note taking/note making Ongoing Analysis
Off Road Break:Weekly or biweekly, depending upon project
Deliberate organization of data & analysis of data
Ongoing AnalysisSome emphasis on synthesis, deconstruction, contextualization
Weekend Break: Every 2 – 4 weeks depending upon project. At least twice throughout project
Analytic Memo Ongoing Analysis Greater emphasis on synthesis, deconstruction, contextualization
Yoga Retreat:End of Project
Final Layers of Data Interpretation
Some AnalysisSynthesis DeconstructionContextualization
On the Road Running:Daily or Frequent Ongoing Data Analysis
•Note taking/Note making
Note Taking/Note MakingGrade 6 Literature Circle: Elena, Richard, Moses & Catherine
Text: Bridge to Teribithia/Date: 14 February/Time: 20 minutesNote Taking Note Making
C: I hated the way this story ended. Didn’t you? How come they always make people die? Talks with hands; leans into group. R: I didn’t like her anyways. C. That’s rude. Leslie was so cool! R: I thought she was a snob – being from the city and everything. Hey, why did they cremate her? Do you think that is weird? Talks to C. only C. My mom is gonna be crea…. E: Aren’t we suppose to start with the questions on this list? She has been looking down until now. M. sitting and listening. C: Mmm…I don’t know. Is that what Mr. Ryan said? C. directs this to R. E: We are suppose to answer these questions.
C. really gets into reading. She just finished PIG – maybe she is reading too many books about death? Not a great conversation starter but R. has certainly read & comprehended the book.How can I encourage these two to invite the others into the conversation? This has happened before – they get off into their own conversation and leave everyone else behind. R. pushes C. Here’s the rub: it is good book conversation. It is authentic. How do you keep this and get everyone involved? Or do I assume that “being involved” has to be through talk? Is this E.’s way of getting into the conversation or is she really concernedabout following directions, being the good student? How do my questions get in the way of the conversation? They are suppose to be a scaffold…are they?
Practice:Note Taking/Note Making
Video Clip: Teacher-Student Writing Conference
Directions:•Divide paper into two columns•Take notes: Focus on what the teacher says & the student’s response
Grade 5 Writing Conference
Practicing Trustworthy Action Research
Directions:•Return to note taking/note making.•Analyze talk according to Choice Words by Peter Johnston.•How does layering analysis with literature make a difference? •How is it useful and dangerous?
Strategies for Organizing Data &
Analyzing Databy
Charts
Organizing & Analyzing by Events
Researcher’s On-going Analysis Journal: Record thinking (self-reflexivity), questions, connections, perspectives
Organizing by Literature
Researcher’s On-going Analysis Journal: Record thinking (self-reflexivity), questions, connections, perspectives
Organizing by Quantitative Charts
Researcher’s On-going Analysis Journal: Record thinking (self-reflexivity), questions, connections,
perspectives
Charts + Narrative
Organizing by Data TypeStudent Post-Multi-
dimensional Fluency Scale
Survey Responses
Self-Evaluation: Fluency Rubric podcast
Post Self-Evaluation Fluency Rubric Podcast
Post Survey Podcast
Antidotal Notes
A Not administered yet
Says she is a reader some of the time but only when reading by herself. Recognizes different kinds of readers in the class. Doesn’t like to read aloud.
G,G,O,O, G,G
O,D,O NW, NW, O
Not completed yet
Chose a rhyming book for the podcast because, “This will sound good.”
Researcher’s On-going Analysis Journal: Record thinking (self-reflexivity), questions, connections, perspectives
Practice Data Set:Grade 5
Building Community
The Researcher’s JournalThe discussions that followed the classroom meeting were probably the worst overall. Three of the five groups bickered throughout the discussion time. As a result of this, two groups (Mouse “A” and Mouse “B”) had minimal discussion. The Applewhites had no discussion because they could not resolve their intrapersonal conflicts. The Hatchet group was unable to have a discussion because there was only one member present. Dark Hills Divide was the only group having a productive discussion. I asked the counselor to observe The Applewhites group and noted that the environment was so tense that it was impossible to hold a discussion. One member was so bossy that she alienated the other members. Another member was unprepared and distracted the group. The other two members stirred the pot and argued with the other two. The counselor questioned whether the literature circle jobs are too complex and whether the groups are too large. He suggested pairing students up with one task to do – a task that they can be successful at. I am having the same thoughts after witnessing the extreme interpersonal conflicts and off task behaviors. Many of these students cannot get past their bickering to hold a discussion. I was discouraged to find that despite teaching behavior skills, these discussions are usually not a productive use of time for four of the five groups. Once again, I find myself asking, is it the task itself or the lack of behavior skills or something I am not even accounting for that causes the problems? Are the books not engaging? Should I have allowed choice? How do I account for context of these kids’ lives?
Weekend Break (Reflective Pause):The Analytic Memo
•Ongoing Analysis with greater emphasis on synthesis, deconstruction, contextualization•Formal Data Analysis•Completed 2 – 4 times throughout project
Why an Analytic Memo?
•Modify the AR design•Recheck & possibly modify CQ•To change course if needed•To practice trustworthy action research through self-reflexivity, multiple perspectives, and making strong connections
The analytic memo is a space to…
Process for the Analytic Memo
1. Read through all organized data, including note taking/note making & research journals
2. Read with literature at your side3. Read for repeating patterns, ideas,
themes4. Cluster, mind map accordingly 5. Develop synthesis statements or
synthesis questions
Teacher Talk does not appear to exist in a vacuum: the environment, occasion,
assignment, my mood, and the student all can influence my talk (and a student’s
response).
Synthesis Statement orQuestion
Relevant Supporting Data
Influencing Literature
Further Questions
Plan of Actions(Interventions)
Teacher Talk does not appear to exist in a vacuum: the environment, occasion, assignment, my mood, and the student all can influence my talk (and a student’s response).
Note data for Jenna, Jorge, Mike & Lelah:
Note taking/Note making:Jan 10, 12, 25, 30
See conference notes for the same days
Johnston:Choice WordsDynamic Classroom
Lenz Taguchi,Going Beyond Theory/Practice Divide
It seems as if teacher talk is just one piece of this energized moment (or flat moment): how can I begin to track these other factors that complicate the dynamics of talk?
Develop a chart with these categories:Physical InterruptionsMy moodStudent’s moodAssignmentUse of Talk.Have rating scales.Quick way to track & analyze data next period
Next Step:Critical Colleague Response & Dialogue.
Document.
Critical to the Analytic Memo
Practice Trustworthiness:•Use raw data•Employ the expertise of the literature•Seek multiple perspectives (critical colleague, students, parents, others)•Practice self-reflexivity: pursue significant questions, stay open, resist conclusions•Make strong connections •Return to Critical Question
The Analytic Memo:An Excerpt
Directions:•Read through the memo.•Be this teacher-researcher’s critical colleague.•What questions do you have for her? What do you want to know more about?•Discuss the memo: How does (or doesn’t) it represent trustworthy action research?
Cycles of Ongoing Data Analysis
Data Collection/AR Memo 1
•Data may appear disconnected; unable to write synthesis questions or statements•Need to refine CQ•Discover gaps in data•Need to refine data collection strategies•Requires adapting teaching/interventions
Data Collection/AR Memo 2•Data is likely more focused•Synthesis statements are likely questions•May still refine CQ•Connects data more closely to literature•Further refines data collection strategies focusing on emerging themes, patterns•Actively seeking multiple perspectives•Further refines teaching/interventions
Data Collection/AR Memo 3•Data richer, multi-layered leads to more connections across data sets & to literature•Enriched & expanded themes & questions•Refined data collection to target specific areas•Continue to refine teaching/interventions•Actively practicing trustworthiness
Yoga Retreat:Final Data Interpretation
Actively practicing:Synthesis
DeconstructionContextualization
Revisit, review & re-read ongoing analysis
& memosCreate mind maps,
charts, timelines and/or generate
categories
ExpandInterpretations
Apply layers of interpretations
Return to Critical Questions
Draft Synthesis Statements
Revisit, review & re-read ongoing analysis
& memos Apply layers of interpretations
Practice active deconstruction
Practice activecontextualization
What is ignored, lost or disregarded?
Create mind maps, charts, timelines and/or generate
categories
ExpandInterpretations
Apply layers of interpretations
How do I know this ?Add Data
How do I know this?Delete categories
Deconstruct & Contextualize Data
Apply layers of interpretations
Draft Synthesis Statements
Different perspectives
Participant Voices
Expert Consultation
Contextualization
Return to CriticalQuestion
Apply layers of interpretations
What do I still wonder?
How are questions
limitations?
How are questions areas
to explore?
Draft Synthesis Statements
Final Check:Critical
Colleagues
EvaluateInterpretations
forTrustworthiness
Data Analysis & Interpretation:Summary of Process
•Collect data•Organize data•Analyze, synthesize, deconstruct, contextualize data•Pause…reflect during formal spaces of ongoing analysis: Analytic Memo•Cycle through above process 2-4 times•Final Data Interpretation
Going PublicKey points I want to
makeConnecting Points
from the dataConnecting points from the literature
Questions I still have from this study
Writing Your Story: Traditional
•Abstract•Introduction•Literature Review•Clarifying AR Project (Who, what, how, where, when & limitations)•Road Map: Research design/Methodology•The Story of the AR Project•Further Reflection & Continuing Questions
Sharing Your Story: Alternatives
•Digital Witness•Narrative, poetry•Drama•Illustrated Timelines•Artwork, graphica•Poster Presentations•Snapshots
Venues for Sharing
SchoolEARCOS
Open Source JournalsPractitioner’s Journal
ConferencesWeb Site
Online PublicationsBlogs
Newsletters
AR Data Interpretation
Revisit, review: reread ongoing data analysis & memos
Create mind maps,charts, and/or timelines;
generate categories
Expand your interpretation
Apply layers ofinterpretation
Return to the questions
Draft synthesisStatements
Interpretation Scaffold
GoingPublic
Prewriting AudienceKey Points“Thick Description”“Results”LingeringQuestions
Going PublicWeb pageBlogConferencesPractitioners’JournalsNewspaper &Letters
Critical Questions
Critical Questions
Discover an area of focus
Literature review
AR Design
What I willdo in my AR
Project?
What data will I collect?
How will Icollect the data?
How will I organize
the data?
AR Plan &Timeline
Becoming a Teacher Action
Researcher
Clarify Critical
Questions
Collect Data Set
ObservationsArtifacts
Interviews
Researcher’s Journal
On-Going AnalysisOrganize and readthe dataThink about the dataChart, free write,cluster process the data in theresearcher’s journalUse AR question as guide
Write AR memoDraft synthesis statements
Questions
AR Data Analysis
Action ResearchAsPractice
#4 Knowing Theory & Research Gives Teachers an Authoritative Voice• Speaking from your own experience is
• Critical.• Speaking from your own experience with
supportive data is Persuasive
. • Speaking from your own experience with
supportive data framed by theory & research is
Powerful
Why do Action Research?
Doing Action Research keeps YOU In the Conversation
• Don’t know theory/research perspectives? • Don’t engage in research of your own?
Someone is positioned to TELL YOU. (You listen.)
• Know the theory/research perspective?• Have research & data of your own?
You ENGAGE, DEBATE(You Decide.)