Data Analysis
Types of Validity
Internal validitydegree to which results are true for the participants
External validitydegree to which results can be generalized beyond the participants in the study
Test validityDegree to which a test or assessment measures what it is supposed to measure
Validity ConsiderationsType of Validity Questions
Truth-value In what ways can I ensure my results are accurate and truthful?
Applicability/transferability How might my results be useful beyond my particular classroom and with other students?
Consistency/dependability Would I get the same results with other students similar to mine?
Neutrality/confirmability How do I show results are indicative of what actually occurred during the study and do not reflect my personal desires or biases?
Democratic validity Which stakeholders should collaborate or have a voice in the study? In what ways should I involve them?
Outcome validity How will I use results for continued planning, ongoing reflection, and deepening my personal understanding?
Process validity What do I need to do to ensure I have looked deeply at the problem so I understand the ways context and processes have impacted results?
Catalytic validity In what ways will processes and outcomes change my practice?
Dialogic validity Do I need a critical friend to discuss my study and results with so I can determine whether my planning and interpretations are on track?
Ways to Increase ValidityPeer debriefing
Discuss study with a colleague not invested in study to gain their insights
Persistent and prolonged observationsHelps to determine whether the intervention is effective after the newness wears off providing credibility
Record data accuratelyBe sure to include detailed notes to help answer questions
Member checksDiscuss your interpretations of data to determine whether your findings accurately represent the participants
Triangulation of dataThe use of multiple sources to corroborate findings
Researcher biasDefine any preconceived ideas you have and note them
Engage in ongoing reflective planningAllows a researcher to change data collection strategies based on experiences during the data collection phase.
Obtaining Permission
Will protect your participants, school and university
Parents will want to know if child is involved in action research
Always extend courtesy of obtaining permission from the participants in the study regardless of whether they are children or adults
Example of Consent Form
Dear Parents,
I will be conducting a study in our classroom to determine [include purpose here]. I am writing to ask permission to use the data I collect from your child during this process. Participation in this study involves only regular classroom activities. You may contact me at any time regarding your child’s participation. The principal of the school has approved this study.
The purpose of the study is to [briefly describe purpose here]. The study will take place at [name location] and will lastfor [give time frame]. During the study, I will collect various forms of data to determine whether [name of intervention] was successful. Possible types of data I will collect include [list data collection strategies such as samples of student work, survey/questionnaires, interviews, observations, test scores, etc.].
Benefits of participating in this study include [describe the benefits]. Only [name(s) of researcher and collaborators] will have access to the data collected in this study. Your child’s participation in this project is strictly confidential. Only [and names of collaborators and/or supervising professor if study is part of a university assignment] will have access to your child’s identity and to information that can be associated to your child’s identify. [If applicable, state when data or documentation will be destroyed].
Use of data from your child is voluntary. You may contact me at any time if you do not wish to have your child’s data included in this study. Please check the appropriate choice below, sign the form and return the form with your child.____ I give permission for my child’s data to be used in this study. I understand that I will receive a signed copy of this consent form. I have read this form and understand it.____ I do not give permission for my child’s data to be included in this project.
______________________ _____________________________ ________________ Student’s Name Signature of parent/guardian Date
Data Collection
Any form of information, observations, or facts that are collected or recorded
Should be collected systematically
Plan should be in place prior to collection
Describes what data will be collected
How, when and how often it will be collected
May want to use a calendar or checklist to record when and how data is collected
Types of Data Collection
Log or research journalNotebook used to record thoughts and observations related to all parts of research
Field notesWritten observations of what is taken place in the classroom
ChecklistsA list that specifies certain attributes such as behaviors, traits, assignments, or skillsChecked off when attribute is seen
InterviewsStudents respond to a specific set of questions that are prepared in advancedQuestions are asked in same manner and order each time to maintain consistency
Data retrieval chartsVisual organizer used to collect and organize data
Continued…
Student products or performancesA representative sample of each student’s work at different time periods to provide feel of student work
SurveysAllow you to get a variety of information quickly
Scores and other Quantifiable DataScores on tests, homework, grades
Attitude and rating scalesProvide respondents with a question or statement for which they are asked to select one of several answers that determines the strength of responseFive point rating scale
Example: 5= strongly agree to 1= strongly disagree
Archival dataPast scores or grades
Strategies for Data Analysis
Interim analysisOngoing analysis of the data throughout the study
Looking at and thinking about data as it is collected and then making changes or additions to strategies if necessary
Allows the researcher to make changes to data collection strategies during the study based on problems or questions that arise when analyzing the data
Analysis of Quantitative Data
Quantitative dataAny data that can be counted or averaged the number of responses for each item
Example: number of readers who indicated that they are strong, average, or weak readers
– Strong = 3, average=2, weak=1
– Provide each student with a score and then find the percentage of class that felt they were a 3,2, or 1
– Could average scores to find the mean score of the class to look at the average score of how students’ perceived their reading ability
Steps of Analysis for Quantitative Data
ReportingDisplaying the data and describing what it shows
ComparingLooking at various attributes and comparing to see if any differences appear
Example: Does gender make a difference in the scores? What does the data show?Example: Sample one as compared to sample two Is there a difference between the two samples? For example, did scores increase based upon what was implemented? (pre and post data)
Displaying dataUse charts and graphs to visual display data collectionMicrosoft – analyzing data using excel 2007 tutorial: www.microsoft.com/office/newday/default.mspx?nav+v1xcsubmenuxi6
Steps for Organizing and Analyzing Data
Gather together data sources and record informationRecord data in a grade book, journal or computer spreadsheet
Create graphical displays of dataDisplay data so that it is easy to understandCharts, tables, graphs
Examine displayed data to determine the best ways to explain results
Be sure to accurately describe data and data comparisonsShare your analyses and graphical displays with collaborators or peers
Colleagues can help you determine whether your analyses, charts, figures, graphs make sense
Make notes in your journal about how your analyses can be used to answer research questions
Refer often to your research questionMake sure to analyze data in ways to help answer the research question
Qualitative Data Analysis
The process of making meaning from data sources that can be interpreted in various ways
Requires a deeper analysis of data
Must first be converted to textLook for patterns or themes within the data
Steps for Analyzing Qualitative Data
Data must be written as textTextual data are studied for patterns to determine ways to code the dataTextual data and codes are analyzed so that patterns and themes can be developedAnswers to research questions are offered based on themes found in the data
Coding Examples (Bogdan and Bilken 2003)
Setting/contextProvides descriptive information on aspects of the research setting
Participants’ perspectivesDescribes what participants think about certain issues
Definition of situationIllustrates participants’ understanding of setting and context
Ways of thinkingDescribes participants’ understanding of self and/or others
ProcessesExplains patterns of behavior over time
ActivitiesIllustrates recurring, typical behaviors
EventsDescribes specific or particular nonrecurring, meaningful events
StrategiesDescribes methods used by participants to accomplish certain tasks
RelationshipsDefines social roles and typical behavioral patterns among or between people
Triangulating Data Sources and Drawing Conclusions from Data
TriangulationDetermining whether data sources corroborate with one another
Draw reasonable conclusions based on the interpretation of the data
Include the answers to the such questions as– Was the intervention effective? Why or why not?
– What would you do differently?
– You need to include future plans regarding how you intend to use the results and conclusions of the study to inform your practice as an educator.