SESSION 9SESSION 9

Analyzing DataAnalyzing Data• Exercise twenty Questions

– Develop a list of five questions that you would like the data from your research to answer

– Review these with your number group, allow them to refine or add to them

– The next slide has some sample questions• Some of you are at different stages on your project, If

you do not have findings, try and antic-pate as if you do

• Exercise twenty Questions– Develop a list of five questions that you

would like the data from your research to answer

– Review these with your number group, allow them to refine or add to them

– The next slide has some sample questions• Some of you are at different stages on your project, If

you do not have findings, try and antic-pate as if you do

Sample QuestionsSample Questions

• What sorts of findings do you expect to get?

• How can you best organize these expected findings?

• What sorts of findings did you get that you did not expect?

• How can you best organize the unexpected findings?

• Which of these findings do you feel are the most important?

• What sorts of findings do you expect to get?

• How can you best organize these expected findings?

• What sorts of findings did you get that you did not expect?

• How can you best organize the unexpected findings?

• Which of these findings do you feel are the most important?

Common Phases of Qualitative AnalysisCommon Phases of Qualitative Analysis• Define the type of analysis• Classify the data• Make connections among classes

of data• Present the results of analysis

• Define the type of analysis• Classify the data• Make connections among classes

of data• Present the results of analysis

Type of AnalysisType of Analysis

• What approach– Qualitative Science– Qualitative Inquiry

• What approach– Qualitative Science– Qualitative Inquiry

Classify the DataClassify the Data

• Must decide what information to include• What data to ignore• How will you group the data that

remains?• How will you classify it?• How many categories will I have, Will it

be too much and delude the information?

• Must decide what information to include• What data to ignore• How will you group the data that

remains?• How will you classify it?• How many categories will I have, Will it

be too much and delude the information?

Make ConnectionsAmong DataMake ConnectionsAmong Data• See what the data are telling us?• Does the data indicate a pattern

that leads us to a theory or a larger theme?

• See what the data are telling us?• Does the data indicate a pattern

that leads us to a theory or a larger theme?

Coding and AnalysisCoding and Analysis

• Coding is the process where you select data and determine if it is important in some way.

• In qualitative analysis coding categories are built as the research develops- in quantitative they are established at the start

• Another name for coding and analysis is (TA) Thematic Analysis

• Coding is the process where you select data and determine if it is important in some way.

• In qualitative analysis coding categories are built as the research develops- in quantitative they are established at the start

• Another name for coding and analysis is (TA) Thematic Analysis

TATA

• Thematic Analysis is about Searching for patterns in data

• When a pattern is identified, it is assumed that something systematic is creating the pattern

• As a researcher becomes familiar and aware of the data, they begin to see patterns

• Thematic Analysis is about Searching for patterns in data

• When a pattern is identified, it is assumed that something systematic is creating the pattern

• As a researcher becomes familiar and aware of the data, they begin to see patterns

TATA• Inductive

approach-• logic the

inference of a general law from particular instances.

• Inductive logic in research is best compared to the concept of the null hypothesis

• Inductive approach-

• logic the inference of a general law from particular instances.

• Inductive logic in research is best compared to the concept of the null hypothesis

The Null HypothesisThe Null Hypothesis

• The idea of the null hypothesis is that no matter how much data we review, there will be no relation to the two variables

• Another way to remain objective

• The idea of the null hypothesis is that no matter how much data we review, there will be no relation to the two variables

• Another way to remain objective

Null HypothesisNull Hypothesis

• Is a statistical strategy to determine if there was a change related to a treatment

• It is easier to determine if there is a change if you start out looking for no change

• The null hypothesis states there is no difference between populations that are being compared.

• If you can reject the hypothesis then you theory is true

• Is a statistical strategy to determine if there was a change related to a treatment

• It is easier to determine if there is a change if you start out looking for no change

• The null hypothesis states there is no difference between populations that are being compared.

• If you can reject the hypothesis then you theory is true

Null HypothesisNull Hypothesis

• Think of a bell Curve• If the null hypothesis

is true, all data lies within the middle of the curve

• If data falls on either end ( on two way measures) the it proves a difference exist

• Think of a bell Curve• If the null hypothesis

is true, all data lies within the middle of the curve

• If data falls on either end ( on two way measures) the it proves a difference exist

TA, Null Hypothesis, and InductionTA, Null Hypothesis, and Induction• In qualitative, induction more often

refers to moving from the specific to the general

• You gather specific data and try and develop a general pattern

• In qualitative, induction more often refers to moving from the specific to the general

• You gather specific data and try and develop a general pattern

InductionInduction

• Some qualitative researchers do not like the term induction, they would prefer the term “Abductive.”

• Some qualitative researchers do not like the term induction, they would prefer the term “Abductive.”

Feedback and ComparisonFeedback and Comparison

• One way to evaluate themes is through a Feedback approach

• This allows for the refinement of themes

• Two ways to do this– Incident comparison – Theoretical comparison

• One way to evaluate themes is through a Feedback approach

• This allows for the refinement of themes

• Two ways to do this– Incident comparison – Theoretical comparison

Incident ComparisonIncident Comparison

• When you compare incidents that you have developed into a theme to newly occurring incidents

• During your observation at a hotel pool, you noticed that men getting out of the pool chose a white towel to dry themselves while women looked for the softest towels

• When you compare incidents that you have developed into a theme to newly occurring incidents

• During your observation at a hotel pool, you noticed that men getting out of the pool chose a white towel to dry themselves while women looked for the softest towels

Theoretical ComparisonTheoretical Comparison

• Here the researchers in more interested in categories than what the incidents might belong to

• If you observed women buying shoes and they selected shoes not based on fit or comfort, but on the basis of the status of the brand

• Here the researchers in more interested in categories than what the incidents might belong to

• If you observed women buying shoes and they selected shoes not based on fit or comfort, but on the basis of the status of the brand

Feedback and comparisonFeedback and comparison

• Incident comparisons deal with concrete issues like fit and body type

• Theoretical-deal with more abstract issues

• Incident comparisons deal with concrete issues like fit and body type

• Theoretical-deal with more abstract issues

SaturationSaturation

• When you can develop a theory and test its validity– Develop a theory– Use the theory to predict or control

other phenomena, when this can be accomplished, this proves validity

• When you can develop a theory and test its validity– Develop a theory– Use the theory to predict or control

other phenomena, when this can be accomplished, this proves validity

SaturationSaturation

• As you pile up your data, see what you can live without, take out any irregularities

• As you pile up your data, see what you can live without, take out any irregularities

Thematic CodingThematic Coding

• Grounded theory does not suggest that you look at prior research, the want to build theory from ground up-

• However, other than grounded theory, most other qualitative procedures make extensive use of thematic coding with the help prior research and data.

• Grounded theory does not suggest that you look at prior research, the want to build theory from ground up-

• However, other than grounded theory, most other qualitative procedures make extensive use of thematic coding with the help prior research and data.

Thematic CodingThematic Coding

• Suggestions for building a good code– Give it a label– A definition of the theme– A description of how you would

recognize it– A description of any qualifications or

exceptions– Positive and negative examples

• Suggestions for building a good code– Give it a label– A definition of the theme– A description of how you would

recognize it– A description of any qualifications or

exceptions– Positive and negative examples

Thematic CodingThematic Coding

• Basic themes are found by looking at actual examples of data

• Then try and take the basic themes and integrate them into clusters that contain similar points and issues

• Then look if there were global themes

• Basic themes are found by looking at actual examples of data

• Then try and take the basic themes and integrate them into clusters that contain similar points and issues

• Then look if there were global themes

Confirming DataConfirming Data

• Now that you have the data you need to confirm it.

• Before it is confirmed, you need to bring meaning to this

• Now that you have the data you need to confirm it.

• Before it is confirmed, you need to bring meaning to this

Developing MeaningDeveloping Meaning

• Bringing meaning to data and themes can be very simple or very complex

• On one end, is just counting an verifying how often the instances occurred.

• On the other end is developing psychological theories or metaphoric comparisons

• Bringing meaning to data and themes can be very simple or very complex

• On one end, is just counting an verifying how often the instances occurred.

• On the other end is developing psychological theories or metaphoric comparisons

ActivityActivity

• Look at the data, try and develop • A basic theme ( based on

numbers)• A psychological theme• A metaphoric theme

• Look at the data, try and develop • A basic theme ( based on

numbers)• A psychological theme• A metaphoric theme

Accounting for VariablesAccounting for Variables

• As you examined the data, you probably asked many questions, These are variables that were not identified

• Try and identify all variables

• As you examined the data, you probably asked many questions, These are variables that were not identified

• Try and identify all variables

Verifying MeaningVerifying Meaning

• The greatest threat to research is Researcher’s bias– Have you become too involved – Are your personal interest making

sure you are:• Are you overlooking data• Are you seeing all of the data

• The greatest threat to research is Researcher’s bias– Have you become too involved – Are your personal interest making

sure you are:• Are you overlooking data• Are you seeing all of the data

Verifying MeaningVerifying Meaning

• Sometimes, you want to generalize your data, if so, you need to make sure you have– Have sampled a broad enough

sample of informants– Have you gathered enough data– Occasionally researchers try to

replicate the experiment, not done often with qualitative

• Sometimes, you want to generalize your data, if so, you need to make sure you have– Have sampled a broad enough

sample of informants– Have you gathered enough data– Occasionally researchers try to

replicate the experiment, not done often with qualitative

ValidityValidity

• Weigh your data properly– First hand information more valuable than

second hand information– Information gathered in in formal situations

more valuable than formal settings– Data from trusted informants more valuable

than those of dubious informants– Try and if then test and see if it works– If women chose soft towels are concerned

about their skin, then those that do not are not concerned about their skin?

• Weigh your data properly– First hand information more valuable than

second hand information– Information gathered in in formal situations

more valuable than formal settings– Data from trusted informants more valuable

than those of dubious informants– Try and if then test and see if it works– If women chose soft towels are concerned

about their skin, then those that do not are not concerned about their skin?

Meaning Verification and Research DisplaysMeaning Verification and Research Displays• Data displays go beyond describing

data to explaining patterns and variables within the data.

• Two types of displays– Within-case Displays– Cross-Case Displays

First you must explore and describe, then you explain, order and predict

• Data displays go beyond describing data to explaining patterns and variables within the data.

• Two types of displays– Within-case Displays– Cross-Case Displays

First you must explore and describe, then you explain, order and predict

Within-Case DisplaysWithin-Case Displays

• Helps sort out the order of a given case

• Seek out the variables that are responsible for order

• The simplest is a Context Chart• A context Chart is a list of key

behaviors observed within the case

• Helps sort out the order of a given case

• Seek out the variables that are responsible for order

• The simplest is a Context Chart• A context Chart is a list of key

behaviors observed within the case

Within-Case DisplaysWithin-Case Displays

• Checklist Matrix• Break down the variables before

hand and make a list( this is how it is different than Context Chart)

• Do not rank or order variables

• Checklist Matrix• Break down the variables before

hand and make a list( this is how it is different than Context Chart)

• Do not rank or order variables

ActivityActivity

• Think of principals you have had• Develop a list of their qualities• Good and bad, Include at least ten

items• Then share them with your group.• Discuss what would your observation

reveal if you used this checklist with a principal, what information would you gain?

• Think of principals you have had• Develop a list of their qualities• Good and bad, Include at least ten

items• Then share them with your group.• Discuss what would your observation

reveal if you used this checklist with a principal, what information would you gain?

Cross-Case DisplaysCross-Case Displays

• You try and build a larger picture but also keep focus on the individual

• Can use historical information or information from other cases to help focus order

• You try and build a larger picture but also keep focus on the individual

• Can use historical information or information from other cases to help focus order

Cross-Case DisplaysCross-Case Displays

• Time-order and Conceptually Ordered are two examples of cross-case displays

• When Several cases are examined, you can use time as a marker or a similar concept to organize and compare

• Time-order and Conceptually Ordered are two examples of cross-case displays

• When Several cases are examined, you can use time as a marker or a similar concept to organize and compare

Cross-Case DisplaysCross-Case Displays

• Case ordered effects matrix• You assume that the cases act as

causes and explain effects• Take the matrix you just

developed, Now use it to compare several principals

• Case ordered effects matrix• You assume that the cases act as

causes and explain effects• Take the matrix you just

developed, Now use it to compare several principals

Activity Activity

• Develop a cross case display using a Time order approach with one of the following cases– An astronaut– A teacher– A doctor– An FBI agent– Someone in the Military, not an officer

• Develop a cross case display using a Time order approach with one of the following cases– An astronaut– A teacher– A doctor– An FBI agent– Someone in the Military, not an officer

Synthesis and IlluminationSynthesis and Illumination

• Facets– A facet is similar to a theme, but it

takes a broader more holistic view of the research- compared to a diamond, when you view a facet of a diamond you still see the whole diamond.

– Your goal as a researcher is to present the facets of a case

• Facets– A facet is similar to a theme, but it

takes a broader more holistic view of the research- compared to a diamond, when you view a facet of a diamond you still see the whole diamond.

– Your goal as a researcher is to present the facets of a case

Synthesis and IlluminationSynthesis and Illumination

• Perspective• Example look at a house as a

burglar and a perspective new owner? Very different views

• Unlike Quantitative, personal perspective is allowed and even encouraged ( different than Bias)

• Perspective• Example look at a house as a

burglar and a perspective new owner? Very different views

• Unlike Quantitative, personal perspective is allowed and even encouraged ( different than Bias)

Synthesis and IlluminationSynthesis and Illumination

• Identification- The researcher has a need to identify unique and new aspects of the research

• Identification- The researcher has a need to identify unique and new aspects of the research

ArtifactsArtifacts

• An Artifact is something a person has made to transform nature, the transformation can be material, intellectual, emotional, spiritual, ideological or any combination thereof.

• Two main types Non- written and Written

• An Artifact is something a person has made to transform nature, the transformation can be material, intellectual, emotional, spiritual, ideological or any combination thereof.

• Two main types Non- written and Written

ArtifactsArtifacts

• Non-written– Tools, buildings, toys

• Written– Formal documents, records

• Non-written– Tools, buildings, toys

• Written– Formal documents, records

ArtifactsArtifacts

• Examples– Public records, birth records, deaths,

land records, arrest, convictions– Actuarial records are the most

accessed public records in a culture

• Examples– Public records, birth records, deaths,

land records, arrest, convictions– Actuarial records are the most

accessed public records in a culture

ArtifactsArtifacts

• Not all records are public• The others are called Episodic or

personal• These are business records, union

records school records• Personal include diaries, letters

• Not all records are public• The others are called Episodic or

personal• These are business records, union

records school records• Personal include diaries, letters

InternetInternet

• The internet has become its own sub culture– New users, dedicated lifetime users– A lot of fertile area for studying

• The internet has become its own sub culture– New users, dedicated lifetime users– A lot of fertile area for studying

Activity, in your number groupActivity, in your number group• Make a list of unique personalities

or sub cultures that you find that use computers develop at least five

• Make a list of unique personalities or sub cultures that you find that use computers develop at least five

Qualitative Evaluation ModelsQualitative Evaluation Models• Evaluation is used for

accountability reasons • Many programs that are initiated

need to be evaluated.• Funding sources, the government

want to know if they should keep investing money into new programs

• Evaluation is used for accountability reasons

• Many programs that are initiated need to be evaluated.

• Funding sources, the government want to know if they should keep investing money into new programs

Qualitative Evaluation ModelsQualitative Evaluation Models• Originally these programs were

evaluated using quantitative measures

• But these measures fall short, Now qualitative measures are used either solely or in conjunction with quantitative measures

• Originally these programs were evaluated using quantitative measures

• But these measures fall short, Now qualitative measures are used either solely or in conjunction with quantitative measures

Qualitative Evaluation ModelsQualitative Evaluation Models• Can you think of any quantitative

tools that the government uses to evaluate programs that you think qualitative measures would be superior?

• Can you think of any quantitative tools that the government uses to evaluate programs that you think qualitative measures would be superior?