Quantitative data analysis part 2

Why is it so important to know the level of measurement.

!!!!!!!!!!!!• IN ACTUALITY, MOST OUTCOME MEASURES in

outcome evaluations ARE DONE WITH INTERVAL LEVEL DATA OR ORDINAL LEVEL DATA THAT IS “TREATED AS INTERVAL LEVEL DATA.

• INTERVAL LEVEL OF DATA IS SOMETIMES REFERRED TO AS ‘CONTINUOUS MEASURMENT’.

• THIS MEANS IT IS MEASURED WITH NUMBERS 1,2,3, 4 5 ETC.

• NUMBER OF ABSENCES ARE CONTIUOUS, I.Q. SCORES ARE CONTINUOPUS, DAYS IN HOSPITAL ARE CONTINUOUS, SCORES ON STANDARDIZED SCALES ARE CONTINUOUS

!!!!!!!!!!!!

• IN REALITY ANYTHING YOU CAN COUNT USING REGULAR NUMBERS IS TREATED AS CONTINUOUS DATA AND

• IS ANALYZED AS INTERVAL LEVEL DATA

WHY IS THIS IMPORTANT AND WHY IS IT IMPORTANT TO KNOW

WHETHER WE ARE LOOKING FOR DIFFERENCES?

• WHEN WE CAN ANSWER THE FIRST TWO QUESTIONS ABOUT LEVEL OF MEASUREMENT AND DIFFERENCES. WE CAN USE A FLOWCHART TO PICK THE TEST.

• LOOK AT THE QUESTION ON THE NEXT SLIDE.

Data analysis quantitative How to know which statistical test to use.

four questions, one can pick the correct statistical test (and also know whether the researcher picked it as well. For now. Lets look at these two

1. What level of measurement is(are) the outcome measures? Most program evaluation outcomes are continuous (interval or ordinal0 # absences, scores on scales #days in hospital. A few are nominal (categories) i.e. passed failed, graduated did not graduate etc,

2. Is it a hypothesis of difference or a hypothesis of association. All comparisons are tests of difference! (before/after group 1/group 2)

3. If comparison, Are the sample groups independent (for example comparing two groups) or are they correlated (comparing a group with itself, before and after).

all single group comparisons - pre and post or pre-during and post are CORRELATED!!!

4. How many groups or measures are there that I am comparing? Or how many variable are there that I am associating/

CorrelatedMore than two

FriedmansANOVABy ranks

ANSWERS QUESTION #1OUTCOME MEASURE

ANSWERS QUESTION#2

ONCE WE HAVE DETERMINED THAT THE LEVEL OF OUTCOME MEASUREMENT IS NOMINAL AND WE ARE LOOKING FOR DIFFERENCESWE HAVE NARROWED OUR CHOICE OF TESTS DOWN TO THREE

variables variables

OUTCOME MEASURES ANSWERS Q. #1

ANSWERS Q #2

LETS ASSUME THAT ALL OF YOUR OUTCOMES MEASURESARE CONTINUOUS. SO YOU WOULD TREAT THEM AS INTERVAL LEVEL DATA. THEN ASSUME THAT YOU ARE LOOKING FOR DIFFERENCES. NOW YOU ARE DOWN TO 4 POSSIBLE TESTS

1. What level of measurement is(are) the dependent variables (outcome measures)? Nominal, ordinal or interval? Depending on the answer to this, go to one of the three charts below.

2. Is it a hypothesis of difference or a hypothesis of association. Thus is it a test of difference or a test of association. All comparisons are looking for differences.

3. Are the sample groups independent (for example comparing two separate groups) or are they correlated (comparing a group with itself, before and after). Note that here correlation does not refer to a statistical test or method but to whether the group is being compared to itself (before/after or before during and after) all single group comparisons - pre and post or pre-during and post are CORRELATED!!!

ALL COMPARISONS USING A CONTROL GROUP OR A COMPARISON STATISTIC ARE INDEPENDENT.

4. How many groups OR MEASURES are there that I am comparing? Or how many variable are there that I am associating/ PRE-DURING AND POST = MORE THAN; TWO TIME SERIES = MORE THAN TWO. ALSO COMPARING MORE THAN TWO GROUPS –LIKE FIRST, SECOND AND THIRD GRADES OR 3 DIFFERENT NURSING HOMES.

SO NOW THAT YOU HAVE ANSWERED QUESTIONS 1 & 2, YOU MUST ANSWER QUESTIONS 3 & 4.

One last thing. You need to know if your groups (samples) ARE CORRELATED OR INDEPENDENT!!

ANYTIME YOU COMPARE A GROUP WITH ITSELF (BEFORE AND AFTER OR BEFORE DURING AND AFTER THE GROUPS ARE CORRELATED. SO A PRE-EXPERIMENTAL SINGLE GROUP DESIGN USING PRETEST AND POST TEST IS A CORRELATED GROUP

ON THE OTHER HAND, IF YOU HAVE A PRE-DETERMINED STANDARD - SUCH AS A PERCENTAGE REDUCTION OF A VARIABLE (INCIDENTS OF BULLYING ABOVE THE YEAR BEFORE ) THE GROUPS ARE UN-CORRELATED (EVEN THOUGH YOU ARE COMPARING NUMBER OF INCIDENTS WITH THE SAME GROUP THE YEAR BEFORE, THE ESTABLISHMENT OF COMPARING IT TO A STANDARD MAKES IT UNCORRELATED OR INDEPENDENT.

LIKEWISE USING A CONTROL GROUP WITH MY PRGRAM GROUP MEANS USING A DIFFERENT GROUP THAT IS NOT REALLY RELATED TO MY PROGRAM GROUP. IT TOO IS UNCORRELATED OR INDEPENDENT.

o.k. so if you know the level of measurement you are using, whether you are using a difference or an association, whether the group is correlated or independent and the number of measures you are using, you can pick the type of statistical analysis needed!

What do you mean the number of measures? If I am comparing a group of clients in a day program on number of days in the hospital BEFORE and AFTER entry into the program, those are two sets of measures.

If I am comparing a group of clients in a day program on number of days in the hospital BEFORE, DURING and AFTER entry into the program, those are three sets of measures.

If I am comparing the number of ‘reported incidents of bullying’ after

the start of the program to an outcome of ‘33% reduction in the pre-program number of incidents of bullying the year before’, those are two sets of measures. On the other hand,

If I am comparing the number of ‘reported incidents of bullying’ after the start of the program to outcomes of ‘33% reduction in the pre-program number of incidents of bullying the year before’, and a‘50% reduction in the pre-program number of incidents of bullying the year before’ those are three sets of measures.

• Lets go back to the charts

OUTCOME MEASURE

WITH NOMINAL MEASURES OF OUTCOMES, WHEN I COMPARE A GROUP TOA CONTROL GROUP (OR C.S.), THE CHI SQUARE IS THE TEST. WHEN COMPAREA GROUP TO ITSELF, IT IS CORRELATED AND I USE McNEMAR

variables variables

OUTCOME MEASURE

SO FOR INTERVAL LEVEL DATAWITH TWO GROUPS.MORE THAN TWO GROUPS

FOR INTERVAL DATAWITH A SINGLE GROUPPRE AND POST (2 MEASURES). BUT A SINGLE GROUP WITH MULTIPLE MEASURES

• NOW YOU SHOULD BE ABLE TO PICK THE CORRECT METHOD OF DATA ANALYSIS FOR OUTCOME RESEARCH.

Writing section 3

1. process = primarily qualitative; outcome = primarily quantitative.

2. read examples for each on the web see mine and students.3. pay attention to important concepts i.e. comparison statistic,

triangulation etc.4. Make an outline-look at the form of the examples. Follow the

outline I give – take notes5. you may pick one person to work with you-it must be

reciprocal and you must tell me who they are.6. use “questions about section 3”. I will stop answering on april

11th.7. this section requires forethought and planning. You should

have many questions before you write.