1

Adapting Research Methods for Diverse Ethnic Groups

Eliseo J. Perez-Stable, M.D.Anita Stewart, Ph.D.

Anna Nápoles-Springer, Ph.D.University of California, San Francisco

Clinical Research with Diverse CommunitiesEPI 222, Spring 2002

2

Overview

Integrating qualitative and quantitative methods

Using focus groups to develop questionnaires and community interventions

Using cognitive interviewing to develop questionnaires

3

What are Qualitative Methods?

Data consist of words, not numbers Richly descriptive, open-ended Focus on inductive analytic approaches Many types: ethnography, participant-

observation, direct observation, interviews, focus groups

4

When are Qualitative Methods Useful?

Open-ended interviews typically used in quantitative research in new areas of study

Especially critical in cross-cultural studies due to lack of information

Useful when need in-depth knowledge about issues especially with less studied cultural groups

5

When are Qualitative Methods Useful?

To understand the meaning of participants’ events, situations, and actions

To understand contextual influences on participants’ actions

To identify unanticipated phenomena and influences (e.g., exploratory studies to design questionnaires and identify variables for study)

To understand the processes underlying observed relationships between variables

6

Combining Qualitative and Quantitative Methods

Qualitative Methods to Develop Quantitative Tools

Qualitative Methods to Explain Quantitative Results

Qualitative ResultsQuantitative

Quantitative Results

Qualitative

7

Combining Qualitative and Quantitative Methods cont.

Quantitative Methods to Expand on Qualitative Study

Quantitative and Qualitative Methods Equal

Qualitative Results

Quantitative

Qualitative Results Quantitative

(Tasjakkori A. Sage Publications, Inc. 1998)

8

Sampling In Qualitative Research

Quantitative research-2 types of sampling:

– Probability

– Convenience Qualitative research-3rd type of sampling:

– Purposeful»Deliberately select settings, persons or

events to best answer research questions

9

Goals of Purposeful Sampling

Achieve representative, typical settings, individuals, or activities

Adequately capture heterogeneity, sample for broadest range of variation rather than typical individuals

Examine extreme cases that test theory Set up contrasts to examine differences between

settings or individuals(Maxwell JA. Sage Publications, Inc., 1996)

10

Qualitative Methods to Develop Quantitative Tools

Focus Group-– Open-ended guided group discussion with

probing of responses

Cognitive Interviewing– Individual interviews using open-ended probes to

assess how items are interpreted and adequacy of response choices

11

Open-ended Interviews to Develop Structured Questionnaire

Example: Constructing attitudinal scales Use open-ended interviews (individual or

group) to obtain statements on topic of interest Organize statements into “item pool” Pretest closed-ended items Modify items based on pretest and administer

in structured interview

12

Overview

Integrating qualitative and quantitative methods

Using focus groups to develop questionnaires and community interventions

Cognitive interviewing to develop questionnaires

13

Focus Groups

Group discussion led by experienced moderator usually 1.5 - 2 hours

Purposeful sampling of 6-10 homogenous participants per group

Use open-ended questions with follow-up probes for clarification, usually covering 10-12 topics

Participants stimulate comments of others Audio-record and transcribe discussion

14

Focus Groups-Logistics

Moderator skills: listening, communication, negotiation, cultural similarity to participants

Costs of group: $600 - $1000 per group (incentives, audio-taping, transcription, translation, food)

Convenient and hospitable community setting In-person recruitment with telephone and mail

follow-up

15

Latino Smoking Attitudes - From Focus

Groups to Community Intervention

Family concerns - 2nd hand smoke Heightened health issues Importance of appearance and

interpersonal relations - Simpatía Habitual use less important Addiction less of a concern

16

The Subjective Culture of Smoking: Focus Groups

Examine similarities and differences in attitudes, beliefs and behaviors (Triandis)

Sampling stratified by comparison groups Probability vs. purposeful sampling Antecedents and consequences Use information to design intervention

17

The Subjective Culture of Smoking - Structured Questionnaire

17 antecedents to smoking (habitual activities, social activities, emotional states)

15 reasons to quit (family, appearance, health)

3 reasons to keep smoking (weight, nervousness, concentration)

18

Reasons for Trying to Quit SmokingLatinos and Anglos, 1990

Latinos Anglos Difference 95% CI diff

Effect on others 62.8 51.7 +11.1 (+18.9, +3.3)

Smell 51.6 33.7 +17.9 (+25.6, +10.2)

Cost 50.6 48.6 +2.0 (+9.9, –5.9)

Wrinkles 49.0 33.9 +15.1 (+22.8, +7.4)

Criticized by family 43.9 24.0 +19.9 (+27.3, +12.5)

Family pressure 40.7 23.4 +17.3 (+24.6, +9.9)

Burn clothes 40.1 25.1 +15.0 (+22.4, +7.6)

19

Reasons for Continuing to SmokeLatinos and Anglos, 1990

Latinos Anglos Difference 95% CI diff

Not to gain weight 19.9 26.1 –6.2 (+0.5, –12.8)

Feel less nervous 49.4 36.8 +12.6 (+20.4, +4.8)

Helps concentration 28.8 29.7 –0.9 (+6.3, –8.1)

20

Multivariate Model for LatinosReasons to Quit or Continue

OR 95% CI

Criticized by family 1.93 (1.26, 2.98)

Burn clothes 1.57 (1.02, 2.42)

Children's' health 1.67 (1.08, 2.57)

Bad breath 2.07 (1.40, 3.06)

Family pressure 1.69 (1.10, 2.60)

Good example 1.83 (1.21, 2.76)

Not to gain weight 0.38 (0.24, 0.59)

21

Community Intervention: Programa Latino Para Dejar de Fumar

Guia Para dejar de Fumar--self-help guide

Electronic media in Spanish

Emphasis on family and collective orientation of

culture

Immediate effects of smoking

Health effects in different context

Physicians role

22

Overview

Integrating qualitative and quantitative methods

Using focus groups to develop questionnaires and community interventions

Cognitive interviewing to develop questionnaires

23

Distinction Between International and U.S. Studies - Why Pretest?

International studies assume conceptual non-equivalence to begin with– Different nations, languages

Usually dealing with translated measures During translation, items can be added or

modified to improve conceptual and semantic equivalence– Product is an “adapted” instrument

24

Typical International Approach

AssessConceptual Equivalence(Qualitative)

AssessPsychometricEquivalence(Quantitative)

Begin here(assumesconceptualdifferencesacross countries)

• If new domains or definitions are found, can revise and add items• Translated “adapted” version is the goal• Assures conceptual adequacy prior to testing psychometric adequacy

25

Typical U.S. Approach in Studies of English Speaking Diverse Groups

Select existing well-tested measures (developed in mainstream) and assume they will work (universality)

Assumes perspectives of diverse group are similar to mainstream– “Cultural hegemony” (Guyatt)– “Middle-class ethnocentrism (Rogler)

26

Typical U.S. Subgroup Approach When No Translation is Done

Most studiesbegin here(assumes universality of constructs)

If problems

Ifequiv.

AssessConceptualEquivalence(Qualitative)

AssessPsychometricEquivalence

(Quantitative)

No Guidelines!If refine items based

on qualitative studies, no longer have comparable

instrument

Proceed with analysis. May miss important domains

and definitions

27

Special Script for Recruiting People for Cognitive Interviewing

Often do not understand their role Explain how their help fits into the larger

study, goal of main study, process of creating questions

Explain their role clearly: – “help us learn how to ask better questions”– “help us make questions clearer for others”

28

Cognitive Interviewing:Purposes of Probing Questions To learn .. .. if respondents understand the words and

phrases the way you intended (meaning)? .. about the process of answering the questions .. about usefulness of response choices

-Whether response choices are adequate -How they use the response choices .. whether item might be hard or unacceptable

Results can be used to revise items

29

Cognitive Interviewing:Example of Probing Questions

Meaning of words/phrases and questions What does the word _______ mean to you? What does the phrase ________ mean to

you? I asked you ______. What did you think of

when I said ______?

30

Sample Result:Probing the Meaning of a Phrase

I’m going to ask you questions about how the office staff treated you personally ….What does the phrase “office staff” mean to you?

“the receptionist and the nurses”“nurses and appointment people”

“the person who takes your blood pressure and the clerk in the front office”

(We intended receptionist and appointment staff)

31

Sample Result:Probing the Meaning of a Phrase

I asked you how often doctors ask you about your health beliefs. What does the term “health beliefs” mean to you? “.. I don’t want medicine”

“.. How I feel, if I was exercising…” “.. Like religion? --not believing in

going to doctors?”

We changed the question to “personal beliefs about your health”

32

Sample Result: Probing the Meaning of a Phrase (Pasick et al. 2000)

During the last 12 months, how many times have you visited a doctor or other health professional just for a checkup (physical examination) - even when you were feeling well.

Chinese women: why would you go to a doctor if you were not sick?

Latina women: could not rephrase the question African American women: hypertension so prevalent,

could not distinguish “monitoring” from general checkup

33

Probe on Difficulty in General

Can ask respondents whether they think others would have difficulty answering a question or would answer the question honestly

34

Example of Probe on Difficulty: CES-D Item

“During the past week, how often have you felt that you could not shake off the blues, even with help from family and friends”

Probe: Do you feel this is a question that people would or would not have difficulty understanding?

– Latinos more likely to report people would have difficulty than other groups

TP Johnson, Health Survey Research Methods, 1996

35

Example:Probing the Process of Answering

I asked you _____ and you answered____. – Why did you pick this answer?– What were you thinking of when you picked this

answer? – Can you tell me what you were thinking when you

answered this way?» Can you give me some examples?

What came to mind when I asked you _____?

36

Sample Result: Probing the Process of Answering

When I asked you how often doctors gave you a chance to say what you thought was important, you answered “rarely” - what were you thinking of when you picked your answer?

“Sometimes I would be slow thinking, by the time I ask something, they are gone.... my doctor knows how to exit that door.”

37

Example: Use of Response Scale

Do diverse groups use the response scale in similar ways?

Sample Result: on questions about cultural competence of providers… .. interviewers reported that Asian respondents who were completely satisfied did not like to use the highest score on the rating scale

CPEHN Report, 2001

38

Sample Result: Use of Response Scale

In an exercise class of Samoans, instructor asked them to rate the difficulty of the exercise he just did on a 1-10 scale

They did not understand what a 1-10 scale was

39

Sample Result:Learning about Acceptability

Comments during interview to question “How often did doctors ignore your feelings?” “What do you mean ignore my feelings? Does that mean I’m bawling in front of him?”

“What type of feelings? ..when they ask “how are you feeling? Other kinds of feelings?”

“This type of question would only be asked in extreme cases. This question is too intimate.”

40

Other Cues to Problems in Face-to- Face Pretests

When administering the survey… Be aware of behavioral cues related to

specific items or to questions in general– Long pauses in answering

– Discomfort

– Yawning

– Looking at their watch

41

Interviewer’s Role

Be flexible during interview– probe on items that appear to be problematic

If a long pause in answering..– “I noticed you pausing - what were you

thinking about answering that question?”

42

Approaches to Adapting Standard Measures

Add parenthetical phrases where words are hard for target group to understand

Substitute more culturally appropriate examples– e.g., limitations in moderate activities such as

playing tennis - substitute soccer If items need substantial adaptation,

– Administer new and “standard” items – Can analyze measurement properties of standard

measure and “adapted” measure

43

Tradeoffs of Using Adapted Measures

If “adapted” measure works better…– You improved internal validity - able to answer

your question with this measure – You lost external validity - can’t compare your

scores to other studies If “adapted” measure does not work …

– Can still use original measure » It may not work either

44

What to do if Measures Are Not Adequate or Equivalent in a Specific Study

Need guidelines for how to handle data when substantial non-comparability is found in a study– Drop bad or biased items from scores

» Compare results with and without biased items

– Analyze study by stratifying diverse groups The current challenge for measurement

45

Conclusions:Integrating Quantitative and Qualitative Methods

In diverse populations, qualitative work is necessary in addition to more traditional quantitative studies

Prior to quantitative: to develop concepts, items appropriate to culture

After quantitative: – to help identify reasons for items not performing well

quantitatively– to explore possible explanations for unexpected results