Quantitative and Qualitative Research Design
Prepared By:Dr. Nik Rahimah Yacob
Positivist Paradigm
Quantitative Methods
Verification
Interpretivist Paradigm
Qualitative Methods
Discovery
The Research Process by Cavana et al. (2001)Catalyst for business research
• Opportunity
• Problem
Preliminary information gathering and literature survey
Problem definition
Framework development
• Conceptual
• Theoretical
Research objectives
• Research questions
• Hypotheses
Research design Data collection
• Qualitative
• Quantitative
Data analysis
• Qualitative
• Quantitatve
Interpretation of findings
Report preparation and presentation
Management action
• Plan
•Implement
• Monitor
Research Design by Cavana et al. (2001)
Problem Statem
ent
Purpose of the study
• Exploration
• Description
• Hypotheses testing
• Case Study
Types of investigation
• Clarification
• Causal
• Correlational
• Experimental
Extent of Researcher Interference
• Minimal
• Manipulation
Measurement and measures
• Operational definition
• Items (measure)
• Scaling
Qualitative data collection
• Interviews
• Focus groups
• Observation
Data analysis
Unit of Analysis
• Individuals
• Dyads
• Groups
• Organisations
• Machines, etc.
Study setting
• Contrived
• Non-contrived
Time Horizon
• One-shot (cross-sectional)
• Longitudinal
Sampling design
• Probability/ non-probability
• Sample size
Quantitative data collection
• Questionnaires
• Experimental designs
A Classification Of Research Data
RESEARCH DATA
SECONDARY DATA PRIMARY DATA
QUALITATIVE DATA
QUANTITATIVE DATA
DESCRIPTIVE CAUSAL
EXPERIMENTAL DATA
OBSERVATIONAL & OTHER DATASURVEY DATA
QUANTITATIVE DATA
QUALITATIVE DATA
A Classification of the Qualitative Research Methods
QUALITATIVE RESEARCH PROCEDURES
DIRECT (NONDISGUISED)
INDIRECT (DISGUISED)
FOCUS GROUPS
PROJECTIVE TECHNIQUES
EXPRESSIVE TECHNIQUES
CONSTRUCTION TECHNIQUES
COMPLETION TECHNIQUES
DEPTH INTERV
ASSOCIATION TECHNIQUES
SECON-DARY DATA
CASE STUDY
Quantitative Research Approaches/Methods
A Classification Of Survey Methods
SURVEY METHODS
TELEPHONE PERSONAL MAIL
IN-HOME MALL INTERCEPT
COMP-ASSISTED PERSONAL
INTERVIEWING
MAIL INTERVIEW
MAIL PANELTRADITIONAL
TELEPHONECOMPUTER-ASSISTED
TELEPHONE INTERVIEWING
A Classification Of Observation Methods
OBSERVATION METHODS
PERSONAL OBSERVATIO
N
MECHANICAL OBSERVATIO
NAUDIT
CONTENT ANALYSIS
TRACE ANALYSIS
A Classification Of Experimental Designs
EXPERIMENTAL DESIGNS
PREEXPERIMENTAL TRUE EXPERIMENTAL
QUASI EXPERIMENTAL STATISTICAL
One-Shot Case StudyOne Group Pretest-PosttestStatic Group
Pretest-Posttest Control GroupPosttest Only Control GroupSolomon Four Group
Time SeriesMultiple Time Series
Randomized BlocksLatin SquareFactorial Design
Qualitative Research Approaches/Methods
The continuum model for interviews
Structured interviews Semi-structured interviews Unstructured interviews
• Standardised interviews* Survey interviews
* In-depth interviews* Survey interviews* Group interviews
* In-depth interviews* Group interviews• Oral or life-history interviews
The Pattern of an Interview (Source: Delahaye (2000:166))
Level of defence barriers
Time elapsed
Intimacy
Entrance time investment
Exit time investment
Rapport zone
Activity no.1
Activity no. 2 – a series of question sequences
Preview
Rules
Reason
Pass time
Ritual
Future action
Final questions
Pass time
Ritual
Focus Groups
Depth interviewing of a group of 5 to 12 people; researcher serves as a moderator
• Logistics• Group Composition• Homogeneity• Representation• Strangers vs acquaintances• Size of group
Case StudyBy understanding a single system a researcher canbetter understand similar instances and address the problems and issues identified in one case.
As bounded systems of time and space (context), casesrely on multiple sources of information to providean in-depth picture of an organisation or situation (phenomenon) under study.
Via observation, interviews, documents, or surveys.
Survey Methods
Survey as a Research Approach• It is a quantitative method• Capitalizes on the communication approach (respondents are required
to communicate their responses to the researcher through a structured or unstructured questionnaire)
• Involves the creation and selection of the measurement questions• Sampling issues which drive contact and call-back procedures• Instrument design which incorporates attempts to reduce error and
create respondent-screening procedures• Data collection processes which create the need for follow-up
procedures and possible interviewer training
A Classification of Survey Methods
Survey Methods
Cross-sectional
design
Longitudinal design
On Sampling…..• Capitalizes on a relatively large sample size• The sample can either be drawn on a probability
sampling procedure or a nonprobability sampling procedure, depending on the purpose of the study
On Instrument Design….• The instrument for a survey is a questionnaire. • The questionnaire can either be highly structured (close-ended
questions) or highly unstructured (open-ended questions) • The norm is to utilize a structured questionnaire for ease of
data coding and analysis. • The questionnaire design has to take into consideration the
data collection method (personal interview, telephone interview, mail survey, Internet survey, mall intercept or self-administered)
Data Collection Method
SURVEY METHODS
TELEPHONE PERSONAL MAIL
IN-HOME MALL INTERCEPT
COMP-ASSISTED PERSONAL
INTERVIEWING
MAIL INTERVIEW
MAIL PANELTRADITIONAL
TELEPHONECOMPUTER-ASSISTED
TELEPHONE INTERVIEWING
Examine the pros and cons of each method prior to selecting a particular method for your study.
Observation Methods
Observation as a Research Approach
• It can either be a quantitative method or a qualitative method depending on whether the purpose of your study is to verify or to discover
• Capitalizes on visual data collection. It also involves listening, reading, smelling and touching
• Monitors a full range of behavioural (nonverbal, lingusitic, extralinguistic and spatial analysis) and nonbehavioural (record, physical condition and physical process analysis) activities and conditions
• Sampling issues • Instrument design• Data collection process
A Classification of the Observation Methods
Observation methods
Direct observation
Indirect observation
On Sampling…..• Capitalizes on a relatively large sample size for a
quantitative study and a small sample size for a qualitative study
• The sample can either be drawn on a probability sampling procedure or a nonprobability sampling procedure for a quantitative study
• The sample is almost always drawn on a nonprobability sampling procedure for a qualitative study
On Instrument Design….• The instrument for an observation study is known as an
observation checklist or an observation form • The observation checklist or form can either be highly
structured (clear indications of what to observe and how to tally the observation) or highly unstructured (vague ideas on the scope of observation and what to observe)
• Observation checklist or form for quantitative research would tend to be more structured than that of a qualitative research
• The observation checklist or form has to take into consideration the data collection method (personal observation, mechanical observation, audit, content analysis or trace analysis)
Data Collection Methods for Observation Studies
OBSERVATION METHODS
PERSONAL OBSERVATIO
N
MECHANICAL OBSERVATIO
NAUDIT
CONTENT ANALYSIS
TRACE ANALYSIS
Examine the pros and cons of each method before selecting a particular method for your study
Experimentation
“Ex post facto research designs, where a researcher interviews respondents or
observes what is or what has been, also have the potential for discovering causality. The
distinction between these methods and experimentation is that the researcher is
required to accept the world as it is found, whereas an experiment allows the researcher to alter systematically the variables of interest
and observe what changes follow.”Source: Cooper & Schindler (2003, pp.424-425)
Beaumind Soft sdn bhd
What is experimentation?
• Involves at least one independent variable (IV) and one dependent variable (DV) in a causal relationship.
• The IV constitutes the intervention or manipulation and its effect on the DV is measured.
• Three requirements for drawing a causal conclusion are:
Concomittant variationTime occurrence of variablesControl over extraneous factors
Beaumind Soft sdn bhd
Advantages & Disadvantages of Experimentation
Beaumind Soft sdn bhd
ADVANTAGES:
• Ability to manipulate the IV
• Increases the probability that changes in the DV are a function of the changes in the IV
• Use of control group strengthens the causality finding
• Convenient and cost effective
• Ease of replication
DISADVANTAGES:
• Artificiality of the setting
• Generalizability of the study findings
• For some experiments, they can be costly
• Limited to issues of present and immediate future not past
• Due to ethical considerations, there is a limitation on manipulation on people
IV and Experimental Treatments
• Experimental treatments are the various levels in the manipulation of the IV
• Normally, one level of the IV constitutes the control• For the following RQ of an experiment:
Does the new ad generate more sales than the existing ad?
IV = Advertisement DV = SalesLevels of IV = Treatments = 1. New ad (experimental
treatment) 2. Existing ad (control)
Beaumind Soft sdn bhd
Sampling and Randomization• With two treatments in the experiment, there must be two
experimental groups• Randomization can happen at three stages: Stage 1 - Sample selection
Stage 2 – Group division Stage 3 - Assignment of treatment to group • Stage 1 randomization is not as important as the other stages. • Stage 2 randomization is important to equate the characteristics
of the two groups. When this is not possible then use a matching technique.
• Stage 3 randomization defines the experimental method. Without this randomization the experimental design is flawed.
Beaumind Soft sdn bhd
Experimental Research Designs
• Pre-Experimental DesignsOne-Shot Case Study X OOne-Group Pretest-Post-Test O X O
Static Group Comparison X O1
O2
• True Experimental Designs Pretest-Post-Test Control Group R O1 X O2
R O3 O4
Post-Test-Only Control Group R X O1
R O2
Beaumind Soft sdn bhd
Internal Validity of Experiments
• Maturation• Testing• Instrumentation• Selection• Statistical regression• Experiment mortality
Beaumind Soft sdn bhd
Depth Interview
Accuracy and Replicability1. Trustworthiness
2. Verification
3. Acknowledging subjectivity and bias
4. Process and sequence
5. Interpretation
6. Referential adequacy
7. Paint the path
It takes two people to speak the truth-one tospeak, and the other to listen (Henry Thoreau 1849)
Field Interview Techniques
• Interviewing in the field requires a researcher to develop a rapport with interviewees or informants.• Researchers often apprehensive about engaging strangers in conversation.• The objective is to keep the informant talking .
Field InterviewsBuilding Rapport / Relationships
Gain confidence through purposeful conversation.
By explaining what you want to know.Repeating key phrases used by interviewee.Restating what they say in your own words.
Don’t have questions asking for meanings or motives:“what do you mean?” or “why would you?”
These contain hidden judgments, interviewees may thinkthey have not clearly explained things or answered your questions!
Improving Interviewee Responses
The basic probe: repeat the initial question, is useful when interviewee wanders off track. Explanatory probe: building onto incomplete or vague statements;
asks: what did you mean by that? Focused probe: to obtain specific information;
asks: what sort of X? The silent probe: pause and let interviewee break silence drawing out: when interviewee has halted or slows simply repeat the last few words from the interviewee then look expectant or say tell me more Giving ideas or suggestions: offering an idea or suggestion to think about; have you thought about? Did you know? Mirroring or reflection: express in your (interviewer) own words what interviewee just said; what you seem to be saying is X?
Focus Group Method
Design Issues
• Logistics• Group Composition• Homogeneity• Representation• Strangers vs acquaintances• Size of group
Conducting the Focus Group
1. Use pattern of interview as guide2. Specific considerations
– Facilitator team– Recording– Use of visual aids– Thinking time– Group dynamics
Procedure for Planning and Conducting Focus Groups
Determine the objectives of the Marketing Research Project and Define the Problem
Specify the objectives of Qualitative Research
State the objectives/questions to be answered by Focus Group
Write a Focus Group Protocol
Develop a moderator’s outline
Conduct the Focus Group interviews
Review tapes and analyze the data
Summarize the findings and plan follow-up research or action
Case Study Method
Case studies
Case studies can be single or multiple
Within case analysis or across cases analysis
Case studies are: where a researcher explores a singleentity or phenomenon (the case) bounded by time and activity(a program, event, institution, process or group) and collects
detailed information by using a variety of data collection procedures (Yin 2001).
Selecting a Research Method Research Strategy Type of research
Question Requires Control over Events
Focus on Contemporary Events
Experiment HOW , WHY YES YES Survey WHO, WHAT, WHERE,
HOW MANY, HOW MUCH
NO YES
Archival Analysis WHO, WHAT WHERE, HOW MANY, HOW MUCH
NO YES/NO
History HOW, WHY NO NO Case Study HOW, WHY NO YES
Source: Yin (1989:17)
Case Study for four design tactics. Tests Case Study Tactic Phase of Research Construct Validity Use multiple sources of evidence.
Establish chain of evidence. Have key informants review draft report.
Data collection Data collection
Composition
Internal Validity Do pattern matching
Do explanation building Do time-series analysis
Data analysis Data analysis Data analysis
External validity Use replication logic in multiple case studies
Research design
Reliability Use case study Protocol Develop case study data base
Data collection
Data collection
Source: Yin(1989:p41)
Case study protocol components Case Study Protocol Components Component Requirements Overview Objectives & Auspices
Study Issues Relevant Readings
Field Procedures Credentials Access to Site General sources of Information Procedural Reminders
Case Study Instrument Questions Specific Questions Potential sources for Answers
Case Report Guide Outline Format Additional Documentation
Source: Yin(1989:p70)
Quality Case Study Designs(design & collection issues)
Construct Validity:Establish & follow appropriate (correct) operational measures for theConcepts (phenomenon) being studied.
External Validity:Establish the domain to which the findings can be generalised.
Reliability:Demonstrate that the operations of the study (data collection methods)Can be repeated, achieving similar results.
• All aspects must be adhered to throughout the case study life and in effect there is a strong link between design and collection.
Quality Case Study Designs(design & collection issues)
Establishing the Chain of Evidence (construct validity)
Case study report
Case database
Reference to specific sources in the database
Case study protocol
Case study questions
Source: Yin, 2003
Quality Case Study Designs(design & collection issues)
Generalisation (external validity)
Generalisation to THEORY not populations
Criticisms: cases cannot be generalised (or poor generalisation)
Generalisation to samples – universe is not correct as survey designs rely on statistical generalisation and
cases use theoretical generalisation through the results.
Generalisation requires replication via multiple cases of the phenomenon (this is replication logic)
Quality Case Study Designs(design & collection issues)
Reliability
Emphasis on replication
The key is doing the same case again, not replicating the findings, i.e., results
The objective of reliability is to minimise error and bias
This is achieve through correctly detailed plans of actionsundertaken (a-priori and post gathering) through protocols,Questions, histories etc
Basic Types of Case DesignSingle V Embedded Designs
Context
Case
SINGLE MULTIPLE
Case 1
Context
Case 2
HOLISTICSINGLE
EMBEDDEDMULTIPLE
Context
Case
Context
Case
Context
Case
Context
Case
Case 1Context
Case 2Case 1
Context
Case 2
Case 1Context
Case 2Case 1
Context
Case 2
Source: Yin, 2003
Basic Types of Case Design
Multiple Case Studies
Literal Replication V Theoretical Replication
Literal sees the purpose of the cases being to predictsimilar results (findings).
Theoretical sees the purpose of the cases being topredict different results (outcomes) for theoreticallypredictable reasons.
Basic Types of Case DesignSingle Case Study
Critical case: tests well established theory is correct or alternative explanations are found and acceptable.Extreme case: rare situation is found and studied to establish theory.Representative case: study situation identified to be common and theory explains how situation occurs or functionsRevelatory case: study a situation not previously able to be examined.Longitudinal case: same situation studied at multiple points of time to see how things change over the stages (time).