Date post: | 20-Feb-2023 |
Category: |
Documents |
Upload: | fami4real2000 |
View: | 1 times |
Download: | 0 times |
Quantitative Research: Important issues pertaining to research methodology &
analysis strategy
Helene Muller
1
Quantitative Research: Important issues pertaining to research methodology & analysis strategy
Outline
Quantitative research
– Where does Quantitative Research slot in in the bigger Research picture?
– The survey research design in quantitative research
– Questionnaire design
– Reliability and validity
Analysis strategy: options explained by means of examples
– Example 1 (1 variable, categorical)
– Example 2 (2 variables, categorical)
– Example 3 (2 variables, continuous, different measures)
– Example 4 (2 variables continuous, same measure)
– Example 5 ( 2variables, continuous and categorical)
2
Quantitative Research: Important issues pertaining to research methodology & analysis strategy
Concepts in Quantitative Research
3
Concept 1: Where does Quantitative Research slot in within the Research Process?
1. When conducting research, the research approach can be either: – A qualitative (collected data: words (interviews), pictures (video)or objects (artefacts) – or a quantitative ( data collected: numerical data) – or a mixed approach. The research approach followed determines the Research Design
2. Research Designs A quantitative approach is appropriate when the research purpose is to test predictive and/ or
cause-effect type of hypotheses. Deductive logic is used and the design is suitable for phenomena that have been well developed with regard to theory and concept
Qualitative designs on the other is applicable when the research purpose is to accumulate knowledge of a (social) phenomenon that is relatively unknown and seeks understanding. Reasoning in this instance is more inductive
A mixed research design often strengthen research in the sense that the abovementioned approaches can be used to validate and enhance the research findings of each approach. (De Vos, et al 2005)
Quantitative Research: Important issues pertaining to research methodology & analysis strategy
Concept in Quantitative Research (Cont.)
Concept 2: The Survey Research Design in Quantitative Research
Most of the quantitative research in Educational studies adopts a survey design type.
This presentation will focus on the survey research design
Why is the Survey design appropriate? These design type has certain properties
• In quantitative research a ‘tool/ or measuring instrument’ is required to evaluate the phenomenon under investigation • The ‘tool-of-measurement’ in survey designs is the questionnaire (standardized or designed) • The design type generates/ collects numeric data • The information collected is objective and reliable (if properly structured, administered and validated) • Reliable findings because respondent selection & analysis processes adhere to statistical principles • The questionnaire is purpose-designed according to structured theoretical knowledge and research questions • Collects only essential information
4
Quantitative Research: Important issues pertaining to research methodology & analysis strategy
Concept in Quantitative research (cont)
Concept 3: Questionnaire design
**Three crucial principles to remember/ consider when designing a questionnaire
1. The format/type of questions determine the type of data generated – Closed ended questions (A list of possible options to choose from)
o Single choice (‘male’ / ‘female’) Categorical data, nominal o Multiple choice response (“all hobbies of interest listed”) Categorical o Rating levels (‘agree’ / ‘undecided / ’disagree’) Categorical, ordinal o Exact figures (weight, age) Continuous
– Open ended (Written text response) Categorical data
2. The type of data generated dictates the type of statistical analyses that can be performed
3. The analysis techniques applied determines the type of research questions (hypotheses) that can be addressed and answered
5
Quantitative Research: Important issues pertaining to research methodology & analysis strategy
Concepts in Quantitative research (cont)
Concept 4: Reliability and validity in research
Quantitative research strives to present valid and reliable research finding.
• Reliability refers to the consistency of a measure. A questionnaire/ or test is considered reliable if the same result is obtained repeatedly when the questionnaire is re-administered/ or tested repeated (Thermometer reading)
• Validity refers to the extent to which a questionnaire / or test measures what it purports to measure (Thermometer does not measure blood pressure)
There are various types of validity and reliability involved in the various stages of the quantitative research process. Measures have been developed to ensure valid & reliable research findings. That is why quantitative research adhere to
– Protocol for questionnaire design
– Protocol while administering a questionnaire
– Measures to ensure data integrity
– Measures to ensure validity and reliability (appropriate analysis strategy)
6
Quantitative Research: Important issues pertaining to research methodology & analysis strategy
Analysis strategy
• Once a sample for the research has been selected (according to a selection protocol/ sampling method) and questionnaires administered, the responses of the respondents is electronically captured as data(examples to follow)
• The next step is the research process is then the analysis strategy. The analysis strategy refers to the type of statistical analysis techniques the researcher plans to use in analyzing the data
• The analysis strategy, along with questionnaire design, determines the type of research questions and hypotheses that the researcher will be able to answer from the analysis results.
• The importance of the previous statement is best illustrated by means of example. The rest of the presentation discusses analysis techniques and their applicability to answering specific types of research questions
7
Questionnaire: Reading methods
Section A
1. Please indicate your age
2. Please indicate your gender
3. Years of experience teaching junior level primary school
Reading methods questionnaire 2
4 – 8: Other biographical attributes
Section B
9. Please tick the reading approaches/ methods that you use in your classroom
Method √
1. Phonics
2. Look-&-say
3. Eclectic
4. Language expression
5. THRASS
6. Readers
7. Other books in class
1. >30
2. 31-40
3. 41-50
4. 51-65
1. male 2. Female
1. <11 yrs 2. 11-20 yrs 3. 21+yrs
8
Quantitative Research: Important issues pertaining to research methodology & analysis strategy
Example questionnaire
Reading methods questionnaire 3
10. Please provide the class average for the following first term evaluations
11. Please provide class average of the second term reading evaluation before & after introduction of Look & Say reading approach
Reading methods questionnaire 4
12. Please indicate your level of agreement , with the following statements, where 1: agree++ 2: agree 3: undecided 4: disagree 5: disagree++
Class average, Reading ………
Class average, Spelling ……….
Class average Before ……….
Class average After ……….
Agreement statements 1 2 3 4 5
1. Reading skills are important
2. Reading skills form the basis of the learning process
3. Sound reading habits should be fostered early
4 . Bed time stories contribute towards love of reading
5. Parents’ reading habits determine their children‘s
6. One can never read enough
9
Quantitative Research: Important issues pertaining to research methodology & analysis strategy
Example questionnaire (continued)
Quantitative Research: Important issues pertaining to research methodology & analysis strategy
Example: Questionnaire, electronically captured data
Electronically captured data from Reading Methods Research Questionnaire
no
Age
Gender
Exp
Q9.1
Q9.1
Q9.3
.
.
.
.
Q9.7
R e a d
Write
Before
After
q1
q2
q3
q4
q5
q6
1 <30 M 1 1 1 1 . 0 43 55 43 55 2 3 1 2 2 1
2 51-56 F 3 1 1 0 . 0 56 77 56 77 3 1 2 2 3 1
3 31-40 M 3 0 1 0 . 1 78 79 76 79 1 1 3 2 2 1
4 31-40 M 2 0 1 1 . 0 54 56 78 71 4 4 5 3 3 3
5 <30 F 3 1 1 1 . 1 65 79 56 67 1 2 2 1 1 2
6 41-50 F 1 1 0 1 .. 0 66 72 65 78 2 1 1 1 1 3
7 41-50 F 1 1 1 0 .. 0 79 84 78 82 2 2 1 3 1 1
… … … … … … … … …
...
… … … … … … … … …
10
• Type of data recorded and analysis techniques applicable:
• Type of research questions that can be answered with the data collected from the questionnaire
Categorical data on biographical properties (age, gender, exp.)
Therefore limited analyses & non-parametric tests, such as • One-way frequency tables & graphs (bar graphs, spider plots, pie charts) •Descriptive techniques
Research questions pertaining to a description of the sample (the respondents selected) in terms of their biographical attributes can be answered:
“Fifty seven percent of the respondents in the sample were younger than 35” ….. Gender ………. Experience ……….
11
Quantitative Research: Important issues pertaining to research methodology & analysis strategy
Example 1: Section A, Analysis strategy
Analysis technique: One way frequency table
Table 1 Frequency distribution of student respondents
in Reading methods research project
Age category
Frequency Percentage Total
<30 27 36.0 27
30-35 16 21.3 43
36-40 15 20.0 58
41-45 11 14.7 69
46+ 6 8.0 75
0
5
10
15
20
25
30
Age distribution
Age distribution
12
Quantitative Research: Important issues pertaining to research methodology & analysis strategy
Example 1: Analysis strategy, Section A (Age)
• Type of data recorded and analysis techniques applicable:
• Type of research questions that can be answered with the data collected from the questionnaire
Categorical, multiple responses (‘select’ (0) or ‘not selected’ (1))
Non-parametric classification techniques (one-way and two-way ), such as • One-way and two-way frequency tables & graphs (bar graphs) • Chi-square tests to establish dependencies are appropriate techniques
Research questions pertaining to • a general overview of a specific trend in the research
“Phonics & Readers are the two most commonly used reading methods”
• dependency between two classification attributes (yes/no & reading
method) “ The frequency-of-use(yes/ no) and reading method is dependent: implying frequency of
use of Reading methods differ significantly”
13
Quantitative Research: Important issues pertaining to research methodology & analysis strategy
Example 2: Analysis strategy Section B, q9
Method Yes, use No Total
Phonics 74 2 76
Look&Say 48 27 75
Eclectic 43 32 75
L _Express 31 44 75
THRASS 11 64 75
Readers 69 4 73
O_ Books 57 17 74
Probability of (Chi-square statistic = 168.40) < 0.0001 ***
74
48
43
31
11
69
57
2
27
32
44
64
4
17
Utilization of reading methods
Yes No
14
Quantitative Research: Important issues pertaining to research methodology & analysis strategy
Example 2: Analysis strategy, Section B q9 .
• Type of data recorded and analysis techniques applicable:
• Type of research questions that can be answered with the data collected from the questionnaire
Continuous data: two sets of attribute measures (read & spell tests)
Parametric analysis techniques, such as • Linear regression analysis & graphs (linear regression line and scatter plot) • F & t tests to establish significance of linear trend describing the relationship between the two measures
Research questions pertaining to a more detailed description of the relationship between two measures on the same respondents :
•“A statistically significant relationship between reading & spelling averages of educator respondents exist” • “ The relationship can be describe by the following regression line: Read avg = 51.1 + 1.33*(Spell avg.)”
15
Quantitative Research: Important issues pertaining to research methodology & analysis strategy
Example 4: Section B q11, Analysis strategy
30
40
50
60
70
80
90
100
30 50 70 90 110
Y-a
xis:
Tes
t 2
Regression Test 1 against Test 2 X-axis test 1
Test 2
Linear (Test 2)
Respondent no
Read Test1
Spell Test2
1 43 55
2 56 77
3 78 79
4 54 56
5 65 79
6 66 72
7 79 84
… … …
16
Quantitative Research: Important issues pertaining to research methodology & analysis strategy
Example 3: Analysis strategy, Section B, q10
• Type of data recorded and analysis techniques applicable:
• Type of research questions that can be answered with the data collected from the questionnaire
Continuous data: one attribute two ‘paired measures’ (‘before’ & ‘after’ )
Parametric & non-parametric analysis techniques, such as • Paired difference test Students’ t-test (parametric) or Signed Rank test (non-parametric)
Research questions pertaining a different kind of relationship between two measures of the same attribute – ‘before’ and after’ measures:
• “Does a statistically significant relationship exist between the two measures of the same
attribute? (reading skills)” • “Did the intervention treatment implemented between the before & after evaluation improve reading skills?’
17
Quantitative Research: Important issues pertaining to research methodology & analysis strategy
Example 4: Analysis strategy, Section B q10
Respond no
Before After Diff(After-Before)
1 43 55 12
2 56 77 21
3 76 79 3
4 78 71 -5
5 56 67 11
6 65 78 13
7 78 82 4
….. ….. ….. …..
Parametric and non-parametric difference test results
Students’ t-test: Probability of (t-statistic = 2.64) = 0.039 *
Signed Rank test Probability of (S-statistic=11) = 0.078
Significance legend: * : 5% level of significance ** : 1% level of significance ***: 0.1% level of significance
18
Quantitative Research: Important issues pertaining to research methodology & analysis strategy
Example 4: Analysis strategy, Section B q10
• Type of data recorded and analysis techniques applicable:
• Type of research questions that can be answered with the data collected from the questionnaire
Continuous & Categorical data (attitude & age)
Various parametric and non-parametric analysis techniques are applicable • Calculation attitude scales, verification of internal consistency reliability • Analysis of variance on attitude scale values to evaluate effect of biographical attributer on attitudes • Bonferroni multiple comparison of means test to describe nature of biographical effect
Research questions pertaining to the general attitude-trend exhibited by respondents and the effect of biographical attributes on their reading attitude • “ Do respondents exhibit a positive attitude towards reading?” • “ Does age significantly affect respondent’s attitude towards reading? “ • “ Do younger people exhibit a more negative attitude than older people?”
19
Quantitative Research: Important issues pertaining to research methodology & analysis strategy
Example 5: Analysis strategy, section B q12
Data for q 12
Respond no
Age q1 q2 q3 q4 q5 q6
1 <30 2 3 1 2 2 1
2 51-56 3 1 2 2 3 1
3 31-40 1 1 3 2 2 1
4 31-40 4 4 5 3 3 3
5 <30 1 2 2 1 1 2
6 41-50 2 1 1 1 1 3
7 41-50 2 2 1 3 1 1
….. …..
• Scale reliability testing results
• Calculate attitude score for each respondent .
Example 1st respondent:
score₁ = (2+3+1+2+2+1)/5 = 2.2
• Analysis of variance results
Scale reliability test
Construct Items
included Cronbach
alpha Mean score
Std dev
Read attitude
q12.1-q12.6
0.79 1.23 0.23
Summary Analysis of variance & Bonferroni multiple comparisons of means test results
Construct F statistic (F prob)
Age Read att. Score
Mean
Read score 171.23
(0.0001) *** 0.011**
<30: 4.21 a 31-40: 2.98 a 41+: 1.44 b
20
Quantitative Research: Important issues pertaining to research methodology & analysis strategy
Example 5: Analysis strategy, section B q12
Quantitative Research: Important issues pertaining to research methodology & analysis strategy
Conclusion
• The importance of – properly planned quantitative research,
– carefully formulated research questions
– cleverly designed questionnaire according to the research questions/ hypotheses stated cannot be over-emphasized
• Enjoy your studies!
21
• http://psychology.about.com/od/researchmethods/f/reliabilitydef.htm
• Wikipedia
• De Vos, Strydom, Fouche, Delport
22