Lecture Research Design

7/27/2019 Lecture Research Design

1/38

LECTURE

1

ELEMENTS OF

RESEARCH DESIGN


2/38

THE RESEARCH PROCESS

2

OBSRVATION

Board area

of research

interest

identified

1

PRELIMINARY

DATA GATHERING

Interviewing

Literature survey

2

PROBLEM

DEFINITION

Research

problem

delineated

3

THEORETICAL

FRAMEWORK

Variables clearly

identified and

labeled

4

GENERATION

OF

HYPOTHESES

5

SCIENTIFIC

RESEARCH

DESIGN

6

DATA COLLECTION

ANALYSIS AND

INTERPRETATION

7

DEDUCTION

Hypotheses

substantiated?

Research question

answered?

8

Report

Writing

Report

Presentation

Managerial

Decision

Making

No Yes

9 10 11


3/38

THE RESEARCH DESIGN

A research design is a plan, structure andstrategy of investigation so conceived asto obtain answers to research questions or

problems. The plan is the complete schemeor program of the research. It includes anoutline of what the investigator will dofrom writing the hypothesis and theiroperational implication to the finalanalysis of data.

3


4/38

4

THE RESEARCH DESIGN

Purpose of the

study

Exploration

Description

Hypothesis testing

Types of

Investigation

Establishing:

-Casual relationships

-Correlations

-Group differences,

Extent of researcher

Interference

Minimum: Studying events

as they normally occur

Moderate: Minimum

amount of interference

Maximum: High degree

of control and artificial

settings

Study setting

Contrived

Noncontrived

Measurement

and measures

Operational

definition

items (measure)

Scaling

Categorizing

Coding

Unit of analysis

(Population to

be studied)

Individuals

Dyads

Groups

Organizations

Machines

etc.

Sampling

design

Probability/nonprobability

Sample

Size (n)

Time

horizon

One-Shot

(cross-sectional)

Multishot

(longitudinal)

Data-Collection

method

ObservationInterview

Questionnaire

Physical

measurement

Unobtrusive

1. Feel for data

2. Goodness or

data

3. Hypotheses

testing

PROBLEMS

TATEME

NT

DATA

ANALYSIS

DETAILS OF STUDY MEASURMENT


5/38

5

An exploratory study is undertaken when notmuch is known about the situation in hand, or

no information is available on how similarproblems or research issues have been solved inthe past.

Exploratory studies are also necessary whensome facts are known, but more informationis needed for developing a viable theoreticalframework.

EXPLORATORY STUDY

PURPOSE OF THE STUDY


6/38

6

A descriptive study is undertaken in order to ascertain and be able

to describe the characteristics of the variables of interests in asolution. For instance, a study of a class in terms of thepercentage of members who are in their senior and junioryears, sex composition, age groupings, number of semestersleft until graduation, and number of business courses taken,can be considered as descriptive in nature.

DESCRIPTIVE STUDY

Example

A bank manager wants to have a profile of the individuals who haveloan payments outstanding for 6 months and more. It would include

details of their average age, earnings, nature of occupation, full-time/part-time employment status, and the like. This might help him toelicit further information or decide right away on the types ofindividuals who should be made ineligible for loans in the future.


7/38

7

Studies that engage in hypotheses testing usuallyexplain the nature of certain relationships, orestablish the differences among groups or theindependence of two or more factors in a solution.

HYPOTHESES STUDY

Example

A marketing manager wants to know, the sales of thecompany will increase, if he doubles the advertising

dollars. Here, the manager would like to know thenature of the relationship that can be established

between advertising and sales by testing the hypothesis:If advertising is increased, then sales will also go up.


8/38

8

TYPES OF INVESTIGATION

CAUSAL VERSUS CORRELATIONAL

Causal study: The study in which the researcher wants todelineate the cause of one or more problems is called a causal

study.

Correlational study: When the researcher is interested in

delineating the important variables associated with the problem,the study is called a correlational study.

Example

A causal study question:

Does smoking cause cancer?

A correlational study question:

Are smoking and cancer related?

OR

Are smoking, drinking, and chewing tobacco associated with cancer? If so, which ofthese contributes most to the variance in the dependent variable?


9/38

EXTENT OF RESEARCHER INTERFERENCE

9

The extent of interference by the researcherwith the normal flow of work at the

workplace has a direct bearing on whether

the study undertaken is causal orcorrelational. A correlational study is

conducted in the natural environment of

the organization with minimuminterference by the researcher with the

normal flow of work.


10/38

STUDY SETTING: CONTRIVED AND NONCONTRIVED

Noncontrived settings: If organizational researchbe done in the natural environment where work

proceeds normally, the research is in noncontrived

settings.

contrived settings: If organizational research be

done in artificial environment the research is in

contrived settings.

Correlational studies are invariably conducted

in noncontrived settings, whereas most rigorous

causal are done in contrived lab settings.

10


11/38

CONTRIVED AND NONCONTRIVED SETTINGS

1. FIELD STUDY: If various factors are examined in the naturalsettings in which daily activities going on as normal withminimal researcher interference, the study is field study(noncontrived).

2. FIELD EXPERIMENT: If cause and effect relationships arestudied with some amount of researcher interference, butstill in the natural settings where work continues in thenormal environment, the study is field experiment(contrived).

3. LAB EXPERIMENT: If the researcher explores cause andeffect relationship not only exercising a high degree ofcontrol but in an artificial and deliberately created settings(contrived).

11


12/38

12

A bank manager wants to analyze the relationshipbetween interest rates and bank deposit patterns of

clients. She tries to correlate the two by looking at

deposits into different kinds of accounts (such as

savings, certificates of deposit, and interest-bearing

checking accounts) as interest rates changed.

This is a field study where the bank manager has merely

taken the balances in various types of accounts andcorrelated them to the changes in interest rates.

Research here is done in a noncontrived setting with no

interference with the normal work routine.

EXAMPLE OF FIELD STUDY


13/38

13

The bank manager now wants to determine the cause-and-effect relationship betweeninterest rate and the inducements it offers to clients to save and deposit money in the

bank. She select branches within a 60-mile radius for the experiment. For 1 week only,

she advertise the annual rate for new certificates of deposit received during that week in

the following manner: the interest rate would be 9% in one branch, 8% in another, and

10% in the third. In the fourth branch, the interest rate remains unchanged at 5%. Within

the week, she would be able to determine the effects, if any, of interest rates on deposit

mobilization.

The above would be a field experiment since nothing but the interest rate in

manipulated, with all activities occurring in the normal and natural work environment.

Hopefully, all four branches chosen would be more or less compatible in size, number of

depositors, deposit patterns, and the like, so that the interest savings relationships are not

influenced by some third factors. But it is possible that some other factors might affect

the findings. For example, one of the areas may have more retirees who many not have

additional disposable income that they could deposit, despite the attraction of a good

interest rate. The banker may not have been aware of this fact while setting up the

experiment.

EXAMPLE OF FIELD EXPERIMENT


14/38

14

EXAMPLE OF LAB EXPERIMENT

The bank manager now wants to establish the causal connection between interest rates

and saving, beyond a doubt. Because of this she wants to create an artificial

environment and trace the true cause and effect relationship. She recruit 40 students who

are all business majors in their final year of study and are more or less of the same age.

She splits them into four groups and gives each one of them amount of $1,000, which

they are told they might utilize to buy their needs or save for the future, or both. She

offers them an incentive, interest on what they save but manipulates the interest rates by

offering a 6% interest rate on savings for group 1, 8% for group 2, 9% for group 3, andkeeps the interest at the lowest rate of 1% for group 4.

Here the manager has created an artificial laboratory environment and has manipulated

the interest rates for savings. She has also chosen subjects with similar backgrounds and

exposure to financial matters (business students). If the banker finds that the savings by

the four groups increase progressively, keeping in step with the increasing rates of

interest, she would be able to established a cause and effect relationship between interestand the disposition to save.

In this lab experiment with the contrived settings, the researcher interference has been

maximal, inasmuch as the setting is difficult, the independent variable has been

manipulated, and most external contaminating factors such as age and experience have

been controlled.


15/38

Decision points for embarking on an experimental design

15

Is tracing causal

effects necessary?

Yes

and ifNo

Internal validity is

more important than

external validity

Generalizability is

more important than

internal validity.

Both internal validity and

external validity are

important.

Engage in a lab

experiment.

Engage in a field

experiment.

First do a Lab experiment,

then, a FIELD experiment.

Are there cost

constraints?

NoYes

Engage in a simpler

experimental design.

Engage in a more

sophisticated design.

Do not undertake an

experimental design study


16/38

UNITS OF ANALYSIS

The unit of analysis refers to the level of aggregation (bunch) of the data collected during thesubsequent data analysis stage.

If the problem statement focuses on how to rates levels of employees in general, then we areinterested in individuals employees in the organization and would have to find out what we can doto raise their motivation. Here the unit of analysis is the individual.

If the researcher is interested in studying two-person interactions, then several two-person groups,also known as dyads.

If the problem statement is related to group effectiveness, then the unit of analysis would be at thegroup level.

If we compare different departments in the organization, then the data analysis will be done at thedepartmental level.

If we compare different organizations, then the data analysis will be done at the organizationallevel.

If we compare the different cities of any country, then the data analysis will be at the city level.

If we compare the different countries, then the data analysis will be at the country level. etc.etc.

16


17/38

TIME HORIZON

One Shot or Cross-Sectional StudiesIf data are gathered just once, perhaps over a period of days or weeks or

months, in order to answer a research question. are called one-shot or cross-

sectional studies.

17

EXAMPLES1. Data were collected from stock brokers between April and June of last year

to study their concerns in a turbulent (beyond control) stock market. Data with

respect to this particular research had not been collected before, nor will they

be collected again from them for this research.

2. A drug company desirous of investing in research for a new obesity

(reduction) pill conducted a survey among obese people to see how many of

them would be interested in trying the new pill. This is a one-shot or cross-

sectional study to assess the likely demand for the new product.


18/38

Multishot or Longitudinal Studies

If the researcher might want to study people or phenomena atmore than one point in time in order to answer the researchquestion or when data on the dependent variable are gathered

at two or more points in time to answer the research question,the studies are called longitudinal studies.

For instance, the researcher might want to study employeesbehavior before and after a change in the top management, so as

to know what effects the change accomplished. Here, becausedata are gathered at two different points in time, the study is notcross-sectional or of the one-shot kind, but is carriedlongitudinally across a period of time.

18

EXAMPLEOne could study the sales volume of a product before and after an advertisement, and

provided other environmental changes have not impacted on the results, one could

attribute the increase in the sales volume, if any, to the advertisement. If there is no

increase in sales, one could conclude that either the advertisement is ineffective or it

will take a longer time to take effect.


19/38

EXERCISE

In the following scenarios indicate how the researcher should proceed

in each case, that is, determine the following, give reason also:

1. The purpose of study,

2. The type of investigation,

3. The extent of researcher interference,

4. The study settings,5. The time horizon for the study,

6. The unit of analysis.

Scenario A

Ms. Joyce Lynn, the owner of small business (a womens dress

boutique), has invited a consultant to tell her how business is differentfrom similar small businesses within a 60-mile radius with respect touse of the most modern computer technology, sales volume, profitmargin, and staff training.

19


20/38

EXERCISE

Scenario BMr. pall Hodge, the owner of severalrestaurants on the East Coast, is concernedabout the wide differences in their profit

margins. He would like to try some incentiveplans for increasing the efficiency levels ofthose restaurants that lag behind. But beforehe actually does this, he would like to beassured that the idea would work. He asks aresearcher to help him on this issue.

20


21/38

21

EXPERIMENTALDESIGN


22/38

CONTROLLING OF CONTAMINATING FACTORS

When we postulate cause-and-effectrelationships between two variables X andY, it is possible that some other factor, saysA, might also influence the dependentvariable Y. In such a case, it will not bepossible to determine the extent to which Yoccurred only because of X, since we do

not know how much of the total variationof Y was caused by the presence of theother factor A.

22


23/38

EXAMPLE OF CONTROL

For instance, a Human Resource Development manager mightarrange for special training to a set of newly recruited secretariesin creating web pages, However, some of the new secretariesmight function more effectively than others, mainly or partlybecause they have had previous intermittent experience with

the web. In this case, the manager cannot prove that the specialtraining alone caused greater effectiveness, since the previousintermittent experience of some secretaries with the web is acontaminating factor. If the true effect of the training on learningis to be assessed, then the learners previous experience has tobe controlled. This might be done by not including in the

experiment those who already have had some experience withthe web. This is what we mean when we say we have to controlthe contaminating factors.

23


24/38

CONTROLLING THE CONTAMINATING EXOGENOUS OR

NUISANCE VARIABLES

Matching Groups

One way of controlling the contaminating ornuisance variablesis to match the various groups by picking the confoundingcharacteristics and deliberately spreading them across groups.

24

Randomization

In randomization, the process by which individuals are drawn(i.e., everybody has a known and equal chance of being drawn)

and their assignment to any particular group (each individualcould be assigned to any one of the groups set up ) are bothrandom.


25/38

EXTERNAL VALIDITY

25

To what extent would the result found in the lab

setting be transferable or generalizable to the actualorganizational or field settings? In other words, if we

do find a cause-and-effect relationship after

conducting a lab experiment, can we then

confidently say that the same cause-and-effect

relationship will also hold true in the organizational

setting?

Internal validity refers to the confidence we place in the cause-and-effect relationship with in the lab settings.

INTERNAL VALIDITY


26/38

FACTORS AFFECTING INTERNAL VALIDITY

26

Sales promotion Sales

Dairy

farmers advertisement

Independent variable Dependent variable

Uncontrolled variable

Time: t1 t2 t3

History Ef fects

Certain events or factors that would have an impact on the independent variable-dependent variable relationship might unexpectedly occur while the experimentis in progress, and this history of events would confound the cause-and-effectrelationship between the two variables, thus affecting the internal validity.


27/38


28/38

28

Testing Effects

Frequently, to test the effect of a treatment, subjects are givenwhat is called a pretest(say, a short questionnaire eliciting their

feelings and attitudes). That is, first a measure of the dependentvariable is taken (the pretest), then the treatment given, and afterthat a second test, called the posttest, administered. Thedifference between the posttest and the pretest scores is thenattributed to the treatment. However, the very fact that

respondents were exposed to the pretest might influence theirresponses on the posttest, which would adversely impact oninternal validity.

I nstrumentations Effects

Instrumentation effects are yet another source of threat tointernal validity. These might arise because of a change in themeasuring instrument between pretest, and posttest, and not

because of the treatments differential impact at the end.


29/38

29

Selection Bias Effects

The threat to internal validity could also

come from improper or unmatchedselection of subjects for the experimentaland control groups.

Mortality

Another confounding factor on the cause-and-effect relationship is the mortality or

attrition of the members in theexperimental or control group or both, asthe experiment progresses.


30/38

30

Statistical Regression

The effect of statistical regression are brought aboutwhen the members chosen for the experimental grouphave extreme scores on the dependent variable to beginwith. We know from the law of probability that thosewith very low scores on a variable have a greater

probability of showing improvement and scoring closerto the mean on the posttest after being exposed to thetreatment. This phenomenon of low scores tending tocloser to the mean is known as regression towards themean (statistical regression). Likewise, those with

very high abilities would also have a greater tendencyto regress towards the mean-they will score lower onthe posttest than on the pretest.


31/38

TYPES OF EXPERIMENTAL DESIGNS

Group Pretest score Treatment Posttest Score

Experimental group O1 X O2

Treatment effect = (O2-O1)

31

Pretest and Posttest Experimental Group DesignAn experimental group (without a control group) may be given a pretestexposed to a treatment, and then given a posttest to measure the effects of thetreatment. Where Orefers to some process of observation or measurement, Xrepresents the exposure of a group to an experimental treatment, and the Xand Os in the row are applied to the same specific group. Here, the effects of

the treatment can be obtained by measuring the difference between theposttest and the pretest (O2-O1). Note, however, that testing andinstrumentation effects might contaminate the internal validity. If theexperiment is extended over a period of time, history and maturation effectsmay also confound the results.

P tt t O l ith E i t l d C t l G


32/38

Group Treatment Outcome

Experimental group

Control group

X O1

O2

Treatment effect = (O2-O1)

32

Posttests Only with Exper imental and Control Groups

Some experimental designs are set up with an experimental and a controlgroup, the former alone being exposed to a treatment and not the latter. Theeffects of the treatment are studied by assessing the difference in the outcomes-that is, the posttest scores of the experimental and control groups. Here is acase where the testing effects have been avoided because there is no pretest,only a posttest. however, to make sure that the two groups are matched for allthe possible contaminating nuisance (unwanted) variables. Otherwise, thetrue effects of the treatment cannot be determined by merely looking at thedifference in the posttest scores of the two groups. Randomization would take

care of this problem.There are at least two possible threats to validity in this design. If the twogroups are not matched or randomly assigned, selection biases couldcontaminate the results. Mortality (the drop out individuals from groups) canalso confound the results,


33/38

33

Pretest and Posttest Exper imental and Control Group Designs

Two groups-one experimental and the other control-are both exposed to thepretest and the posttest. The only difference between the two groups is that theformer is exposed to a treatment whereas the latter is not. Measuring the

difference between the differences in the post-and pretest scores of the two groupswould give the net effects of the treatment. Both groups have been exposed toboth the pre-and posttests, and both groups have been randomized; thus we couldexpect that the history maturation, testing, and instrumentation effects have beencontrolled. This is so due to the fact that whatever happened with theexperimental group (e.g., maturation, history, testing, and instrumentation) also

happened with the control group, and in measuring the net effects (the differencein the differences between the pre-and posttest scores) we have controlled thesecontaminating factors. Through the process of randomization, we have alsocontrolled the effects of selection biases and statistical regression. Mortalitycould, however, pose a problem in this design. In experiments that take severalweeks, as in the case of assessing the impact of training on skills development, ormeasuring the impact of technology advancement on effectiveness, some of the

subjects in the experimental group may drop out before the end of theexperiment. It is possible that those who drop out are in some way different fromthose who stay on until the end and take the posttest. If so, mortality could offer aplausible (apparently valid) rival explanation for the difference between O2 andO1.


34/38

Group Pretest Treatment Posttest

Experimental group

Control groupO1

O3

X O2

O4

Treatment effect = [(O2-O1) - (O4-O3)]

34

Pretest and posttest experimental and control group


35/38

35

To gain more confidence in internal validity in experimentaldesign, it is advisable to set up two experimental groups andtwo control groups for the experiment. One experimentalgroup and one control group can be given both the pretest

and the posttest. The other two groups will be given only theposttest. Here the effects of the treatment can be calculatedin several different ways. To the extent that we come up withalmost the same results in each of the different calculations,we can attribute the effects to the treatment. This increases

the internal validity of the results of the experimental design.This design, known as the Solomon four-group design, isperhaps the most comprehensive and the one with the leastnumber of problems with internal validity.

SOLOMON FOUR GROUP DESIGN


36/38

.

Group Pretest Treatment Posttest

1. Experimental

2. Control

3. Experimental

4. Control

O1

O3

X

X

O2

O4

O5

O6

Treatment effect (E) could be judged by:

E= (O2-O1)

E= (O2-O4)

E= (O5-O6)

E= (O5-O3)

E= [(O2-O1) - (O4-O3)]

If all Es are similar, the cause-and-effect relationship is highly valid.

36

SOLOMON FOUR GROUP DESIGN MODEL


37/38

37

Solomon Four-Group Design and Threats to Internal Validity

Let us examine how the threats to internal validity aretaken care of in the Solomon four-group design. It is

important to note that subjects have been randomlyselected and randomly assigned to groups. This removesthe statistical regression and selection biases. Group 2,the control group that was exposed to both the pre-and

posttest, helps us to see whether or not history,maturation, testing, instrumentation, regression, ormortality threaten internal validity. If scores O3 and O4(pre-and posttest scores of group 2) remain the same, then

it is established that neither history, nor maturation, nortesting, nor instrumentation, nor statistical regression, normortality has had an impact. In other words, these havehad no impact at all.


38/38

THANK YOU

FOR YOURCONCENTRATION

38

Date post:	14-Apr-2018
Category:	Documents
Upload:	zunaira-azhar
View:	222 times
Download:	0 times

Lecture Research Design

Documents