UNIT II Research Design

8/7/2019 UNIT II Research Design

1/31

RESEARCH DESIGN

Meaning of Research Design:

Research is any form of systematic and arranged investigation to organize factsor gather data, and is often related to a problem that has to be solved. A plan forcollecting and utilizing data so that desired information can be obtained withsufficient precision or so that a hypothesis can be tested properly.

Components of Research Design:

a. The Sampling Design:

It deals with the method of selecting sampling items to be observed for

the given study.

b. The Observational Design:

It relates to the conditions under which the observations are to be

made.

c. The Statistical Design:

It concerns with the question of how many items are to be observed

and how the information gathered are to be analysed.

d. The Operational Design:

It deals with the techniques by which the procedures specified in the

sampling, statistical and observational designs can be carried out.

Important Concepts relating to Research Design:

1. Dependent & Independent variables:

A concept which can take qualitative & Quantitative values is called A

variable. As such the concept like Wight Height and Income level are

all examples of Quantitative Variables. The Qualitative Variables arethe concepts describing a persons / an objects Nature, Quality and

Characteristics like: He is a Democratic type of leader. X is rude man.

If one variable depends upon or the consequence of the other variable,

it is termed as Dependent variable. Eg: the Height. (because Height


2/31

depends upon the Age). The variable that is predecessor to the

Dependent variable is called as Independent variable.

2. Extraneous variable:

Independent Variables that are not related to the purpose of the study

but may affect the Dependent variable are termed as Extraneous

Variable.

3. Research Hypothesis:

The research hypothesis is a predictive Statement that relates an

Independent variable to a Dependent variable.

4. Experimental & Control group:

In an Experimental research, when a Group is exposed to

Usual/Normal conditions, it is termed as a Control Group, whereas

when the Group is exposed to some Novel/Special conditions, it is

termed an Experimental Group.

5. Treatments:

The different conditions under which Experimental and Control Groups

are put (or) exposed to are usually referred to as treatment.


3/31

6. Experiment:

The process of examing the truth of a Statistical Hypothesis relating to

a study is known as Experiment.

Different Research Designs:

1. Exploratory Research Design:

2. Descriptive and Diagnostic Research Design:

3. Experimental research Design:

Basic Principles of Experimental Designs:

a. The Principle of Replication:

According to replication, the experiment should be repeated more than

once in order to get statistical accuracy of the Experiment.

b. The principle of Randomisation:

The principle indicates that we should design or plan the experiment in

such a way that the variations caused by Extraneous factors can all be

combined under the general heading Chance.

c. The Principle of Local Control:

We first divide the filed into several homogeneous parts, known as

blocks and then each such block is divided into parts equal to the

number of Treatments. Then the treatments are randomly assigned to

these parts.


4/31

Experimental Design:

Introduction:

Experimental design enables a researcher to alter systematically the variablesinvolved in the study. The experimental design involves intervention by theresearcher. The researcher intervenes by way of manipulating the variables in asetting and observes the effect on the subjects studied. Under experimentaldesign the independent variables are manipulated and the effects of the same onthe dependent variables are observed.

This units deals with the discussion on activities involved in conducting anexperiment, the factors affecting the validity in experimentation and the varioustypes of experimental designs. Measurement of variables is necessary for testing

the hypotheses.

The nominal, ordinal, interval and ratio scales are dealt in detail. The processinvolved in selection and construction of measurement scales are discussed indetail.

Activities involved in conducting an experiment:

The following activities are involved in the experimental research:

Selecting relevant variables

The researcher in the course of the conduct of the study develops hypotheses tomeet the objectives of the research. The hypothesis describes the relationshipbetween two or more variables. The researcher should select the variables thatbest represent the concepts to be tested, determine the number of variables tobe tested and select or design appropriate measures for them. The number ofvariables selected in a research study is subject to the budget allocated, the timeframe available, number of subjects being tested and the like.

Specifying the levels of treatment

The treatment levels of the independent variable are the distinctions, theresearcher makes between different aspects of the treatment conditions. For eg.,if attitude is hypothesized to have influence on the purchase behaviour, theattitude may be grouped into three levels viz., positive, negative and neutral.

Controlling the experimental environment


5/31

The control may be exercised on the experimenter/researcher, subject or theenvironment. The environmental control is concerned with holding the physicalenvironment in which the experiment is conducted as constant. The subjects maynot know that the experiment is being conducted. This situation is mentioned asblinding the subject. When the experimenter/researcher is also unaware of the

experiment, then it is called as double - blinded. The control refers to avoidingthe effect of extraneous variables on the research study conducted.

Choosing the experimental design

Choosing an apt experimental design improves the probability that the observedchange in the dependent variables are caused by the manipulation of theindependent variable only and not by another variable. It strengthens thegeneralization of the result.

Selecting and assigning the subjects

The question of selecting and assigning the subjects do not arise in case wherethe entire population is considered for the study. However, mostly the researcherwill depend on sample to conduct the study. In order to validate and generalizethe findings of the research study, the samples selected should be representativeof the population. The sample may be selected on the basis of random selectionor systematic sampling. Random assignment of subjects to groups should befollowed. When it is not possible to randomly assign the groups then matchingmay be used. Matching is based on non-probability quota sampling approach.The object of matching is to ensure that each experimental and control subjectmatches on every characteristics used in research.

Pilot-testing, revising and testing

Pilot - testing reveals error in the design and improper control of extraneous orenvironmental conditions. Pretesting the instruments enables refinement of thesame before the final test. It enables to revise scripts, take stock of controlproblems with laboratory conditions and scan the environment for factors thatmight confound the results.

Analyzing the Data

Pretesting and proper planning enable to have an order and structure in theexperimental data collected. The data are more conveniently arranged as a resultof the levels of treatment condition, pretest and post-test and the groupstructures. This enables to apply statistical technique in a simplified manner.

Validity in Experimentation:


6/31

The findings of Experimental design are judged by measuring the internal andexternal validity. The Validity is the extent to which a measure accomplishes itsclaims. Two groups of validity exists; internal and external. Internal validity isconcerned with identifying whether the conclusions drawn from a demonstratedexperimental relationship truly imply cause. External validity is concerned with

the generalization of observed casual relationship across persons, settings andtimes.

Factors affecting internal validity:

Internal validity refers to the confidence that one can place in the cause-effectrelationship. In other words, it addresses the question, "To what extent does theresearch design permit us to say that the independent variable "A" causes thechange in the dependent variable "B" ?". Factors affecting internal validitycauses confusion as to whether the observational differences are due toexperimental treatment or extraneous factors. An experiment has high internal

validity if the researcher has the confidence that the experimental treatment hasbeen the source of change in the dependent variable. The factors listed belowaffects the internal validity:

HistoryMaturationTestingInstrumentationSelectionStatistical regressionExperimental mortality

Diffusion or imitation of treatmentCompensatory equalizationCompensatory rivalryResentful demoralization of the disadvantagedLocal history

1. History

In the experimental designs a control measurement (O1) of dependent variable istaken before introducing the manipulation (X). After the manipulation an aftermeasurement (O2) of the dependent variable is taken. Then the difference

between O1 and O2 is attributed to the manipulation. However, some events mayoccur during the course of the experimental study which will affect therelationship between the variables under the study.

2. Maturation


7/31

The subjects considered for experimentation might change with the passage oftime and may not be due to the occurrence of any specific event. This happensparticularly when the study covers a long period of time.

3. Testing

The process of taking a test can affect the scores of further tests. The first testwould have created some awareness and learning experience which influencesthe results of the subsequent tests.

4. Instrumentation

The threat to validity may arise due to the observer or the instrumentation. Usingdifferent observers or interviewers affects the validity of the study. If the sameobserver is used for a longer period of time, it may affect the validity due toobservers experience, boredom, fatigue and anticipation of results. Difference in

the questions for each measurement affects the validity

5. Selection

Differential selection of subjects for experimental and control groups affects thevalidity. Validity considerations require the groups to be equivalent in everyaspect. The problem can be overcome by randomly assigning the subjects toexperimental and control groups. In addition, matching can be done. Matching isa control procedure to ensure that experimental and control groups are equatedon one or more variables before the experiment. Matching the members of thegroups on key factors also enhances the equivalence of the groups.

6. Statistical Regression

This factor operates especially when members chosen for the experimentalgroup have extreme scores on the dependent variable. For eg., If a managerwants to test if he can increase the salesmanship qualities of the sales personnelthrough training program, he should not choose those with extremely low orextremely high abilities for the experiment. This is because, those with very lowscore i.e., those with low current sale abilities have a greater probability ofshowing improvement and scoring closer to the mean test after being exposed tothe treatment. This phenomenon of low scorers tending to score closer to the

mean is known as "regressing towards the mean". Likewise, those with very highabilities would also have a greater tendency to regress towards the mean theywill score lower on the post-test than on the pretest. Thus, those who are ateither end of the continuum with respect to the variable would not "truly" reflectthe cause-and-effect relationship. This phenomenon of statistical regression is athreat to internal validity.


8/31

7. Experimental mortality

This factor arises due to the changes in the composition of study groups duringthe test. There may be drop outs in the study group leading to the changes in themembership of the group. This problem does not arise for the control group as

they are not affected by the testing situation and they are less likely to withdraw.All the above threat factors can be controlled to a certain extent by randomassignment. However, the following factors affecting internal validity cannot becontrolled by randomization. Both the control group and the experimental groupare affected by the first three factors.

8. Diffusion or imitation of treatment

The interaction between the experimental and the control group may lead thecontrol group to learn about the experiments eliminating the difference betweenthe groups.

9. Compensatory equalization

If the experimentation treatment leads to a desirable and beneficial outcome,then it may lead to an administrative reluctance to deprive the control members.Compensatory action directed at the control groups may confound theexperiment.

10. Compensatory rivalry

Compensatory rivalry arises when the member of the control group know that

they are in the control group. This will generate competitive pressures causingthem to try harder which will affect their normal behaviour.

11. Resentful demoralization of the disadvantaged

When the treatment is desirable and the experiment is obtrusive, control groupmembers may become resentful of their deprivation and lower their cooperationand output.

12. Local history

When all experimental persons are assigned to one group or session and allcontrol people to another, there is a chance for some idiosyncratic event toconfound results. This problem can be handled by administering treatments toindividuals or small groups that are randomly assigned to the experimental orcontrol sessions.


9/31

Factors affecting External Validity:

External validity is concerned with the interaction of the experimental treatment

with other factors and the resulting impact on the ability to generalize the findingsacross times, settings or persons. External validity is high when the results of anexperiment are applicable to a larger population. The following are the threats toexternal validity:

1. The reactivity of testing on the experimental treatment

In the case of conducting pretest in an experimental design the subjects aresensitized and they react to the experimental stimulus in a different manner. Thebefore-measurement effect can be particularly significant in case of experimentswhere the independent variable involved in the study is concerned with the

change in the attitude.

2. Interaction of selection and the experimental treatment

This threat is concerned with the process by which the test subjects are selectedfor the experiment. The population from which the subjects are selected may notbe the same as the population to the result are extended to. It limits thegeneralizations of the findings.

3. Other reactive factors

The experimental settings may have a biasing effect on the subjects response tothe experimental treatment. An artificial setting will produce results that are notrepresentative of the larger population. If subjects know that they areparticipating in an experiment, they may not behave in a normal way; this affectsthe validity of the experimental treatment.

Experimental research designs:

Informal Experimental Research Design:

Before and After without Control Group

After only with Control GroupBefore and After with Control Group

Many experimental designs are available and they widely vary on the basis oftheir power to control contamination of the relationship between independent anddependent variables. On the basis of the characteristics of control theexperimental design can be grouped under the following three heads:


10/31

1. Pre-experiments2. True experiments/Lab Experiment3. Field experiments

A set of commonly used symbols are:

X = The exposure of an independent variable to a group of test subjects for whichthe effects are to be determined

O = The process of observation or measurement of the dependent variable(effect outcome ) on the test subjects

R = The random assignment of test subjects to separate treatment groups

Pre-experimental designs

Pre- experimental design represents the crudest form of experimentation and isundertaken only when nothing stronger is possible. The designs arecharacterized by the absence of randomization of test subjects. The pre-experimental designs are weak in their scientific measurement power becausethey fail to control adequately the various threats to the internal validity. As aresult they fail to meet the internal validity criteria. Three pre-experimentaldesigns are detailed below:

1. One-Shot Case Study

1. One-Shot Case Study:

A single group of test subjects is exposed to the independent variable treatmentX, and then a single measurement on the dependent variable is taken (01).One-Shot case study does not use pretest and control group. As a result this design isinadequate for establishing causality. For e.g., a study on the employeeeducation campaign about the automation of the office activities without a priormeasurement of employee knowledge. Result would reveal only how much theemployees know after the education campaign, but there is no way to judge theeffectiveness of the campaign. This may be represented as shown below:


11/31

True experiments/ Lab experiments

The lab experiments are conducted in order to ascertain the cause and effectrelationship between independent and dependent variables. In order tounderstand the cause-effect relationship all other variables which might

contaminate the relationship should be controlled so that the actual cause effectsof the investigated independent variable on the dependent variable can bedetermined. Thus, the experiments performed in an artificial or contrivedenvironment is known as lab experiments. In the lab experiments the researcherhas complete control over all aspects. The researcher has a control over theexperiment, who, what, when, where and how. The researcher can assignsubjects to conditions randomly. Random assignment is an unbiased assignmentprocess that gives each subject an equal and independent chance of beingplaced in every condition. Random assignment is preferable because it allowsone to conclude that any other variable could be confounded with theindependent variable only by chance. Control over what, where, when and how

of the experiment means that the experimenter has complete control over theway the experiment is to be conducted.

Terms used:

The following are the meaning of the terms normally used in experimentaldesign:

Factors: The independent variables of an experiment are often called thefactors of the experiment. Active factors are those the experimenter canmanipulate by causing a subject to receive one level or another. Blocking

factor is one where the experimenter can only identify and classify the subjecton an existing level.Level: A level is a particular value of an independent variable.Condition: The term condition is used to discuss the independent variables. Itrefers to a particular way in which subjects are treated.Treatment: this is another word used for condition. It also refers to thestatistical test of the effect of various conditions of the experiment.Test unit: the experimental subjects are referred as test unit. The test unitmay be people, organizations, machine type, materials and other entities.

The basic principles of experimental research design are:

1. The existence of a control group or a control condition and2. The random allocation of the subjects to groups

The internal validity is higher in case of lab experiments. Internal validity asalready explained increases the degree of confidence in the casual effects. In labexperiments the cause-and effect relationship are substantiated and hence theinternal validity is higher. However the external validity i.e., the extent of


12/31

generalizations of study is lesser as the research is executed in a contrivedenvironment and the real world situation may not be same.

Some of the true experimental designs are discussed below:

1. Pre-test and post-test Control Group Design

Test subjects are assigned to the groups by a random procedure ( R ) to eitherthe experimental or control group. Each group receives a pretreatment measureof the dependent variable. Then the independent treatment is exposed to theexperimental group, after which both groups receive a post treatment measure ofthe dependent variable. The concept is represented in the following manner:

The internal validity problems discussed earlier are addressed to a greater extentin this design. Local history may occur in one group and not in the other.Maturation, testing and regression are handled well as the same would be

equally felt in experimental and control groups. Selection is equally dealt byrandom assignment. Mortality can be a problem if there are different drop outrates in the study groups. The external validity of the design is howeverquestionable.

2. Post- Test Only Control Group Design

Test subjects are randomly assigned to either the experimental or control group.The experimental group is then exposed to the independent treatment, afterwhich both groups receive a post treatment measure of the dependent variable.In this design, the pretest measurements are omitted. The design is

The experimental effect is measured by the difference between O1 and O2. Thedesign is more simple and attractive. Internal validity threats from the history,maturation, selection and statistical regression are adequately controlled byrandom assignment. Since the subjects are measured only once, the threats oftesting and implementation are handled. The different mortality rates betweenexperimental and control groups continue to be a problem. The design reducesthe external validity problem of testing interaction effect.

3. Extensions of True Experimental Design

The researcher normally uses an operational extension to the basic design.These extensions differ from classical design in terms of

The number of different experimental stimuli that are consideredsimultaneously by the experimenter andThe extent to which assignment procedures are used to increase precision.


13/31

a. Completely Randomized Design

It involves two principles viz., the principle of replication and the principle ofrandomization. The essential characteristic feature of the design is that thesubjects are randomly assigned to experimental treatments. This design is

generally used to when experimental areas happen to be homogeneous. In caseof this design all the variations due to uncontrolled extraneous factors areincluded under the heading of chance variation. Two forms of such design are:

1. Two-group simple randomized design: In this design first all the populationis defined and then from the population a sample is selected randomly. Theselected sample is then randomly assigned to experimental and control groups.Thus, the design yields two groups as representatives of the population viz theexperimental and control group. The two groups are given different treatments ofthe independent variable. This design of experiment is quite common in researchstudies concerning behavioural sciences.

a. Randomized block design

In the randomized block design the subjects are first divided into groups knownas blocks such that within each group the subjects are relatively homogenous inrespect to some selected variable. The number of subjects in a given block wouldbe equal to the number of treatments and one subject in each block would berandomly assigned to each treatment. This design is used when there is a singlemajor extraneous variable. Random assignment is the basic way to produceequivalence among treatment group. Blocking is done to learn whethertreatments bring different results among various groups of subjects. In this

design two effects could be studied

1. The main effect is the average direct influence that a particular treatmenthas independent of other factors.

2. The interaction effect is the influence of one factor on the effect of another.

The precision of this experimental design depends on how successfully thedesign minimizes the variance within blocks and maximizes the variancebetween blocks. If the response patterns are about the same in each block, thereis little value to the more complex design and blocking in case may be counterproductive. The randomized block design is analysed by the two-way ANOVA

technique.

b. Latin Square Design

The randomized block design is used to minimize the effects of one extraneousvariable whereas the Latin square design is used when two blocking factors areto be controlled. Each treatment occur an equal number of times in any oneordinal position in each row. Treatments are randomly assigned so that each


14/31

treatment appears only once for each factor. Due to this aspect, the Latin squareshould have the same number of rows, columns and treatments. For example, inorder to find the effect of offering discount at 5%, 10%, 15%, two blocking factorscan be identified viz., customer income and size of the store. Three levels can beidentified on the basis of the blocking factor Customer income viz., high, medium

and low. On the basis of store size, three levels can be identified viz., large,medium and small. On the basis of both the blocking factors customer incomeand size of the store, nine groups can be identified. for each of the group onetreatment will be given and in the row or column it will not be repeated. This isillustrated below:

Treatments are assigned based on random number tables. From the above, theeffects of price reduction can be ascertained. The major limitation of Latin squareis that it is assumed that there are no interaction between treatments and theblocking factors.

d. Factorial design

In case of factorial design a researcher can deal with more than one factorsimultaneously. This design is especially important in several economic andsocial phenomena where usually, large number of factors affect a particularproblem. Factorial designs can be of two types:

1. Simple factorial designs2. Complex factorial designs

(1). Simple Factorial designs: When the effect of varying two factors on the

dependent variables is dealt, the design is called simple factorial design. Thisdesign is also known as two-factor-factorial design. A simple factorial design maybe either a 2 X 2 , 3 X 4 or 5 X 3 or the like type of design. A 2 X 2 simplefactorial design can be depicted as below:


15/31

In this design extraneous variable to be controlled by homogeneity is called asthe control variable and the independent variable which is manipulated is calledexperimental variable.

(2). Complex factorial design: A design which considers three or more

independent variables simultaneously is called a complex factorial design. This isalso known as multi-factor factorial design.

MEASUREMENT

In normal parlance, measurement refers to an attempt to fix quantitatively theform or other features of a physical object. In research, measurement refers toassigning numbers to empirical events in compliance with a set of rules. Thisdefinition brings out the three steps involved in the process of measurement:

1. Selecting the observable empirical events.

2. Developing a set of mapping rules i.e. a scheme for assigning numbers orsymbols to represent aspects of the event being measured.

3. Applying the mapping rules to each observation of that event.

The goal of measurement is to provide the highest quality, lowest error data forthe purpose of testing the hypotheses identified and other related analysis andinterpretations. Variables dealt in research studies can be classified as objectsand properties. Objects include the things of ordinary experience, such as thelaptop, chair and car. It also includes things which are not concrete such asattitude, peer group pressures, perception etc. Properties are the characteristicsof the objects. It includes level of motivation, leadership skills etc. strictly

speaking researchers are not involved in measuring objects or properties butrather they measure the indicants of the properties or indicants of the propertiesof the objects.

Mapping rules

Measurement involves developing mapping rules and applying the same torecord the happenings. The assumptions regarding the mapping rules alsoaffects the choice of data. The mapping rules have the following fourcharacteristics:

1. Classification: numbers are used to group or sort responses. Order doesnot exist

2. Order: Numbers are arranged in some order in such a way that onenumber is greater than /smaller than / equal to another number.

3. Distance: differences between numbers are ordered. The differencebetween any pair of numbers is greater than, less than or equal to thedifference between any other pair of numbers


16/31

4. Origin: the number series has an unique origin indicated by the numberzero

Combination of these four characteristics gives raise to data types.

The scales

Based on the characteristics of the mapping rules i.e., classification, order,distance and origin, four classifications of measurement scales could be arrivedat: nominal, ordinal, interval and ratio scale. A detailed discussion follows:

1. Nominal scale

A nominal scale allows the researcher to assign subjects to certain categories orgroups. For e.g., the respondents can be grouped as male and female. The twogroups can be assigned numbers for the purpose of coding and further analysis

as 1 and 2. These numbers are simple and convenient labels and have nointrinsic values. It only assigns subjects into either of the two mutually exclusivecategories. In other words, nominal scale allows the researcher to collectinformation on a variable that naturally or by design can be grouped into two ormore categories that are mutually exclusive and are collectively exhaustive.

The nominal scale provides only the basic, categorical, gross information.Counting of members in each group and calculation of frequency or percentageis possible when nominal scale is employed. The researcher is restricted to theuse of mode as the measure of central tendency. One can conclude whichcategory has more members. Chi-square test can be used to measure the

statistical significance and for measures of association, phi, lambda or othermeasures may also be appropriate.

Nominal scales are weak but they are still useful to classify the data. It isvaluable in exploratory work where the objective is to uncover relationshipsrather that to secure precise measurements. Nominal data type is also widelyused in survey and ex post facto research when data is classified by majorsubgroups of the population.

2. Ordinal scale

Ordinal scale indicates the order. It includes the characteristics of nominal scalealso. Thus an ordinal scale not only categorizes the variables but also rank-orders categories in some meaningful way. The use of ordinal scale implies astatement of greater than or less than or equal to without stating how muchgreater or less. Other descriptors may also be used viz., superior to, happierthan, poorer than, above. It is also possible to rank more than one property ata time. For e.g., researcher can ask the respondent to rank various air lines onthe basis of certain properties.


17/31

In ordinal scaling the differences in the ranking of objects, persons or eventsinvestigated are clearly known. However, the ordinal data does not give anyindication of the magnitude of the differences among the ranks

3. Interval scale

Interval data has the power of the nominal and ordinal data and in addition itincorporates the concept of equality of interval. The interval scale allows tomeasure the distance between any two points on the scale. It not only enables togroup the individuals according to certain categories and taps the order of thegroups; it also measures the magnitude of differences in the preferences amongthe individuals. The interval scale is more powerful than the nominal and theordinal scales. The measure of central tendency the arithmetic mean, isapplicable. Its measures of dispersion are the range, the standard deviation andthe variance.

4. Ratio scale

Ratio data has the power of the nominal, ordinal and interval scale in addition italso has the provision for absolute zero or origin. It covers the disadvantage ofthe arbitrary origin point of the interval scale, i.e., it has an absoultue zero point.The ratio scale not only measures the magnitude of the differences betweenpoints on scale but also the proportion in the differences. Multiplication or divisionwould preserve the ratios. It is the most powerful of the four scales because ithas a unique zero origin and subsumes all the properties of the other threescales.

The measure of central tendency of the ratio scale could be either the arithmeticor the geometric mean and the measure of dispersion could be either thestandard deviation or variance or the coefficient of variation. Some examples ofratio scales are those pertaining to actual age, income and work experience inorganizations.

Sources of measurement differences

Normally, any variation of scores among the respondents would reflect truedifferences in their opinions about the object/issue. However, four major sourcesmay contaminate the results: the respondent, the situation, the measurer and the

data collection instrument

.1. The Respondent

Opinion differences that affect measurement come from relatively stablecharacteristics of the respondent. The skilled researcher will anticipate many ofthese dimensions, adjusting the design to eliminate, neutralize or otherwise dealwith them. However, the respondents may still suffer from temporary factors like


18/31

fatigue, boredom, anxiety or other distractions which may limit their ability torespond accurately and fully. Likewise variations in mood due to hunger,impatience etc may also have an impact.

2. The Situational factors

Situational factors include any condition that places a strain on the interview ormeasurement session which may have a serious effect on the interviewer-respondent rapport. If another person is present during the interview or if therespondents believe that anonymity is not ensured then they may be reluctant toexpress their true feelings.

3. The Measurer

The interviewer can distort responses by rewording, paraphrasing or reorderingquestions. The tone used the body language, smiles, nods and so forth may

encourage or discourage replies. Likewise in data analysis stage, incorrectcoding, careless tabulation and faulty statistical calculation may introduce errors.

4. The Instrument

A defective instrument can cause distortion in two major ways. First, the use ofcomplex words and syntax beyond respondents comprehension may causeconfusion and ambiguity. Leading questions, ambiguous meanings, mechanicaldefects and multiple questions suggest the range of problems.

The characteristics of sound measurement:

The instrument developed to measure the concept should be an accurateindicator of the aspects that are being measured. The scale developed cannot beimperfect and prone to errors. The use of better instruments will ensure accuracyin results and will enhance the scientific quality of research. It should also beeasy and efficient to use. The goodness of the measures developed should beassessed on the basis of three major criteria viz., validity, reliability and

practicality.

I. Validity:

Validity refers to the extent to which a test measures what we actually wish tomeasure. It refers to the extent to which differences found with a measuring tool


19/31

reflect true differences among respondents being tested. Validity can beclassified into three major forms viz., content validity, criterion-related validity andconstruct validity.

a. Content Validity

The content validity refers to the extent to which a measuring instrument providesadequate coverage of the investigative questions guiding the study. Contentvalidity is good if the instrument contains a representative sample of the universeof subject matter of interest. Determination of content validity is judgmental andcan be approached in several ways. Generally the content validity is treated to behigher , if the scale items used represents to a greater extent the domain oruniverse of the concept being measured. The researcher may determine thecontent validity through a careful definition of the topic of concern, the item to bescaled and the scales to be used. Another way is to use a panel of persons to

judge whether the instrument meets the standards.

Face validity is considered as a basic and very minimum index of content validity.It indicates that on the face of it the, items look as if they measure the intendedconcept.

b. Criterion-Related Validity

Criterion related validity reflects the success of measures used for prediction orestimation. Predictive validity refers to the extent to which an outcome could bepredicted and concurrent validity refers to the extent to which estimate of currentbehaviour or condition could be made. The researcher must ensure that the

validity criterion used is itself valid. This can be judged in terms of four qualitiesviz., relevance, freedom from bias, reliability and availability.

c. Construct validity

This is the most complex and abstract feature. Construct validity testifies that theresults obtained from the use of measure fits the theories around which the testis designed. In other words a measure has construct validity to the degree that itconforms to predicted correlations of other theoretical propositions. Theresearcher may wish to measure or infer the presence of abstract characteristicsfor which no empirical validation seems possible. Attitude, aptitude and

personality scales generally fall in this category. Although it is difficult, assuranceis still needed that the measurement has an acceptable degree of validity. This isassessed through convergent and discriminant validity. Convergent validity isestablished when the score obtained with two different instruments measuringthe same concept are highly correlated. Discriminant validity is established when,based on theory two variables are predicted to be uncorrelated and it is alsoempirically proved. The validity can be proved through the use of correlationalanalysis, factor analysis etc.


20/31

II. Reliability:

Reliability refers to consistency i.e. a measure is reliable to the degree that itsupplies consistent results. Reliability is concerned with estimates of the degreeto which measurement is free from random or unstable error. Reliable

instruments can be used with confidence that transient and situational factors arenot interfering. Reliable instruments are robust and they work well at differenttimes under different conditions. The reliability of an instrument is measured onthe basis of the stability, equivalence and internal consistency.

a. Stability

Stability is securing consistent results with repeated measurements of the sameperson with the same instrument. An observation is said to be stable if it givesthe same reading on a particular person when repeated one or more times.Stability measurement in survey situations is more difficult then in observational

studies. Observation can be done repeatedly but the resurvey can be conductedonly once. Two tests of stability are test-retest reliability and parallel-formreliability.

III Practicality

The operational requirements of the project require it to be practical. Practicalityis defined as economy, convenience and interpretability. Economy is concernedwith minimizing the cost concerned with conducting the research project. Themethod of data collection, length of the instrument etc will have an implication onthe research budget. Convenience refers to ease in administering the

questionnaire. This can be achieved by giving clear and complete instructionsand by paying proper attention to design and layout. The interpretability issuearises in case when the persons other than the test designers must interpret theresults. To enable interpretation, the designer of the data collection instrumentshould provide enough information regarding the scoring keys, norms, guidelinesfor test use etc.

Measurement scales:

Scaling is a procedure for the assignment of numbers (or other symbols) to aproperty of objects in order to impart some of the characteristics of numbers to

the properties in question. The numbers are assigned to indicants of theproperties of objects. In case of measuring the attitude of respondents towards anew product introduced in the market numbers may be assigned. 1 may beassigned to positive attitude, 2 to neutral and 3 to negative attitude.

Measurement can be performed using standardized scales or through customdesigned scales. Standardized scales may be opted in case of measuringconcrete objects. Developing customized scale is needed in the case where a


21/31

researcher wants to measure more abstract and complex constructs like thecustomer attitudes towards a new product introduced in the market. In this casestandardized scales may not exist. This situation warrants the development ofcustomized scales.

Selection of measurement scale

Selection or construction of a measurement scale requires decision in thefollowing six key areas:

1. Study objective: Researchers may have two general study objective viz.,to measure the characteristics of the respondents and to use respondentsas judges of the objects or indicants presented to them.

2. Response form: Three types of measuring scales viz., rating, ranking andcategorization can be used. Rating scale is used when respondents scorean object or indicant without making a direct comparison with another

object or attitude. Ranking scales enable to make comparison among twoor more indicants or objects. Categorization enables to put the subjectsinvolved in groups or categories

3. Degree of preference: Measurement scales may involve preferencemeasurement or non preference evaluation. In case of preferencemeasurement respondents are asked to choose the object preferred. Incase of non preference evaluation the respondents are asked to make

judgments without any personal preference towards objects or solutions.4. Data properties: The data properties should also be viewed in case of

decision regarding measurement scales. The data can be classified asnominal, ordinal, interval and ratio. The statistical application depends on

the assumptions underlying each data type.5. Number of Dimensions: Measurement scales can be unidemensional ormultidimensional. In case of unidimensional scale only one attribute of therespondent is measured. Multidimensional scaling recognizes objects asconsisting ofn dimensions.

6. Scale construction: Five construction approaches are available viz.,arbitrary, consensus, item analysis, cumulative and factoring. Theresearcher should take into consideration of both the type of measurementand the scales construction when selecting an appropriate scale.

METHODS OF SCALING:

The methods of assigning numbers or symbols to the attitudinal responses of therespondents towards objects, events or persons is an important aspect of theresearch. There are two main categories of attitudinal scales - the rating scalesand the ranking scale. Rating scales have several response categories and areused to elicit responses with regard to the object, event or person studies.Ranking scales are used to make comparison between or among objects,events, persons and elicit the preferred choices and ranking among them.


22/31

1. RATING SCALES:

Rating scales are used to judge properties of objects without reference to othersimilar objects. In rating scales, an object is judged in absolute terms againstcertain specified criteria. The scale can be used to elicit responses with regard to

an object, event or person studied.

The number of scale points may range from three to five or ten. Researchersbelieve that more points on a rating scale provide an opportunity for an accuratemeasurement of variance.

Some of the rating scales used often by researchers are explained below:

Dichotomous scale:

The dichotomous scale offers two mutually exclusive response choices. It

may be used to elicit a "Yes" or "No" answer, "agree" and "disagree" etc., Thisis useful to elicit responses for demographic questions or where dichotomousresponse is adequate. e.g., Do you have a credit card? Yes No.

Category scale:

The category scale uses multiple items to elicit a single response. Themultiple choice, single-response scale is appropriate when there are multipleoptions but only one answer is sought. Example: AGE, as shown below:

Less than 20 years- 21 to 40 years- 41 to 50 years- Above 50 years

Multiple choice, Multiple- response scale:


23/31

The multiple choice, multiple- response scale allows the respondent toselect one or several alternatives. E.g., in eliciting the response regarding thesource through which the information about a new product is obtained, arespondent may select all or more than one of the choices given below:

Source of information:

Advertisement

Sales personSales materialsShowroomsFriends/ relatives/ NeighboursOther sources

Likert scale:

The Likert scale is designed to examine how strongly the respondents agreeor disagree with statements relating to the attitude or object on a 5-point scale.The scores on the individual items are summed to produce a total score for therespondent and hence it is also called summated scales. A Likert scale usuallycontains two parts, the item part and the evaluative part. The item part usuallycontains statement about a product, event or attitude. The evaluative part is alist of response categories ranging from " strongly agree" to "stronglydisagree". The item and evaluative part are showbelow:

The responses over a number of items or statements tapping a particularconcept or variable are summated for every respondent. It is assumed that allthe statements measure some aspect of a single common factor. This is aninterval scale and the differences in the responses between any two points onthe scale remain the same.


24/31

Semantic Differential Scales:

The Semantic Differential Scales are used widely to describe the set of

beliefs a person holds. Several bipolar attributes are identified at the extremesof the scale and respondents are asked to indicate their attitudes on semanticspace toward a particular individual, object or event on each of the attributes.The semantic space may consist of five or seven-point rating scales boundedat each end by polar adjectives or phrases. There may be as many as 15 to 25semantic differential scales for each attitude or object. The procedure is alsoinsightful for comparing the images of competing brands, stores or services.The semantic differential also may be analyzed as a summated rating scale.Each of the scale is assigned a value from -3 to 3 or 1 to 7 and the scoresacross all adjective pairs are summed for each respondent. Individuals can becompared on the basis of the total scores.An example of semantic differential

scale is given below:

The semantic differential has several advantages. It produces interval data. It is

an efficient and easy way to elicit responses from a large sample. The attitudescan be measured both in terms of direction and intensity. The total set ofresponses provides a comprehensive picture of the meaning of an object. It is astandardized technique which can be easily repeated and at the same timeescapes many problems of response distortion.

Numerical scale:

The Numerical scale is similar to the semantic differential scale with thedifference that numbers on a 5 point or 7 point scale are provided with bipolaradjectives at both ends. This is also an interval scale. The scale provides bothan absolute measure of importance and a relative measure of the variousitems rated. The scales linearity, simplicity and production of ordinal or intervaldata makes it very popular. An example :


25/31

Itemized rating scale:

The itemized rating scale is a 5 point or 7 point scale with anchors providedfor each item and the respondent states the appropriate number on the side ofeach item or circles the relevant number against each item. The responses tothe items are then summated. This uses an interval scale . Example is shownbelow:

The itemized rating scale provides the flexibility to use as many points in thescale as considered necessary ranging from 4,5,7,9, etc., It is also possible touse different anchors. When a neutral point is provided, it is a balanced ratingscale. When a neutral point is missing it is an unbalanced rating scale. Theitemized rating scale is frequently used in business research as it adapts to thenumber of points desired to be used , as well as the nomenclature of the anchorscan be accommodated to suit the needs of the researcher.

Fixed or constant sum scale:

In Fixed or constant sum scale the respondents are asked to distribute agiven number of points across various items. It enables the researcher todiscover the proportions and is more in the nature of ordinal scale. A minimumof two categories and a maximum of ten can be presented to the respondents.Presenting too many stimuli will be a hindrance to the precision and thepatience of the respondents. A respondents ability to add is also taxed if toomuch of stimulus is provided. For example in selecting a particular brand ofcomputer a respondent may be asked to rate the following aspects:


26/31

Staple Scales:

Staple Scales are simplified version of semantic differential scales. It is usedwhen it becomes difficult to find bipolar adjectives that match the investigativequestions. It uses only one pole rather than two. Respondents are asked toindicate the object by selecting a numerical response category. The higher the

positive score the better the adjective describes the object. Similarly , the lessaccurate the description, the larger the negative number chosen. Ratings mayrange from + 3 to -3, or + 5 to -5 , very accurate to very inaccurate. It producesinterval data. It is easy to administer and construct as there is no need toprovide adjectives to assure bipolarity. For eg, the respondents may be askedto rate their job using staple scales as follows:

Graphic rating scale:

The graphic rating scale is simple and commonly used in practice. In thisscale various points are marked along the line to form a continuum. Therespondent indicates his rating by simply making a mark at the appropriatepoint on a line that runs from one extreme to the other. A brief description onthe scale points are given to act as a guide in locating the rating. The faces


27/31

scales depicting faces ranging from smiling to sad can be used on a ratingscale to obtain responses regarding peoples feelings with respect to someaspect. A major limitation of this scale is that the respondent may selectalmost any position on the line which will pose difficulty in analysis.

Consensus scale:

The Consensus scale as the name suggests is developed by consensus bya panel of judges. The judges select certain items which enable to measure aconcept. The items are selected based on pertinence or relevance to theconcept. The items are also tested for validity and reliability. For e.g.Thurstone Equal Appearing Interval Scale is a consensus scale. A panel of

judges selects the statements which describe the concept under study. The

scale is developed based on the consensus. Developing this scale involvestime and as such is rarely used in the organizational concept.

Errors in Rating scales:

The respondents rating should be evaluated taking into consideration thefollowing three types of errors; leniency, central tendency and halo effect.

1. The error of leniency occurs when the respondent is either a easy rater orhard rater. Respondents are inclined to give higher score to people theyknow well. The opposite is also possible where a lower score may be

given.2. The central tendency refers to the respondents reluctance to give extremejudgments which will lead to the error of central tendency. This happensbecause the respondent may not know the object or property being rated.

3. The halo effect happens because of carrying over a generalizedimpression of the subject from one rating to another. Halo is a pervasiveerror. It is difficult to avoid when the property being studied is not clearlydefined, not easily observed, not frequently discussed, involves reactionswith others or is a trait of high moral importance.

2. RANKING SCALES:

The ranking scales are used to tap the preferences of respondents among two ormore objects and make choices among them. The respondents are usuallyasked to select the best or most preferred. This approach is satisfactory whenthere are only two choices involved. When more than two choices are present, itresults in ties. For e.g., in a response if 40 % choose product A , 35% chooseProduct B and 25% choose product C, then it cannot be concluded that A is themost preferred product because 60% did not prefer A. Three techniques can be


28/31

used to avoid this ambiguity viz., Paired Comparisons, forced choice and thecomparative scale. A brief discussion follows:

Paired comparison scale:

The paired comparison scale is used when the respondents are expected toexpress attitudes or choice between two objects at a time. It helps to assess thepreferences. In the previous example, the preference for Product A over B and Cwill enable to make better decision. However, as the number of objects to becompared increases the number of paired comparisons also increases. Thepaired choices for n objects will be [(n)(n-1)/2]. When three products arepresented to the respondents the number of paired comparison would be three[(3)(3-1)/2]. If the number of products is four then the number of pairedcomparisons would be six. More the number of objects more will be the numberof paired comparisons presented to the respondents. Paired comparison is agood method, if the number of objects to be compared is small. If too much of

comparisons are to be made then the respondents may become tired andprovide wrong answers or refuse to continue. It is suggested that 5 or 6 stimuliare not unreasonable if other questions are to be accompanied with thecomparisons. If paired comparisons are only to be dealt with, then upto 10 stimulican be accommodated.

Forced Ranking scale:

Forced ranking scale is easier and faster compared to the paired comparisonmethod. It requires the respondents to rank a list of attributes. It enablesrespondents to rank objects relative to one another among the alternatives

provided. It is more suitable in case where the number of alternatives to beranked is limited in number.

For Eg. Rank the following newspapers in the order of preference

The Hindu -

Business Line -

Indian Express -

If the number of stimuli to be ranked is 5 or less, then it is comparatively an easytask. The respondents may be normally careless if the items exceed 10.

Comparative scale:

It involves a standard against which comparison is done. The comparison scaleprovides a point of reference against which the current object under study iscompared. It enables benchmarking. However this can be used only when the


29/31

respondents have the knowledge regarding the standard against whichcomparison is made. The researchers can treat the data produced bycomparative scales as interval data since the scores reveal the interval betweenthe standard and the actual. It can also be treated as ordinal data as the rank orposition of the items are dealt with.

The Construction of measurement scales:

Five techniques are available to construct the measurement scales viz., Arbitraryapproach, Consensus Item analysis, Cumulative methods and Factoring. Theyare explained below;

1. Arbitrary scaling

Arbitrary scales are developed on ad hoc basis. It is largely based onresearchers own subjective selection of items. Several items which are

appropriate and unambiguous to the theme of study may be selected. Each itemis scored from 1 to 5 depending on the responses obtained. The results are thentotaled. Arbitrary scales are easy to develop, inexpensive and highly specific tothe theme of the study. However the major limitation is that the design approachis subjective. There is no assurance other than researchers insight that the itemschosen are representatives of the universe of content.

2. Consensus Scaling

In consensus scale the items are selected by a panel of judges after evaluationon the basis of some criteria like - relevance to the topic area, the risk of

ambiguity and the level of attitude represented by the items. This approach iswidely known as Thurstone equal appearing Interval Scale. The procedurefollowed in construction of the scale is described below

i. A large number of items/statements expressing different degrees offavourableness towards an object relating to the subject of the study,usuallymore than twenty are collected by the researcher.

ii. A panel of judges evaluates the statements. The statements are written inthecard. One statement is written in each card. The judges sort each card

into one of the 11 piles representing the degree of favourableness the statementexpresses.

iii. The sorting yields a composite position for each of the items. In case ofdisagreement between the judges, the item is discarded.

iv. For the items that are retained, median scale value between one andeleven is assigned.


30/31

v. A final selection of statements are made on the basis of the median score.Ofthe 11 piles 3 are identified by the judges as favourable , unfavourableand neutral. The eight intermediate piles are unlabelled.

The Thurstone method is widely used for developing differential scales tomeasure attitudes. The scale is more reliable for measuring a single attitude.This method of construction involves cost, time and people and hence it isimpractical. The values are assigned to the items by judges which is subjective.

3. Item Analysis scaling

In Item analysis scaling, an item is evaluated on the basis of how well itdiscriminates between those persons whose score is high and those whose totalscore is low. It involves calculating the mean score for each scale item amongthe low scorers and high scorers. The item means between the high-score group

and the low-score group are then tested for significance by calculatingt

values.Finally the items that have the greatest t values are selected for inclusion in thefinal scales.

SUMMATED SCALES OR LIKERT SCALES are developed by the item analysisapproach. Summated scales consist of a number of statements which expresseither favourable or unfavourable attitude towards an object to which therespondents is required to react. The respondents indicate the agreement ordisagreement with each of the statement. Each response is given a numericalscore and the total is obtained to measure the respondents attitude. Theprocedure for developing a Likert type scale is described below;

i. A large number of statements relevant to the object being studied iscollected. The statements express definite favourableness orunfavourableness towards the subject

ii. A trial test can be conducted with a small group of respondents who formpart of the final study. The agreement or diasagreement towards eachstatement is obtained on a five point scale.

iii. The response is scored in such a way that the response indicating themost favrourable attitude is given the highest score of 5 and the mostunfavourable attitude is given the lowest score 1.

iv. The total score of each respondent is obtained by adding the score for

each individual statementsv. The next step is to array the total scores and find out those statementswhich have a high discriminatory power. For this purpose the researchermay select some part of the highest and the lowest total scores, for eg, top25 percent and bottom 25 percent. These two extreme groups areinterpreted to represent the most favourable and the least favourableattitudes and are used as criterion groups by which to evaluate individual


31/31

statements. Thus the statements which consistently correlate with lowfavourability and with high favourability are identified.

vi. The statements which correlate with the total test are retained in the finalinstrument and all others are discarded.

The advantages of Likert scale is that it is relatively easy to construct, consideredto be more reliable and less time consuming. One of the major limitations is thatthe scale simply examines whether respondents are more or less favourabletowards the subject under study, but it cannot reveal how much more or less theyare. There is no basis for belief that the five positions indicated on the scale areequally spaced.

Date post:	09-Apr-2018
Category:	Documents
Upload:	kamal-kamalmba
View:	216 times
Download:	0 times

UNIT II Research Design

Documents