+ All Categories
Home > Documents > Psychometric Methods in Marketing Research: Part … · Psychometric Methods in Marketing Research:...

Psychometric Methods in Marketing Research: Part … · Psychometric Methods in Marketing Research:...

Date post: 29-Jul-2018
Category:
Upload: vuhanh
View: 222 times
Download: 0 times
Share this document with a friend
8
J. DOUGLAS CARROLL and PAUL E. GREEN* Psychometric Methods in Marketing Research: Part I, Conjoint Analysis Guest Editorial Marketing research, similar to the business disciplines in general, has been a long time borrower of models, tools., and techniques from other sciences. Economists, statisticians, and operations researchers have made significant contribu- tions to marketing, particularly in prescriptive model build- ing. Over the past 30 years, psychometHcians and mathe- matical psychologists have also provided a bounty of research riches in measurement and data analysis techniques. Our editorial comments on those parts of the psyehome- trician's tool kit that seem most applicable to marketing researchers. Our purview is limited. For example, we do nol discuss covariance structure analysis and latent trait models, despite their popularity and utility, and we present a limited coverage of the subareas that we do survey. Here, we focus on conjoint analysis, discussing il in terms of the problems that have motivated its more recent contributions to market- ing research, [n subsequent editorials, we will consider mul- tidimensional scaling and cluster analysis. Currently, conjoint analysis and the related technique of experimental choice analysis represent the most widely applied methodologies for measuring and analyzing con- sumer preferences. Note that the seminal theoretical contri- bution to conjoint analysis was made by Luce, a mathemat- ical psychologist, and Tukey. a statistician (Luce and Tukey 1964). Early psychometric contributions to nonmetric con- joint analysis were also made by Kruskal (196-'i). Roskam (1968), Carroll (1969. 1973). and Young (1972). The evolution of conjoint analysis in marketing research and practice has been extensively documented in reviews by Green and Srinivasan (1978, 1990), Wittink and Cattin (1989). and Wittink, Vriens. and Burhenne (1994). In addi- tion. Green and Krieger (1993) have surveyed conjoint methodology from the standpoint of new product design and optimization. A TAXONOMY OF CONJOINT METHODS The last fifteen years have witnessed a remarkable variety of new models and parameter estimation procedures for con- joint analysis. Figure 1 (adapted from Green, Krieger, and *J. Douglas Carroll is ihe Board of Governors Professor ol" Marketing and Psychology, Graduate School of Management. Rutgers University, Paul E. Green is the Sebastian S. Kresge Professor of Marketing. Whanon School, University of Pennsylvania. The authors ihank JcrTy Wind and the editor lor their comment.^ on the editorial. Sehaffer 1993a) provides a taxonomy of various approaches and a sampling of early contributions to the field. The far left-hand branch of the tree lists techtiiques for analyzing traditional, full-profile-only data. The principal parameter estimation methods are MONANOVA (Kruskal 1965), the non- metric version of PREFMAP'S vector model (Carroll 1973) and LiNMAP (ShtKker and Srinivasan 1977). Increasingly, Ordinary Least Squares (OLS) regression (Carmone, Green, and Jain 1978; Cattin and Wittink 1976) is being used for parameter fitting. The analysis of full-profile conjoint data benefits from a variety of approaches, including models that conserve degrees of freedom by fitting either prespecified functional forms or constrained parameters. For example, researchers (Herman 1988; Krishnamurthi and Wittink 1989; Pekelman and Sen 1979) augment traditional part-worth modeling with mixtures of linear, quadratic, and part-worth parame- ters. Gains in reliability and validity can also be obtained by constraining part-worths to respect within-attrihute monoto- nicity (Srinivasan, Jain, and Malhotra 1983) or various par- tial aggregation methods, such as those proposed by Green and DeSarbo (1979). Hagerty (1985), and Kamakura (1988). If tbe researcher also collects self-explicated data on indi- vidual attribute-level desirabilities and attribute impor- tances, further improvements are possible, as is illustrated by the Bayesian-Iike method of Cattin, Gelfand and Danes (1983) and the parameter constrained approach of van der Lans and Heiser (1992). In both cases, considerably more data collection is entailed, because each of these methods assumes that a large enough set of full profiles is obtained to estimate parl-worths from either profile or self-explicated data. In contrast, the hybrid models (Green 1984; Green, Goldberg, and Montemayor 1981) and the Adaptive Conjoint Analysis (ACA) model (Johnson 1987) collect a limited number of full or partial profiles that serve as ways to refine self-explicated part-worths (ACA) or estimate additional group-level parameters (hybrid models).' Because these latter approaches have fewer data demands than the Bayesian methods, they have received extensive commercial application. 'Noie thai in Part II of our editorial we will also UKC the lenn "hybrid" to refer to mixtures of continuous (spatial) and discrete (e.g.. tree structure) components in multidimensional scaling. Hopefully, the context will make the distinction clear. 385 Journal of Markeling Research Vol. XXXII (November 1995), 385-391
Transcript
Page 1: Psychometric Methods in Marketing Research: Part … · Psychometric Methods in Marketing Research: Part I, Conjoint Analysis Guest Editorial Marketing research, similar to the business

J. DOUGLAS CARROLL and PAUL E. GREEN*

Psychometric Methods in MarketingResearch: Part I, Conjoint Analysis

Guest Editorial

Marketing research, similar to the business disciplines ingeneral, has been a long time borrower of models, tools., andtechniques from other sciences. Economists, statisticians,and operations researchers have made significant contribu-tions to marketing, particularly in prescriptive model build-ing. Over the past 30 years, psychometHcians and mathe-matical psychologists have also provided a bounty ofresearch riches in measurement and data analysistechniques.

Our editorial comments on those parts of the psyehome-trician's tool kit that seem most applicable to marketingresearchers. Our purview is limited. For example, we do noldiscuss covariance structure analysis and latent trait models,despite their popularity and utility, and we present a limitedcoverage of the subareas that we do survey. Here, we focuson conjoint analysis, discussing il in terms of the problemsthat have motivated its more recent contributions to market-ing research, [n subsequent editorials, we will consider mul-tidimensional scaling and cluster analysis.

Currently, conjoint analysis and the related technique ofexperimental choice analysis represent the most widelyapplied methodologies for measuring and analyzing con-sumer preferences. Note that the seminal theoretical contri-bution to conjoint analysis was made by Luce, a mathemat-ical psychologist, and Tukey. a statistician (Luce and Tukey1964). Early psychometric contributions to nonmetric con-

joint analysis were also made by Kruskal (196-'i). Roskam(1968), Carroll (1969. 1973). and Young (1972).

The evolution of conjoint analysis in marketing researchand practice has been extensively documented in reviews byGreen and Srinivasan (1978, 1990), Wittink and Cattin(1989). and Wittink, Vriens. and Burhenne (1994). In addi-tion. Green and Krieger (1993) have surveyed conjointmethodology from the standpoint of new product design andoptimization.

A TAXONOMY OF CONJOINT METHODS

The last fifteen years have witnessed a remarkable varietyof new models and parameter estimation procedures for con-joint analysis. Figure 1 (adapted from Green, Krieger, and

*J. Douglas Carroll is ihe Board of Governors Professor ol" Marketingand Psychology, Graduate School of Management. Rutgers University, PaulE. Green is the Sebastian S. Kresge Professor of Marketing. WhanonSchool, University of Pennsylvania. The authors ihank JcrTy Wind and theeditor lor their comment.^ on the editorial.

Sehaffer 1993a) provides a taxonomy of various approachesand a sampling of early contributions to the field. The farleft-hand branch of the tree lists techtiiques for analyzingtraditional, full-profile-only data. The principal parameterestimation methods are MONANOVA (Kruskal 1965), the non-metric version of PREFMAP'S vector model (Carroll 1973)and LiNMAP (ShtKker and Srinivasan 1977). Increasingly,Ordinary Least Squares (OLS) regression (Carmone, Green,and Jain 1978; Cattin and Wittink 1976) is being used forparameter fitting.

The analysis of full-profile conjoint data benefits from avariety of approaches, including models that conservedegrees of freedom by fitting either prespecified functionalforms or constrained parameters. For example, researchers(Herman 1988; Krishnamurthi and Wittink 1989; Pekelmanand Sen 1979) augment traditional part-worth modelingwith mixtures of linear, quadratic, and part-worth parame-ters. Gains in reliability and validity can also be obtained byconstraining part-worths to respect within-attrihute monoto-nicity (Srinivasan, Jain, and Malhotra 1983) or various par-tial aggregation methods, such as those proposed by Greenand DeSarbo (1979). Hagerty (1985), and Kamakura(1988).

If tbe researcher also collects self-explicated data on indi-vidual attribute-level desirabilities and attribute impor-tances, further improvements are possible, as is illustratedby the Bayesian-Iike method of Cattin, Gelfand and Danes(1983) and the parameter constrained approach of van derLans and Heiser (1992). In both cases, considerably moredata collection is entailed, because each of these methodsassumes that a large enough set of full profiles is obtained toestimate parl-worths from either profile or self-explicateddata.

In contrast, the hybrid models (Green 1984; Green,Goldberg, and Montemayor 1981) and the AdaptiveConjoint Analysis (ACA) model (Johnson 1987) collect alimited number of full or partial profiles that serve as waysto refine self-explicated part-worths (ACA) or estimateadditional group-level parameters (hybrid models).'Because these latter approaches have fewer data demandsthan the Bayesian methods, they have received extensivecommercial application.

'Noie thai in Part II of our editorial we will also UKC the lenn "hybrid" torefer to mixtures of continuous (spatial) and discrete (e.g.. tree structure)components in multidimensional scaling. Hopefully, the context will makethe distinction clear.

385Journal of Markeling ResearchVol. XXXII (November 1995), 385-391

Page 2: Psychometric Methods in Marketing Research: Part … · Psychometric Methods in Marketing Research: Part I, Conjoint Analysis Guest Editorial Marketing research, similar to the business

386 JOURNAL OF MARKETING RESEARCH, NOVEMBER 1995

Figure 1EARLY DEVELOPMENTS IN PART-WORTH ESTIMATION METHODS

TraditionalConjoint

(IndividualAnalysis)

MONANOVA

Kruskai(196S)

PREFMAPCarroll(1973)

LlNMAPShocker SSrinivasan

(1977)

OLSRegression

Early Developments inPart-Worth Estimation Methods

Self-Explicatedand Profiles

Self-ExplicatedData Only

Partial Profiles(Subset)

ContinuousVariables

ConstrainedAttributeLevels

PartiallyAggregated

ModelsComplete Set

of Pull ProfilesSubset o(

Full Profiles

1Pekelman

& Sen(1979)

Bretton-Clark

Herman(1988)

Knshnamunhi& Wiltink

(1989)

1Order

Constraints

Srinivasan,Jain, &

Malholra(1983)

ComponentialSegmentaion

Green &DeSarbo

(1979}

OptimalScalingHagerty(198S)

ClusterAnalysis

Kamakura(1986)

Bayesian

Canin,Gelfand, &

Danes(1983)

MonolonicConstraints

van der Lans& Heiser(1990)

HybridModels

Green.Goldbetg, &Montemayor

(1981)

Gfeen(1984)

AdaptiveConjointAnalysis

Johnson(1987)

CASEMAP

Srinivasan(1988)

Srinivasan& Wyner

(1989)

In the far right-hand branch, we note that in CASEMAP

(Srinivasan 1988; Srinivasan and Wyner 1989). there are noprofile data. The entire exercise consists of self-explicateddata collection and modeling.

TRENDS IN CONJOINT ANALYSIS APPLICATIONS

Two trends have been noted in the application of conjointanalysis lo business problems. First, the early successes ofconjoint analysis have led to industry demands for tech-niques that handle ever larger numbers of attributes andattribute levels. This need, in tum, bas prompted tbe devel-opment ot data collection methods and models tbat marked-ly extend traditional OLS regression, tbe procedure typical-ly used in individualized full-proftle conjoint analysis.

Second, tbere is a growing interest in data collectionmethods and models tbat consider explicit competitive con-texts. In other words, rather than bave a respondent sort,rank, and tben rate a set of full-profile descriptions on a like-lihood-of-purcbase scale, respondents are shown sets of twoor more explicitly defined competitive profiles tbat :ire oftenidentified by brand or supplier name. Tbe respondent isasked to pick his or ber most preferred offering in eacbcboice set or, possibly, allocate 100 points across tbe alter-

natives to indicate tbe alternatives' likelibood of being cho-sen. Quantal choice models (e.g., multinomial logit and pro-bit models) are applied to data collected by tbis means.Batsell and Louviere (1991) refer to this development asexperimental choice analysis (see. also, Carson et al. 1994;Louviere 1988). We comment on eacb trend, in tum.

Coping with Large Conjoint Analysis Problems

In the 1980s, a plethora of models were introduced, inwhich self-explicated responses to attribute-level desirabili-ties and attribute importances were obtained, in addition tothe traditional evaluations of full profiles. Tbe motivationfor collecting tbe two sets of data was to increase part-wortbreliability (particularly for large-scale problems) witboutunduly increasing the data collection burden. Sawtooth'sACA and Green and colleagues' hybrid models (Green1984) collect only a limited set of either partial or full pro-files. Cross validation results for hybrid, compared to full-profile conjoint analysis, are mixed. Green and Srinivasan(1990) report tbat hybrid models tend to outperform self-explicated models, but show lower internal cross-validationtban full profile approaches for problems entailing approxi-mately six attributes or fewer. ACA and bybrid models were

Page 3: Psychometric Methods in Marketing Research: Part … · Psychometric Methods in Marketing Research: Part I, Conjoint Analysis Guest Editorial Marketing research, similar to the business

Guest Editorial 387

originally proposed to deal witb larger-scale problems; andadditional research is needed on tbeir cross-validation per-formance in the class of problems for which tbey were ini-tially designed (for the kind of research that is still needed.see Moore and Semenik 1988).

A related researcb patb deals with data procedures thatcollect only full-profile data, but introduce researcher-sup-plied parameter constraints. As we noted previously,Srinivasan, Jain, and Malhotra (1983) pioneered tbisapproach in wbicb the LINMAP program is used to estimatepart-worths, subject to researcber-supplied constraints onthe ordering of part-worths within the attribute. (The sameauthors suggest that LINMAP could be used to impose con-straints at the individual respondent level.)

Recently, Allenby. Arora, and Ginter (1995) and Lenk andcolleagues (1994) explored tbe potential of hierarcbicalBayesian metbods in conjoint analysis. The first set ofauthors extends Srinivasan, Jain, and Malhotra's (1983)research by utilizing Bayesian methods and the Gibbs sam-pler to incorporate prior ordinal constraints on conjoint part-worths. The second set sbows bow hierarcbical Bayesianmodels can be used to reduce furtber tbe usual orthogonalmain efTects plans and still estimate reliable individual part-worth functions (see, also. Allenby and Ginter 1994). Alltbree articles report good internal cross validation on thedata sets and models used for comparison.

Another research path, which also collects only full-pro-file data, employs models tbat utilize various means of dataaggregation to obtain more stable part-wortb estimates.Hagerty (1985) is the first researcber to consider tbisapproach. He proposes a factor analytic method, which hecalls optimal scaling, to provide a lower rank approximationof the original respondents-by-profiles preference responsematrix. Eacb row of this matrix can then be analyzed with anOLS regression to obtain individualized part-worths. In bisempirical data set, individual-level predictions made by tbe"smootbed" model outperformed those based on tbe originaldata, wben tbey are applied to a boldout sample.

Hagerty's (1985) study was soon followed byKamakura's (1988) cluster-based procedure. Tbis, in turn,was followed by a host of related cluster-wise regressionmethods (DeSarbo, Oliver, and Rangaswamy 1989;Steenkamp and Wedel 1992; Wedel and DeSarbo 1993;Wedel and Kistemaker 1989; Wedel and Steenkamp 1989).DeSarbo and colleagues (1992) introduce a full-fledgedlatent class conjoint model and compared tbis model to tbemore traditional approach of cluster analyzing individualpart-wortbs. Kamakura. Wedel. and Agrawal (1994) tbenextend DeSarbo and colleagues' (1992) model and Ogawa's(1987) approacb to incorporate consumer background vari-ables (see, also. Dillon 1994).

Hagerty's (1985) model leads to part-wortbs that repre-sent an amalgam of the person's and group's data. An empir-ical Bayes approach by Green, Krieger. and Schaffer(1993a) also blends individual- witb group-level responses.In contrast. DeSarbo and colleagues' (1992) cluster-wisemethods provide a set of latent groups, with a single set ofpart-wortbs for each group (i.e.. segment). In practice, eachperson is assigned to tbe group with tbe bigbest posteriorprobability.

Relatively few studies empirically compare the newermodels to other new models or such industry standards asBretton-Clark's full profile conjoint (Herman 1988) andSawtooth's ACA procedure (Johnson 1987). Green andHelsen (1989) found no validation improvement inHagerty's (1985) mode! or Kamakura's (1988) clusteringmodel over full-profile conjoint analysis. A second study,involving four additional data sets, also showed no improve-ment in Hagerty's (1985) optimal scaling over full-profileconjoint analysis (Green, Krieger and Scbaffer 1993b).

Explicit Competitive Sets

Experimental cboice analysis often combines discretechoice responses, a logit model, and fractional factorialdesigns that frequently surpass the usual ortbogonal maineffects plans used typically in ftill-profile conjoint analysis.-Early approaches to explicit competitive set evaluationsinclude Mabajan, Green, and Goldberg's (1982) study,which used Tbeil's logit approacb, DeSarbu and colleagues'(1982) study, which used Carroll, Pruzansky. and Kruskal's(1979) CANDELINC constrained multidimensional scalingmodel, and Louviere and Woodward's (1983) study, whichemployed a multinomial logit model. Experimental cboiceanalysis spurred software developers, such as SawtoothSoftware and Intelligent Marketing Systems, Inc., to devel-op and distribute software for implementing discrete choicemodels with the result that many new classes of experimen-tal designs were proposed (Anderson and Wiley 1992;Krieger and Green 1991; Kuhfield, Tobias, and Garratt1994; Lazari and Anderson 1994; Steekel, DeSarbo, andMabajan 1991).

Experimental choice models require relatively largeamounts of data; parameters frequently are estimated at tbetotal-sample (or possibly segment) level. The respondent'stask is more complex, because he or she must keep track ofeach brand's profile in what may be a set of four or morebrands with idiosyncratic attributes and levels. Altbough thedata administration task is clearly more realistic, it can bedaunting, compared to tbe relative simplicity of ACA or full-profile evaluation.

Tcx) little is currently known about the extent to whichconjoint analysis and experimental cboice analysis lead tosimilar results. Empirical comparisons by Eirod, Louviere,and Davey (1992) and Olipbant and colleagues (1992) sug-gest reasonably close correspondence in total market shareestimates, particularly if tbe attributes are monotonic.Experimental cboice modeling avoids the use of choice sim-ulators and enables the researcher to estimate limited sets ofinteraction terms—but whether tbe interactions are reliableis another matter. Can respondents dea! with the more com-plex tasks associated with experimental choice analysis?Are segments obtained from cboice analysis similar to thosefound from a post hoc clustering of part-worths? Howrestrictive is tbe Independence of Irrelevant Alternatives(IIA) assumption of the multinomial logit?

seminal thcorelical contributions lo experimenial choice analysishave been made by an econometrician, Daniel McFadiicn (1974). a mathe-matical psychologist, Duncan Luce (1959), and » psychometrician. L. L.Thurstone(1927).

Page 4: Psychometric Methods in Marketing Research: Part … · Psychometric Methods in Marketing Research: Part I, Conjoint Analysis Guest Editorial Marketing research, similar to the business

JOURNAL OF MARKETING RESEARCH, NOVEMBER 1995

One approacb, discussed by De Soete and Carroll (1983),uses the wandering vector model, witb constraints ondimensions imposed by user-supplied factorial structures ofthe stimuli (e.g., brands). Tbis method relaxes tbe HAassumption. Altbough the approach was applied to pairedcomparison data, the procedure can be extended to moregeneral cboice situations. However, many potential alterna-tives to tbe basic multinomial logit still remain to beexplored.

SORTING THINGS OUT

A new researcber entering the conjoint analysis fieldwould probably evince suiprise tbat so many models andprocedures exist side by side, eacb purporting to offeradvantages over traditional, full-profile conjoint analysis.Tbe commercially available personal computer (PC) pack-ages of Bretton-Ciark (Herman 1988), SPSS (1990)Categories, and Intelligent Marketing System's (1993a)CoNSURV are all full-profile programs. In addition, SawtootbSoftware offers a commercial (and computer administered)hybrid-like program (ACA). What, then, is happening to allof tbe other model developments?

Perhaps it is only a matter of time before hierarchicalBayes and latent class conjoint analyses become commer-cially available to industry practitioners. However, if pasthistory is any guide, there seems to be less hope for rapiddiffusion. For example, bybrid conjoint procedures havebeen used since tbe late 1970s and Green and Krieger (1994)recently extended tbeir class of models to cases in whicb allparameters are estimated at the individual level. Still, thereis no indication tbat a commercial version will beforthcoming.

Sawtooth Software and Intelligent Marketing Systems,Inc. have each responded to market demands for logit-based,experimental choice prt>cedures. Are still more computerpackages needed? What are researchers' experiences withthe current multinomial logit packages for conjointanalysis?

Wbat appears to be lacking is convincing evidence ofwhether (I) tbe newer conjoint metbods for coping witblarger numbers of attributes and levels are markedly superi-or to tbe older approacbes and (2) individualized conjoint,experimental cboice, and lateni class conjoint models lead todifferent market share estimates and, if so, which is betterunder wbich conditions. Practical answers to these interre-lated questions entail multicriterion validation and perfor-mance measures. Also, from a practical standpoint, there isneed for a programmer or entrepreneur willing to undertakethe time and expense necessary to develop, sell, and main-tain user friendly computer packages in the industrymarketplace.

One of the more interesting questions is what is the pullbetween conjoint analysis (with its emphasis on individual-ized part-wortbs) and experimental choice, with its appeal togreater realism, albeit witb aggregated or partially aggregat-ed data. Unfortunately. little empirical evidence is availableon tbe relative merits of tbe approaches. Huber and col-leagues (1992) suggest a possible marriage of the two.DeSarbo and Green (1984) propose a method that combines

conjoint data with choice data (see, also. Green and Krieger1995).

Conjoint analysts often argue for tbe value of individual-ized part-wortbs. First, tbey point out tbal a priori segmen-tation in wbich, for example, the part-worths are used as pre-dictors, can provide useful information on the part-worthprofiles of selected brands or other such prior grouping vari-ables. Second, they suggest tbat functions of the attributes,such as derived attribute itnportances or most preferred lev-els, can be useful segmenting variables.

Experimental choice proponents note the naturalness ofchoice, as opposed to purchase likelibood ratings.(Curiously, bowever, new product concept testing metbods,sucb as Burke's BASES model, routinely use 5-point likeli-hood-of-pure base scales as their primary response variable.)They also note the better-grounded tbeoretical basis under-lying the iogit model.

Unfortunately, the pace with whicb conjoint and cboicemodels bave proliferated appears to outstrip practitioners'abilities (and possibly interest) in testing tbem all. In suchcases, the traditional user often ignores the entreaties of newmodel builders, hoping tbat someone else may assume theevaluation task.

PROBLEMS AND PROSPECTS

Researchers have been so prolific that many more modelsand tecbniques bave been proposed tban have been imple-mented by industry practitioners. Tbe techniques that havereceived industry attention tend to show the subsequentcharacteristics:

1. They are among rtie earliest models proposed and enjoy a firstmover advantage.

2. The models are easy to learn and apply. Relatively inexpen-sive PC software is available to implement them.

3. Marketing research consulting (inns, following their appro-priate selt-lnteresL^, have publicized the methods, including"success stories" about the models' practical value.

4. The ideas underlying the models are relatively easy to under-stand and are credible to the nonspecialtst consumer (e.g..manager).

Examples of successful implementations are not hard tofind and include the Bretton-CIark (Herman 1988).Sawtooth Software (1994). and Intelligent MarketingSystem's (1993a. b) packages.

It is no exaggeration to say tbat a necessary condition fora new psychometric model to receive widespread applica-tion is for convenient, easy-to-use software (preferably PC-based) to be developed, distributed, and maintained.Educational seminars in tbe software's use are belpful, if notessential, as well Software developers, in effect, define tbepractice envelope. It is to tbeir advantage to remain currentwitb new developments and upgrade tbeir offerings as thestate of tbe art advances. Still, tbere are bound to be lags inadoption, as the marketplace evaluates tbe superiorityclaims made by the new models' proponents and decideswbetber the presumed benefits outweigb tbe costs ofadoption.

Page 5: Psychometric Methods in Marketing Research: Part … · Psychometric Methods in Marketing Research: Part I, Conjoint Analysis Guest Editorial Marketing research, similar to the business

Guest Editorial 389

Narrowing the Gap Between Theory and Practice

New developments in conjoint analysis are arriving sofast that even specialists find it difficult to keep up.Hierarcbica! Bayes models, latent class conjoint mtxieling,and individualized bybrid models are only a few of tbe newapproacbes and techniques that are arriving on tbe researchscene. Fortunately, the discipline bas developed a few dis-semination channels, including AMA's Advanced ResearchTechniques Forum and the Marketing Science Institute con-ferences. The Advanced Researcb Techniques Forum pro-vides a useful exchange between academics and profession-al industry practitioners. Many of the new model developershave utilized this channel to obtain user reactions, sugges-tions, and criticisms. The Marketing Science Institute pro-vides a number of outlets—researcb support, seminars, andworksbops—for disseminating new researcb methods, lnaddition, tbe AMA's practitioner magazine. MarketingResearch piays a role in idea dissemination.

Despite these vehicles, tbe gap has not narrowed appre-ciably. Part of the problem is the lack of critical comparisonsamong completing techniques. Consider, for example, thewide variety of new conjoint and experimental choice analy-sis tecbniques that now exist:

1. How do they compare with one another?2. How do they compare with the industry standards: Bretton-

CIark (Herman 1988) and Sawtooth Software (1994)?3. Can level playing fields be set up to make sound reliability

and predictive validity comparisons?4. Which techniques are good for which problem situations?5. What are the costs of type I and type H errors associated with

industry adoption of new conjoint and choice models?

Perhaps the Marketing Science Institute or an AMA taskforce could be induced to initiate procedures by whichresearchers other tban the model's own developers can com-pare the competing models. Perbaps the journals and maga-zines could emphasize the value of researcb contributionsthat implement tbis often less-than-glamorous branch ofempirical research.

Model and method comparisons can also be made at asyntbetic data level A relevant example is Vriens. Wedeland Wilms's (1994) recent article: Tbe autbors compare,using Monte Carlo simulation, nine different models relatedto metric conjoint segmentation. Software package reviewsare also useful to tbe applications researcber. Carmone andSchaffer's (1995) recent review of the Bretton-Clark,Intelligent Marketing Systems, and Sawtooth Software con-joint analysis software is an excellent example of how botbacademics and industry researchers can become apprised ofnew developments on the software scene.

Conclusions

In summary, psychometric methods in marketing haveplayed, and continue to play, an important role in theadvancement of marketing research theory, technique, andapplication. It is noteworthy that conjoint analysis and dis-crete choice modeling are mainstream methods in bothacademia and industry. Marketing has also generated itsown specialists in these methods.

Approximately 30 years ago. the methods we describehere would have been, at best, gleams in tbe eyes of a mere

handful of marketing researcbers. It is gratifying to see wbatcan happen in one and one half generations of concertedresearch. Although we continue to expect gaps between the-ory and practice, we do not gainsay the intellectual andpotential practical value of "keeping tbose models and meth-ods coming." And, at the same time, we must continue totake a critical view of tbeir "value added" over existingapproacbes in practical, business settings.

REFERENCES

Allenby, Greg M.. Neeraj Arora, and James L. Ginter (1995),"Incorporating Prior Knowledge into the Analysis of ConjointStudies." Joumal of Marketing Research, 32 (May), 152-62.

and James L. Ginter (1994), "Using Exlt^mes to DesignProducts and Segment Markets." Working Paper Series 94-41,College of Business. Ohio State University.

Anderson, D. A. and James B. Wiley (1992), "Efficient Choice SetDesigns for Estimating Cross-Effects Models," MarketingLetters, 3 (October). 357-70.

Batsell, Richard R. and Jordan J. Louviere (1991). "ExpctimcntalChoice Analysis," Marketing Letters, 2 (August), 199-214.

Carmone. Frank J., Jr. Paul E. Green, and Arun K. Jain (1978),"Robusmess of Conjoint Analysis: Some Monte Carlo Results,"Joumal of Marketing Research, 15 (May), 3{X)-303.

and Catherine M. Schaffer (1995), "Review of ConjointSoftware," Journal of Marketing Research, 32 (February),113-20.

Carroll, J. Douglas (1969). "Categorical Conjoint Measurement,"paper presented at Meeting of Mathematical Psychology, AnnArbor. MI. (Augu.st).

(1973), "Models and Algorithms for MultidimensionalScaling, Conjoint Measurement, and Related Techniques," inMultiattribute Decisions in Marketing. P. E. Green, Y. Wind, eds.Hinsdale, IL: Dryden Press, 335-37; 341^8.

Sandy Pruzansky, and Joseph B. Kruskal (1979),"CANDELINC: A General Approach to Multidimensional Analysisof Many-Way Arrays with Linear Constraints on Parameters,"Psychometrika., 45 (March), 3-24.

Carson. Richard T. et al (1994), "Experimental Analysis ofChoice," Marketing Letters. 5 (October). 351-68.

Cattin, Philippe and Dick R. Wittink (1976), "A Monte Carlo Studyof Metric and Nonmetric Estimation Methods for MultiauribuleModels," Research Paper No. 341, Graduate School of Business,Stanford University.

DeSarbo, Wayne S,, J. Dougla.s Carroll, Donald R. Lehmann, andJohn O'Shaughnessy (1982), "Three-Way Multivariate ConjointkT\a\'jii\C Marketing Science. I (Fall), 323-50.

, Alan E. Geltand, and Jeffrey Danes (1983), "A SimpleBayesian Procedure for Estimation in a Conjoint Model"Joumal of Marketing Research. 20 (February), 29-35.

and Paul E. Green (1984), "Choice-Constrained ConjointAnalysis," Decision Sciences. 15, 297-323.

-, Richard L. Oliver, and Arvind Rangaswamy (1989). "ASimulated Annealing Methodology for Clusterwise LinearRegression." Psychometrika. 54 (4), 707-36,

. Michel Wedel, Marco Vrien.s. and VenkatramRamaswamy (1992), "Latent Class Metric Conjoint Analysis,"Marketing Letters, 3 (July), 273-88.

De Soete. Geert and J. Douglas Carroll (1983), "A MaximumLikelihood Method for Fitting the Wandering Vector Model."Psychometrika. 48. 553-66.

Dillon, William R.etal (1994). "Issues in the Estimation of LatentStructure Models of Choices." Marketing Letters. 5 (October),323-34.

Elrod. Terry, Jordan J. Louviere. and Krishnakumar S. Davey(1992), "An Empirical Comparison of Ratings-Based and

Page 6: Psychometric Methods in Marketing Research: Part … · Psychometric Methods in Marketing Research: Part I, Conjoint Analysis Guest Editorial Marketing research, similar to the business

390 JOURNAL OF MARKETING RESEARCH, NOVEMBER 1995

Choice-Based Conjoint Models," Journal of MarkeiingResearch , 24 (August). 368-77.

Green, Paul E. (1984), "Hybrid Models for Conjoint Analysis: AnExpository Review," Journal of Marketing Research, 21 (May),155-69.

and Wayne S. DeSarbo (1979), "ComponentialSegmentation in tbe Analysis of Consumer Tradeoffs," Journalof Marketing, 43 (Fall), 83-91,

-, Stepben M. Goldberg, and Mila Montemayor {1981), "AHybrid Utility Estimation Model for Conjoint Analysis,"Journal of Marketing, 45 (Winter). 33-41.

and Kristiaan Helsen (1989), "Cross-ValidationAssessment of Alternatives to Individual-Level ConjointAnalysis: A Case Study." Journal of Marketing Research, 26(August), 346-350.

and Abba M. Krieger (1993), "Conjoint Analysis witbProduct-Positioning Applications." in Handbooks in OR <t MS,Vol, 5, J. Eliasbberg and G. L. Lilien. eds. New York: ElsevierScience Publisbers.

and (1994), "Hybrid Models in Conjoint Analysis:An Update," woHcing paper, Wbarton Scbool. University ofPennsylvania.

and (1995), "Attribute Importance WeigbtsModification in Assessing a Brand's Competitive Potential,"Marketing Science, in press.

-. and Catberine M. Scbaffer (1993a), "A HybridConjoint Model with Individual Level Interaction," Advances inConsumer Research, 20, 1-6.

, , and (1993b), "An Empirical Test ofOptimal Respondent Weighting in Conjoint Analysis," Journalof the Ac(uiemy of Marketing Science, 21 (Fall), 345-55.

and V. Srinivasan (1978), "Conjoint Analysis in ConsumerResearch: Issues and Outlook," Journal of Consumer Research.5 (September), I0.V23.

and (1990), "Conjoint Analysis in Marketing: NewDevelopments wilb Implications for Research and Practice."Journal of Marketing. 54 (October), 3-19.

Hagerty, Micbael R. (1985). "Improving tbe Predictive Power ofConjoint Analysis; Tbe Use of Factor Analysis and ClusterAnalysis," Journal of Marketing Research, 22 (May), 168-84.

Herman. Steve (1988), "Software for Full-Profile ConjointAnalysis," in Proceeding of the Sawtooth Conference onPerceptual Mapping. Conjoint Analysi.s, and ComputerInterviewing. M. Metegrano, ed. Ketchum. ID: SawtoothSoftware, 117-30.

Huber. Joel. Dick R. Wittink. Ricbard M Johnson, and RichardMiller (1992), Proceedings of the Sawtooth SoftwareConferences, M. Metegrano, ed. Ketcbum, ID: SawtoothSoftware.

Intelligent Marketing Systems, Inc. (1993a), "CONSURV—ConjointAnalysis Software, Verson 3.0." Edmonton, Alberta: IntelligentMarketing Systems.

(1993b), "NTELOGrr Version 2.0," Edmonton, Aberta:Intelligent Markeiing Systems.

Johnson, Ricbard M. (1987). "Adaptive Conjoint Analysis," inSawtooth Software Conference on Perceptual Mapping.Conjoint Analysis, and Computer Interviewing. M. Metegrano,ed. Ketchum. ID; Sawtootb Software. 253-65,

Kamakura, Wagner (1988), "A Least Squares Procedure for BenefitSegmentation with Conjoint Experiments," Journal ofMarketing Research, 25 (May). 157-67.

, Michel Wedel. and Jagdish Agrawal (1994). "ConcomitantVariable Latent Class Models for Conjoint Analysis,"Intemationai Journal of Research in Marketing, 11, 451 -64.

Krieger, Abba M. and Paul E. Green (1991), "Designing ParetoOptimal Stimuli for Multiattribute Choice Experiments,"Marketing Letters, 2, 337-48.

Krisbnamurthi. Laksbman and Dick R. Wittink (1989), "Tbe Part-Worth Model and Its Applicability in Conjoint Analysis," work-ing paper. College of Business Administration, University ofIllinois.

Kruskal. Joseph B. (1965). Analysis of Factorial Experiments byEstimating Monotone Transformations of the Data." Journal ofthe Royal Statistical Society, Series B, 27, 251-63.

Kubfield, Warren F., Randall D, Tobias, and Mark Garratt (1994)."Efficient Experimental Designs with Marketing ResearchApplications," Journal of Marketing Research, 31 (November),545-57.

Lazari. Andreas G. and Donald A. Anderson (1994). "Designs ofDiscrete Choice Set Experiments for Estimating Botb Attributeand Availability Cross Effects," Journal of Marketing Research,31 (August), 375-83,

Lenk, Peter J., Wayne S. DeSarbo, Paul E. Green, and Martin R.Young (1994), "Hierarchical Bayes Conjoint Analysis: Recoveryof Part-Worth Heterogeneity from Incomplete Designs inConjoint Analysis," working paper. School of BusinessAdministration. University of Michigan.

Louviere, Jordan J. (1988). Analyzing Decision Making: MetricConjoint Analysis. Beverly Hills. CA: Sage Publications, Inc.

and George Woodward (1983). "Design and Analysis ofSimulated Consumer Cboice or Allocation Experiments,"Journal of Marketing Research. 20 (November), 350-67.

Luce, Duncan R. (1959), Individual Choice Behavior: ATheoretical Analysis. New York: John Wiley & Sons, Inc.

and John W. Tukey (1964), "Simultaneous ConjointMeasurement: A New Type of Fundamental Measurement."Journal of Mathematical Psychology, 1, 1-27.

Mahajan. Vijay, Paul E. Green, and Stepben M. Goldberg (1982),"A Conjoint Model for Measuring Self-and Cross-Price DemandRelationships." Journal of Marketing Research, 19 (August),334-42.

McFadden, Daniel (1974), "Conditional Logit Analysis ofQualitative Choice Behavior," in Frontiers in Econometrics. P.Zarembka. ed. New York: Academic Press. 105-42,

Moore, William L. and Richard J, Semenik (1988). "MeasuringPreferences witb Hybrid Conjoint Analysis: Tbe impact ofDifferent Numbers of Attributes in tbe Master Design." JournalofBusine.ss Research, 16, 261-74.

Ogawa. K. (1987), "Ajn Approach to Simultaneous Estimation andSegmentation in Conjoint Analysis," Marketing Science, 6,66-81.

Olipbant, Karen. Thomas G. Eagle, Jordan J. Louviere. and DonAnderson (1992), "Cross-Task Compari.son of Ratings-Ba.sedand Choice-Based Conjoint." in 1992 Sawtooth SoftwareConference Proceedings, M. Metegrano, ed, Ketchum, ID:Sawtooth Software.

Pekelman. Dov and Subrata L. Sen (1979) "Improving Predictionin Conjoint Analysis." Journal of the American StatisticalAssociation, 75 (December). 801-16.

Roskam, E. C. I. (1968). Metric Analysis of Ordinal Data inPsychology. Netherlands: Vam Voorscboten.

SPSS (1990), Categories. Chicago: SPSS. Inc.Sawtootb Software (1994), Choice-Based Conjoint System.

Ketchum, ID: Sawtootb Software.Shocker, Allan D. and V. Srinivasan (1977). "LENMAP (Version II):

A FORTRAN IV Computer Program for Analyzing OrdinalPreference (Dominance) Judgments Via Linear ProgrammingTechniques for Conjoint Measurement," Journal of MarketingResearch, 14 (February), 101-103.

Srinivasan. V. (1988). "A Conjunctive-Compensatory Approach totbe Self-Explication o( Multiattributed Preferences," DecisionSciences, 19 (Spring), 295-305.

. Arun K. Jain, and Naresh K. Malbotra (1983), "Improvingthe Predictive Power of Conjoint Analysis by Constrained

Page 7: Psychometric Methods in Marketing Research: Part … · Psychometric Methods in Marketing Research: Part I, Conjoint Analysis Guest Editorial Marketing research, similar to the business

Guest Editorial 391

Parameler Estimation." Journal of Marketing Research, 20(November), 433-38.

and Gordon A. Wyner (1989). "CASEMAP: Computer-Assisted Self-Explication of Multi-Attributed Preferences," inNew Product Development and Testing, W. Henry, M. Menasco,and H. Takada, eds. Lexington. MA: Lexington Books, 9 1 - ! II.

Steckel. Joei H., Wayne S. DeSarbo. and Vijay Mahajan (1991),"On tbe Creation of Feasible Conjoint Analysis ExperimentalDesigns," Decision Sciences, 22. 435-42.

Steenkamp, Jan-Benedict E. M. and Micbel Wedel (1992), "FuzzyClusterwise Regression in Benefit Segmentation: Applicationand Investigation into its Validity," Journal of BusinessResearch, 26 (March), 237-49.

Thursone, L. L. (1927), "A Law of Comparative Judgement."Psychological Review, 34, 276-86.

van der Lans, Ivo A. and Willem H. Heiser (1992), "ConstrainedPart-Worth Estimation in Conjoint Analysis Using tbe Self-Explicated Utility Model." Intemationai Journal of Research inMarketing. 9. 325-44.

Vriens, M,, M. Wedel, and T. J. Wilms (1994), "Metric ConjointSegmentation Methods: A Monte Carlo Comparison," workingpaper. Faculty of Economics, University of Groningen.

Wedel, Michel and Jan-Benedict E. M. Steenkamp (1989), "FuzzyClusterwise Regt^ession Approach to Benefit Segmentation,"intemationai Joumal of Research in Marketing. 6. 241-58.

and Cor K. Kistemaker (1989), "Consumer BenefitSegmentation Using Clusterwise Linear Regression,"Intemationai Joumal of Research in Marketing, 6, 45-49.

and Wayne DeSarbo (1993), "A Latent Binomial LogitMethodology for tbe Analysis of Paired Comparison ChoiceData," Decision Sciences, 24 (6), 1157-1170.

Wittink, Dick and Philippe Cattin, (1989), "Commercial Use ofConjoint Analysis: An Update." Joumal of Marketing, 53 (July),91-96.

. Marco Vriens. and Wim Burbenne (1994), "CommercialUse of Conjoint in Europe: Results and Critical Reflections,"International Joumal of Research in Marketing, ! 1, 41-52.

Young, Forrest W. (1972). "A MiMlel for Polynomial ConjointAnalysis Algorithms," in Multidimensional Scaling: Theory andApplications in the Behavioral Sciences. Vol. I, R. N. Sbepard,A. K. Romney. and S. Nerlove, eds. New York: Academic Press.69-104.

Structural Equations Modeling has never been easier! EQS 5.1 revolutionizes path analysis.

Path analysis, confirmatory factoranalysis (CFA). and Lisrel-type mod-els have become dominant in the fieldof Marketing, You can now easily per-form these analyses with Diagrammer,EQS' new modeling tooL

Use path diagram asmodel input/output

EQS 5.1 allows you to draw presen-tation-quality path diagrams like theone above and automatically runs yourmodel according to the diagram. Youneed no prior knowledge of com-mand language or of EQS to builda model. EQS also provides a widerange of statistical and data explora-tion tools, allowing researchers to bet-ter understand their data and formu-late an optima! model.

Multivariate Software, Inc.4924 Balboa Blvd., #368Hncino, CA 91316(800) 301-4456e-mail:eqs@ ix.netcom.com

Performance and Job Satisfaction Model

Modeling Features' Diagram drawing tacility' Automatic EQS command generator' Comprehensive data manager' Nomial theory estimates'Estimates with elliptical corrections' Satorra-Bentler scaled statistics• Asymptotic distribution free• Multi-sample analysis'Analysis of means & covariances' Polyserial/polychoric correlation• RETEST ftinction for faster convergence• Effect decomposition

' Built-in simulation facility' Cut and paste data from spreadsheet' Missing value diagnosis and imputations' Multivariate LM test to improve model'Basic statistics ((tests, ANOVA,descriptive, frequency, cross tabs, varianceand covariance matrix, multiple andstepwise regression, factor analysis)

'Dataexploration plots (line, area,histogram, quainile-quaniile,normalprobability, scatter, matrix, box. Pie, bar,error bar charts)

Page 8: Psychometric Methods in Marketing Research: Part … · Psychometric Methods in Marketing Research: Part I, Conjoint Analysis Guest Editorial Marketing research, similar to the business

Recommended