Consumer Consideration Sets and Product Line Design Strategy

The decision-making approach of consideration uses screening rules to narrow-down choice

alternatives to a smaller set before making a final selection. Researchers have developed novel survey

and estimation strategies for modeling consumer consideration, and has measured success with

improvements in predictive power. A recent, related theme of design engineering, marketing, and

operations research concerns product line design using choice models. This paper compares strategic

decisions made using consideration models of consumer behavior versus traditional choice models. It

simulates the survey data collection, estimates choice models, and optimizes a product line under each

model scenario. For a single firm’s product line design problem, the study reveals that (1) using

different consumer models can lead to different optimal lines, even when models are estimated from

the same population of respondents, (2) models with lower predictive power may suggest product lines

with higher profits, (3) modeling consideration can result in more plausibly-diverse optimal product

lines, and (4) modeling consideration may reduce the risk of overestimating profits when designing a

product line. Our findings suggest that decision makers should understand the assumptions behind

consumer choice models used during product development and that modelers should expand

performance metrics beyond predictive power, including potential strategic value.

1. Introduction

In the past decade, choice models have played an important role in product design by informing

designers of how consumers' choice decisions respond to variation in product features (Besharati et al.,

2006; Chen & Hausman, 2000; Hoyle et al., 2010; Kohli & Sukumar, 1990; Luo, 2010; Macdonald et

al., 2010; Michalek et al., 2004; Morrow & Skerlos, 2011; Wassenaar & Chen, 2003). Widely-applied

traditional choice models with additive utilities (Ben-Akiva & Lerman, 1985; Train, 2009) view

product evaluation as a compensatory process, meaning that consumers allow attractive features to

2

compensate for undesirable ones. Motivated by Payne’s analysis of information search in task

complexity (Payne, 1976), researchers have improved choice modeling by including the concept of

consideration, (Hauser & Wernerfelt, 1990; Roberts & Lattin, 1997). Consideration broadly describes

a non-compensatory screening process in which consumers quickly eliminate a large number of

products to form a small set of candidate products, from which they select one final choice using

compensatory trade-offs over attributes.

While some benefits of using models representing consideration are now known, there are open

questions about the impact on strategic design decisions of using a compensatory versus non-

compensatory consumer model, particularly the impact on: product portfolio recommendations;

strategic value of specific decisions; and predicted profitability. Much research has focused on

improving predictive abilities of models, yet it is unknown whether or not predictive power equates to

strategic value. These topics appeal to product strategists and designers because product development

concerns not only how a model predicts consumer choices but also how a model informs product line

selection and feature innovation within new products.

Previous literature has demonstrated the effects of choice models (multinomial logit in particular)

on product line decisions. Hanson and Martin (1996) solved the pricing optimization of a product line

that coordinated a multinomial logit model. Chen and Hausman (2000) explored a product selection

and pricing problem under a logit model. This line of work focused on the general mathematical

method in an optimization problem without further investigating characteristics of specific optimal

product lines. In a more recent research, Liu and Dukes (2013) showed that a firm may reduce product

offerings in reaction to the search cost of a consumer’s consideration set formation. Rather than

modeling searching cost, this paper focuses on consideration sets formed by product features

screenings.

Distinct from existing research, we propose that product line strategy informed by feature-based

consideration models will differ from traditional choice model in two dimensions - feature diversity

and profit expectation. The research presented here demonstrates that consideration models have

3

different choice and profit predictions than traditional compensatory models. Consider a simple

example screening rule: households with children do not purchase two-seated vehicles for their

primary vehicle. While a multinomial logit model can capture this fact with strongly-negative and

demographic-dependent partworths on two-seater body types, the logit model's compensatory nature

will suggest that there are scenarios in which these households will purchase a two-seater as a primary

vehicle, such as for an extremely low price. In contrast, consideration models that explicitly identify

body type as a non-compensatory feature guide design decisions to body types with sufficient

passenger capacity to satisfy households with children, and capture the nuance that there is no price for

which a household is willing to store their child(ren) in the trunk.

To compare the performance of compensatory and non-compensatory models for product line

design, we address three issues: First, we test model capabilities in an environment where a valid

performance benchmark exists, in other words, where there is a right answer, which we term "true

behaviors". We describe a synthetic data experiment built to compare model performance in scenarios

where synthetic consumers use complex screening rules. Second, different models use different data

collection coupled with model estimation procedures, especially so for adaptive question methods. Our

experiment executes different surveys on the simulated population, coupled with the estimation

approach. Third, we use the estimated models in a product design problem and compare the strategic

value of design predictions. Our experiment simulates a design optimization process, and

quantitatively compares design outcomes in terms of feature commonality and profitability under

different choice model scenarios.

This simulation study provides two insights into consumer models for product design: (1) In the

context of design optimization, consideration models result in different design strategies than

traditional choice models; (2) Consideration models do not over-predict profits, while some

compensatory approaches have this weakness; (3) A model's predictive accuracy for choice does not

correspond to its overall strategic value, e.g. in predicting profit.

4

The remainder of this paper proceeds as follows: Section 2 reviews consideration modeling

methods and research that compared consideration models and choice models. Section 3 describes our

synthetic simulation framework in detail, including experimental design, synthetic data generation

process, model estimation methods, product design problem, and metrics used to evaluate model

performance. Section 4 presents the results of the simulation. Section 5 provides discussions, followed

by the managerial implications in Section 6. Section 7 concludes.

2. Background and Literature Review

Consumers often use “fast and frugal” (Gigerenzer & Gaissmaier, 2011) screening rules to

eliminate products before carefully making trade-off decisions on a smaller subset of products. Factors

such as evaluation complexity (Payne, 1976), information cost (Bröder, 2000), and time pressure

(Rieskamp & Hoffrage, 2008) encourage this screening approach. The screening process is also

referred to as “consideration” (Hauser & Wernerfelt, 1990; Roberts & Lattin, 1997). Over the years, a

number consideration screening theories have developed:

Aspirational: Simon (1956, 1972) first proposed the theory of bounded rationality, in which

decision makers will stop searching through alternatives once they hit upon a benefit from an option

that exceeds some “aspirational limit”. Gilbride & Allenby (2004) implemented this idea in

consideration modeling with a modification: consumers will consider all products for which the

product’s utility exceeds some aspirational criteria.

Conjunctive: Bettman et al. (1998) proposed a decision rule in which a consumer will consider a

product only if all its screened attributes are acceptable. For example, “I will consider a vehicle if it is

hybrid AND it is Toyota”. Such a rule can easily be cast in terms of “attribute-specific aspirational

rules” for all continuous-valued attributes, such as vehicle fuel economy.

Disjunctive: a consumer will consider a product as long as one of its screened attributes is

acceptable (Gilbride & Allenby, 2004). For example, “I will consider a Toyota OR a hybrid vehicle”.

Subset conjunctive: a consumer will consider a product if a certain number of its screened

attributes are acceptable (Jedidi & Kohli, 2005). For example, suppose a consumer screens on brand,

5

price and powertrain and that a considered product must have at least two acceptable features. Then a

subset conjunctive rule may state “I will consider a vehicle if it is a Toyota hybrid OR if it is a hybrid

with price under $25,000 OR if it is a Toyota under $25,000”.

Disjunction of conjunctions: a consumer will consider a product if it satisfies at least one set of

conjunctive rules. For example, if a consumer screens by brand, price and powertrain, a disjunction of

conjunctions rule can be “I will consider a vehicle if it is a Toyota OR if it is a hybrid under $25,000”.

Disjunctions of conjunctions are a general category of screening rules and thus, while potentially

powerful, are extremely difficult to estimate (Hauser et al, 2010).

“Consideration set explosion” is a classic stream of consideration modeling methods relied on

choice panel data. These consideration models probabilistically captured the existence and members of

a consideration set. This formulation presented choice probabilities conditionally on the consideration

set probabilities (Manrai & Andrews, 1998). Yet, the computation burden of this method grows

exponentially as the number of the product alternatives grows. Parameterizing the attribute acceptance

distribution relaxes this problem by reducing required estimation dimensions (Siddarth et al., 1995;

Chiang et al., 1999; Jedidi & Kohli, 2005).

Recent proposals not only inherit parameterization, but also infer screening rules more efficiently

using consideration data. That is, newer methods estimate models of consideration from survey data

asking for consider/not-consider (“reject”) responses. These advanced methods often corresponded to

their own specific models, for example: a greedoid algorithm for lexicographic screening (Kohli &

Jedidi, 2007; Yee et al., 2007); Bayesian adaptive learning for conjunctive screening (Dzyabura &

Hauser, 2011); and support vector machine (SVM) learning for aspirational screening (Huang & Luo,

2015). The following section describes how Bayesian adaptive learning and SVM learning are applied

in our simulation.

It should be clarified that choice models are not necessarily labeled as “compensatory”, since

choice decisions can also include non-compensatory processes, such as “elimination-by-aspects”

6

(Tversky, 1972). In this paper, the phrase “choice models” refers to model structures that do not

explicitly include non-compensatory screening; Section 3.2 provides examples.

Researchers have compared the predictive capabilities of non-compensatory consideration and

compensatory choice models. Table 1 reviews research from a variety of product categories where

modeling consideration improved predictive power.

Table 1. Performance improvement in recent consider-then-choose choice models constructed from stated

preference data. Abbreviations are as follows: HB: Hierarchical Bayes; MLE: Maximum Likelihood; HR:

“hit rate” (frequency of correct prediction on hold-out samples); KLD: Kullback-Liebler Divergence

(Kullback & Leibler, 1951); TAU: Kendall’s Tau (Kendall, 1955); LL: Log-likelihood (increase). See

references for additional details.

Consideration Model (Non-compensatory)

Choice Model

(Compensatory)

%Improvement of Predictive

Power

Product

Reference

Subset- conjunctive

LP Logit

HR 1.1%

Batteries

(Jedidi & Kohli,

2005)

Conjunctive screening

MLE Logit

HR 7.1%

Cameras

(Hauser et al.,

2009)

Lexicographic by aspects

HB Logit

HR 8.7%

Smartphones

(Yee et al., 2007)

Unstructured direct elicitation HB Ranked Logit

KLD 9.1%

Cellphones

(Ding et al.,

2011) Conjunctive

“Cut-off rules”

MLE Logit

LL 14.0%

Rental Cars

(Swait 2001)

Lexicographic

HB Logit

HR 4.5%, KLD

54.5%

GPS Units

(Hauser et al.,

2010)

Conjunctive

HB Logit

KLD 19.5%

Vehicles

(Dzyabura &

Hauser, 2011)

Research has also revealed that compensatory choice models can approximate non-compensatory

behaviors, including consideration. For example, the nested logit model can approximate consideration

set formation and subsequent choice (Swait, 2001). Additionally, an aspect-coded, linear weighted

additive model with specific sequences of partworths can recover lexicographic and conjunctive rules

(Martignon and Hoffrage, 2002).

The capability of choice models to mimic consideration behaviors has limitations, however. For

example, Andrews and Manrai (1998) showed through a synthetic data experiment that multinomial

logit models accommodate consideration only for specific degrees of heterogeneity in consideration

7

and consideration set sizes. Further experiments revealed that underspecifying consideration led to

more predictive errors than did underspecifying compensatory heterogeneity (Abramson et al. 2000;

Andrews et al. 2008), suggesting that capturing consideration is more important than capturing

respondent-specific preferences.

Nested logit models can fit choice data from consideration-based decisions exactly with nests that

mimic successive consideration decisions; however, predictions from these models are fundamentally

compensatory because the “nest” choices are driven by the expected maximum utility of choosing an

option from within the nest (Train, 2009), which introduces a compensatory structure on choices not

necessarily present in the data generating process. Research fields that use these predictions include

operations research and engineering design. Literature comparing consideration models and choice

models is relatively sparse in operations research and engineering design. Extant applications included

the usage of conjunctive and disjunctive models in a product selection system (Besharati et al., 2006),

and the prediction of consideration sets via network analysis (Wang & Chen, 2015). These studies do

not focus on comparing design outcomes when making strategic decisions using choice models. One

previous synthetic data experiment has suggested that consideration models lead to more profitable

product lines with less data compared to logit, mixed logit, and nested logit models when the

population uses consideration behaviors (Long & Morrow, 2015). Shin and Ferguson (2015)

performed a choice survey based synthetic experiment where the latent class logit model and

hierarchical Bayes mixed logit model yielded similar optimal designs to those by using a conjunctive

model. However, neither of these two studies took into account how data collection efforts or

estimation strategies would differ according to the model assumptions (as we do here), given that the

consideration models and choice models in these two studies were estimated from the same discrete

choice data. The following section describes our proposed simulation framework that tailors survey

experiments and estimation methods for two consideration models and two choice models.

8

3. Simulation Framework

The simulation procedure, shown in Figure 1, has four steps: 1) generate synthetic population with

"true behavior", 2) decision experiment & preference estimation to predict this behavior, 3) product

line optimization with respect to preference estimations, and 4) evaluation of product line

configurations & profit.

Figure 1. Simulation framework flowchart

Step 1 generates a synthetic population of true behaviors that allow the population to make

consideration and choice decisions in the subsequent decision experiments. Two scenarios are

investigated, representing two different decision processes. In scenario I, the population uses subset-

conjunctive screening and in scenario II the population uses aspirational screening. Both scenarios are

followed by a compensatory choice behavior to pick a final selected vehicle. Section 3.1further

describes the screening rules and utility partworths that constitute the synthetic true behavior.

In Step 2, the simulation queries the synthetic respondents in four survey experiments: two

adaptive consideration and two discrete choice. The consideration experiments infer individual-

specific consideration rules by adaptively combining vehicle profile acceptance questions and response

analysis in an iterative process. The adaptive estimation methods are: Bayesian method to estimate a

9

conjunctive model (model A) and support vector machine to estimate an aspirational model (model B).

In the section set of survey experiments, discrete choice surveys with pre-designed multiple choice

question sets estimate: a multinomial logit model (model C) and a latent class model (model D).

Sections 3.2 and 3.3 provide more details.

In Step 3, models A-D above inform product line strategy through distinct optimizations,

described in Section 3.5 In Step 4, the optimal product lines are assessed with metrics of feature

commonality and profitability presented, in Section 3.6 The paper then compares models A-D on

predictive power, product line strategies, and profitability with the results represented in Section 4.1-

4.3.

The subject of the simulation is the design of consumer vehicles. A profile vector 𝒙, a binary string,

represents vehicle configuration. Each string element represents an attribute level, with 1 representing

that the profile has the corresponding level and 0 otherwise. The vehicle profile attributes and levels

are given in Table 2, and based on Urban et at (2010) and Dzyabura & Hauser (2011). There are 8

attributes with 53 attribute-levels in total.

Table 2. Attributes and levels for the vehicle case study. Attributes Levels

(#) (Values) Body Style 9 Sports car, Hatchback, Compact Sedan, Standard Sedan, Crossover, Small SUV,

Full-size SUV, Pickup Truck, Minivan Make 21 BMW, Buick, Cadillac, Chevrolet, Chrysler, Dodge, Ford, GMC, Honda, Hyundai,

Jeep, Kia, Lexus, Lincoln, Mazda, Nissan, Pontiac, Saturn, Subaru, Toyota, VW Price 7 $12K, $17K, $22K, $27K, $32K, $37K, $45K Cylinders 3 4, 6, 8 Powertrain 2 Hybrid, Gasoline MPG 5 15, 20, 25, 30, 35 Quality Rating 3 3, 4, 5 Crash Rating 3 3, 4, 5

3.1. Synthetic Behaviors

Step 1 of the simulation generates true behaviors, consisting of screening rules and compensatory

trade-offs, for a synthetic population.

10

Consideration Stage, Scenario I: Subset-conjunctive screening

When exposed to a profile 𝒙𝑗 in a consideration question, respondent 𝑖 in Scenario I generates the

answer according to:

{𝐶𝑜𝑛𝑠𝑖𝑑𝑒𝑟, 𝑖𝑓 𝜹𝑖

𝑇𝒙𝑗 ≥ 𝐾𝑖, 𝐾𝑖 ∈ {1, … , 𝐴 }

𝑅𝑒𝑗𝑒𝑐𝑡, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 (1)

where the binary vector 𝜹𝑖 is an individual-specific screening rule set with elements of value 1

representing that the corresponding attribute-level is acceptable, and A is the total number of attributes;

we require that at least one level of each attribute is acceptable. With this formulation, 𝐾𝑖 =A

corresponds to the conjunctive model, 𝐾𝑖=1 yields a disjunctive screening rule set, and anything in

between is subset-conjunctive. We sampled screening rule sets in the synthetic population as follows:

All possible rule sets (i.e. 𝜹𝒊 and 𝐾𝑖 combinations in Eqn.(1)) were enumerated. The corresponding

proportion of profiles considered for each rule set was computed. The rule sets were then categorized

into 100 bins (i.e.0~0.01, 0.01~0.02, … 0.99~1) according to the proportion of profile considered

given by the rule sets. Suppose 𝑐𝑖 is the proportion of profiles considered by individual 𝑖. We first

draw the proportion of profiles considered, 𝑐𝑖 , from a lognormal distribution with mean 0.4 and

variance 0.1, then one of the rule sets from the bin that 𝑐𝑖 fell in was randomly selected for individual i.

Consideration Stage, Scenario II: Aspirational screening

When exposed to a profile 𝒙𝑗 in a consideration question, respondent 𝑖 in Scenario II generates the

answer according to:

{𝐶𝑜𝑛𝑠𝑖𝑑𝑒𝑟, 𝑖𝑓 𝒗𝑖

𝑇𝒙𝑗 ≥ 𝛾𝑖

𝑅𝑒𝑗𝑒𝑐𝑡, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 (2)

where 𝒗𝑖 is the vector of individual specific partworths and 𝛾𝑖 is the aspirational limit. Note that

subset-conjunctive screening (Eqn. (1)) is a special case of aspirational screening (Eqn. (2)). Our

simulation samples partworths and aspirational limits for each synthetic individual using a scheme

inspired by Jedidi et al. (1996). We first draw the proportion of profiles considered, 𝑐𝑖 , from a

lognormal distribution with mean 0.4 and variance 0.1. The partworths 𝒗𝑖 were then randomly drawn

11

from a uniform distribution on [-1, 1]. Next, the utilities of all possible profiles were computed and

sorted in descending order. An aspirational limit 𝛾𝑖 was assigned such that the upper fraction of

profiles 𝑐𝑖 in descending order would be considered, and others rejected.

Choose Stage: Utility partworths (used with both Scenario I and II)

Either scenario described above results in a "consideration set" of products that the respondents

then evaluate to choose the one with maximum random utility, a compensatory procedure in which the

benefits of one attribute/level can compensate of inadaquacies in another. Respondent 𝒊 chooses option

j from the set of considered profiles 𝑪𝒊 such that 𝒋 = 𝐚𝐫𝐠 𝐦𝐚𝐱𝒌∈𝑪𝒊

{𝜷𝒊𝑻𝒙𝒌} + 𝝐𝒊,𝒌 , where 𝜷𝒊 is the

individual-specific partworth vector. The elements represent the attributes and levels listed in Table 2,

and an outside good term. 𝝐𝒊,𝒌 is the random disturbance following extreme value distribution across

the respondents and options. Table 3 describes the sampling procedure for each 𝜷𝒊. In general, the

synthetic respondents prefer lower prices, higher MPG, higher quality, and higher crash safety ratings

(all other attributes being equal). The uniform distributed random draws of partworths are also scaled

between [-1, 1], so that no attribute is systematically more “important” than any other.

12

Table 3. The sampling process of utility partworths for a synthetic respondent i.

Attributes # of levels

Partworths (𝜷) Sampling

Body Style 9 Draw each level’s 𝛽 from uniform distribution [-1,1].

Make 21 Draw each level’s 𝛽 from uniform distribution [-1,1].

Price 7 Draw 7 random numbers from uniform distribution [-1,1], then assign the random numbers to the 𝛽𝑠 in an order such that: 𝛽$12𝐾 > 𝛽$17𝐾 > 𝛽$22𝐾 > 𝛽$27𝐾 > 𝛽$32𝐾 > 𝛽$37𝐾 > 𝛽$45𝐾 .

Cylinders 3 Draw 3 random numbers from uniform distribution [-1,1], then assign the random numbers to the 𝛽𝑠 in an order such that: with 0.5 chance 𝛽4𝑐𝑦𝑙 = 𝛽6𝑐𝑦𝑙 = 𝛽8𝑐𝑦𝑙 (indifference on engine sizes);

with 0.5 chance 𝛽4𝑐𝑦𝑙 < 𝛽6𝑐𝑦𝑙 < 𝛽8𝑐𝑦𝑙 (prefer larger engine sizes).

Powertrain 2 Draw 2 random numbers from uniform distribution [-1,1], then assign the random numbers to the 𝛽𝑠 in an order such that: with 0.25 chance 𝛽𝑔𝑎𝑠 > 𝛽ℎ𝑦𝑏𝑟𝑖𝑑 (prefer gasoline);

with 0.50 chance 𝛽𝑔𝑎𝑠 = 𝛽ℎ𝑦𝑏𝑟𝑖𝑑 (indifference on powertrain);

with 0.25 chance 𝛽𝑔𝑎𝑠 < 𝛽ℎ𝑦𝑏𝑟𝑖𝑑 (prefer hybrid).

MPG 5 Draw 5 random numbers from uniform distribution [-1,1], then assign the random numbers to the 𝛽𝑠 in an order such that: 𝛽15𝑀𝑃𝐺 < 𝛽20𝑀𝑃𝐺 < 𝛽25𝑀𝑃𝐺 < 𝛽30𝑀𝑃𝐺 < 𝛽35𝑀𝑃𝐺

Quality Rating 3 Draw 3 random numbers from uniform distribution [-1,1], then assign the random numbers to the 𝛽𝑠 in an order such that: 𝛽3𝑠𝑡𝑎𝑟 < 𝛽4𝑠𝑡𝑎𝑟 < 𝛽5𝑠𝑡𝑎𝑟

Crash Rating 3 Draw 3 random numbers from uniform distribution [-1,1], then assign the random numbers to the 𝛽𝑠 in an order such that: 𝛽3𝑠𝑡𝑎𝑟 < 𝛽4𝑠𝑡𝑎𝑟 < 𝛽5𝑠𝑡𝑎𝑟

Outside good 1 Draw from standard normal distribution.

(Subscripts of partworths label the attribute levels)

3.2. Estimating Consideration Models from Adaptive Surveys

In simulation Step 2, the synthetic "respondents" generated in Step 1 were "exposed" to one of

four experiments, two adaptive consideration experiments and two discrete choice experiments. In the

consideration experiments, respondents were exposed to a sequence of single profile to which they

respond “consider” or “reject”. The two consideration experiments and their corresponding models and

estimation methods are described below: Model A, a conjunctive model with a Bayesian estimation

method; and Model B, an aspirational model with a support vector machine estimation method.

Model A. Conjunctive model

The formulation 𝜹𝑖𝑇𝒙𝑗 ≥ 𝐾𝑖 in Eqn.(1) collapses to a conjunctive model when 𝐾𝑖 equals the

number of attributes. Dzyabura and Hauser (2011) propose an adaptive machine learning algorithm to

estimate the individual specific screening rule 𝜹𝒊 ; we summarize the approach here, and refer the

reader for a complete description. They first parameterize the screening rules as attribute-level

13

acceptance probabilities, assuming these probabilities to be independent across attributes and levels.

As illustrated in Figure 2, the process is initialized with asking synthetic respondents to configure their

first considered profile. This configuration is a random draw from the consideration set defined by the

respondent’s synthetic screening rule. Following the first configuration, the priors of the acceptance

probabilities are initialized. Then the next profile is selected to minimize the expected information

entropy over possible questions. Based on a history of profile queries and responses, posterior

distributions are updated using Bayes theorem. The update and selection processes are iterated until a

maximum number of questions are asked. After the survey, estimated posteriors on the attribute level

acceptance probabilities are converted to a binary screening rule such that posteriors with probabilities

< 0.5 are interpreted as “unacceptable” on the corresponding attribute-level, and “acceptable”

otherwise; this is equivalent to taking the modal (most likely) consideration prediction in an acceptable

attribute-levels conjunctive screening model.

Figure 2. Adaptive survey to estimate conjunctive model

Model B. Aspirational model

Given the formulation 𝒗𝑖𝑇𝒙𝑗 ≥ 𝛾𝑖 in Eqn. (2), estimating screening rule 𝒗𝒊 for individual 𝑖 can be

obtained by solving a soft-margin support vector machine (SVM) problem (Bishop 2006):

min1

2||𝒗𝑖||

2+ 𝐶∑𝜉𝑗

𝑤. 𝑟. 𝑡 𝒗𝒊, 𝛾𝑖, 𝜉𝑗

𝑠. 𝑡 𝑦𝑗 ⋅ (𝒗𝑖𝑇𝒙𝑗 − 𝛾𝑖) ≥ 1 − 𝜉𝑗

𝜉𝑗 ≥ 0 for all 𝑗 = 1, ⋯ , 𝑁

(3)

where N is the number of questions asked and 𝑦𝑗 is the response to profile 𝒙𝑗 (value 1 if “consider”, -1

if “reject”). 𝜉𝑗 is the classification error in observation j, and 𝐶 is the penalty on the sum of

14

classification errors. Huang & Luo (2015) have combined SVM learning with an adaptive question

strategy in which the profile with the smallest distance to the decision hyperplane ℎ(𝒙) = 𝒗𝑖𝑇𝒙𝑗 −

𝛾𝑖was selected. Formally, Huang & Luo propose an algorithm with two distinct features we exclude

here. First, they utilize the soft-margin SVM technique (Boser et al, 1992; Cortes & Vapnik, 1995;

Vapnik et al, 1996) that allows for consideration behaviors that are not linearly separable Because we

specify linearly separable consideration behaviors for our synthetic population, we do not need to

utilize the soft-margin SVM. Second, Huang & Luo propose an approach suitable for response errors.

Because we simulate synthetic respondents that respond faithfully with respect to their screening rules,

we exclude the response error relevant portions of their algorithm. The steps of adaptive survey to

estimate aspirational rule is illustrated in Figure 3.

Figure 3. Adaptive survey to estimate aspirational model

3.3. Estimating Choice Models from Discrete Choice Surveys

This subsection presents two discrete choice experiments that estimate two choice models: Model

C, multinomial logit; and Model D, latent-class logit. Both models can be estimated from traditional

discrete choice surveys using the Maximum Likelihood method. In the choice surveys, synthetic

respondents were asked to choose one profile from a set of alternatives, or choose the none option. In

reality, a respondent usually answers only one survey. In this synthetic experiment, we studied the

effect of survey design on predictive power. We generated 25 survey versions for Models C and D,

described below, which varied in the number of questions (from 10 to 35), the number of alternatives

(3 and 4), and the generation methods (SAS macros and Sawtooth software schemes). For Models C

and D, we estimated 25 sets of partworths with these 25 survey versions. We then generated 25 fresh

15

sets of validation choice surveys with the same variety of number of questions, number of alternatives

and generation method. For each of Model C and D, the and the following profit analysis. The surveys

generated by SAS macros aim for “D-efficiency” (Kuhfeld, 2010), and the surveys generated by

Sawtooth schemes (Sawtooth Software 2013) emphasizes level balance and level minimal overlap in

the questions (Huber & Zwerina 1996; Johnson et al. 2013).

Model C. Multinomial logit model

The multinomial logit model is one of the most established models in choice modeling, with over

4 decades of use (Ben-Akiva & Lerman, 1985; McFadden, 1974), where

𝑃𝑗𝑀𝑁𝐿 =

exp 𝜷𝑻𝒙𝑗

∑ exp 𝜷𝑻𝒙𝒌𝐽𝑘=0

(4)

In the weighted additive utility 𝜷𝑇𝒙𝑗, partworths vector 𝜷 is homogeneous across the population. To

be consistent with survey setting, in options 0, …, J, “none” option is labeled as 0, and product

alternatives labels 1, …, J.

Model D. Latent class logit model

The latent class logit model (Swait 1994; Greene 2001; Greene & Hensher 2003) captures

preference heterogeneity by sorting individuals into one of Q classes with probability 𝛼𝑞 . The

conditional choice probability of choosing option j given the sorted class 𝑞 takes the logit formula:

𝑃𝑗|𝑞𝐿𝐶𝐿 =

exp 𝜷𝑞𝑇𝒙𝑗

∑ exp 𝜷𝑞𝑇𝒙𝑘

𝐽𝑘=0

(5)

The likelihood for option j to be chosen is the sum over the product of latent class probability and the

conditional choice probability across all classes:

𝑃𝑗𝐿𝐶𝐿 = ∑ 𝛼𝑞 ⋅

𝑄

𝑞=1

𝑃𝑗|𝑞𝐿𝐶𝐿 (6)

Both multinomial logit and latent-class logit models are fitted via maximum likelihood method by

solving:

maximize ∑ ∑ 𝑁𝑗,𝑚 log 𝑃𝑗,𝑚𝑀𝑜𝑑𝑒𝑙𝐽

𝑗=0𝑀𝑚=1 (𝜽) (7)

16

with respect to 𝜽 (plus possible constraints)

For the multinomial logit model related to Eqn.(4), the estimated coefficients are 𝜽 = 𝜷, for latent

class logit model related to Eqn.(5) and (6), 𝜽 = (𝛼𝑞 , 𝜷𝑞) for 𝑞 = 1, … , 𝑄. For both models, the

partworths of the same attribute are constrained to have a zero sum. The LCL model, in addition, is

also constrained to satisfy ∑ 𝛼𝑞 = 1𝑄𝑞=1 .

3.4. Model Specification Summary

The next step was to use estimated Models A-D to inform the profit formulation in the product line

optimization. Before presenting the product line optimization, we first examine how the four models

aligned with the “true” behaviors. In Table 4, the first two columns compare how the respondents in

Scenario I and II make decisions in the experiments to the Models A-D estimates. The third column of

Table 4 summarizes how to use model information to compute choice probabilities. The consideration

Models A and B can only predict consideration sets, while C and D predict final choice probabilities.

Thus, for Models A and B, all considered products are assigned equal choice probability in the choice

stage. This allows our experiment to clearly separate the effects of investing in models of

consideration versus choice.

17

Table 4. The comparison of the “true” behavior and models specifications in the consideration experiment, choice experiment, predictive power validation, and product line optimization

Consideration Experiment Choice Experiment Choice probabilities in

Predictive Power Validation And Design Optimization

“True” Behavior Scenario I

Use heterogeneous subset-conjunctive screening rule to

decide “consider/reject”

Use heterogeneous subset-conjunctive screening rule to

form consideration set +

Use heterogeneous utility partworths to choose one from the considered profiles or the

none option.

Use heterogeneous subset-conjunctive screening rule to

form consideration set +

Use heterogeneous utility partworths to compute logit choice probabilities within

consideration set +

Aggregate the choice probabilities over the population

“True” Behavior Scenario II

Use heterogeneous aspirational screening rule to

decide “consider/reject”

Use heterogeneous aspirational screening rule to form

consideration set +

Use heterogeneous utility partworths to choose one from the considered profiles or the

none option.

Use heterogeneous aspirational screening rule to form

consideration set +

Use heterogeneous utility partworths to compute logit choice probabilities within

consideration set +

Aggregate the choice probabilities over the population

Model A. Conjunctive

Estimate heterogeneous conjunctive screening rule

N/A

Use heterogeneous conjunctive screening rule to form

consideration set +

Assign equal choice probability within consideration set

+ Aggregate the choice

probabilities over the population

Model B. Aspirational

Estimate heterogeneous aspirational screening rule

N/A

Use heterogeneous aspirational screening rule to form

consideration set +

Assign equal choice probability within consideration set

+ Aggregate the choice

probabilities over the population Model C. Multinomial logit

N/A

Estimate homogeneous utility partworths

Use the homogeneous utility partworths to compute choice

probabilities

Model D. Latent-class logit

N/A Estimate multiple sets of utility partworths for different latent

classes

Use latent-class specific utility partworth to compute choice

probabilities +

Aggregate choice probabilities over the latent classes

18

3.5. Product Line Optimization

In this step, the simulation uses four estimated models to inform a single firm’s product line design

problem. Suppose an automaker aims to design J distinct products (products with non-identical

engineering features) to maximize profit across all J vehicles. With the brand fixed, the firm solves the

following optimization problem:

maximize ∑ ∑𝑖=1𝐼𝐽

𝑗=0 𝑃𝑖,𝑗𝑀𝑜𝑑𝑒𝑙(𝒑1, ⋯ , 𝒑𝐽, 𝒚1, ⋯ , 𝒚𝐽) ( �̅�𝑗 − 𝑐𝑜𝑠𝑡(𝒚𝑗))

with respect to 𝒑1, ⋯ , 𝒑𝐽 and 𝒚1, ⋯ , 𝒚𝐽

subject to (𝒑𝑗, 𝒚𝑗) ⋅ (𝒑𝑘 , 𝒚𝑘)𝑇 ≤ 𝐴 − 1 for any (𝑗, 𝑘) ∈ {1, ⋯ , 𝐽} × {1, ⋯ , 𝐽} AND 𝑗 ≠ 𝑘

(8)

Here, 𝒑𝑗 is the binary level coded vector of price attribute and 𝒚𝑗is the non-price attributes’ binary

vector consistent with the levels in the survey experiment. �̅�𝑗 represents the corresponding numeric

value mapped from 𝒑𝑗. The objective, profit, is the product of the predicted choice probability and the

markup summed over the sampling population with I respondents (I=500) and the whole product line

of J vehicles (J=10). The cost function, cost, is a constant that is the same for all vehicle configurations,

to avoid "control" effects of cost on profitability solutions. The constraint in Eqn.(8) serves to ensure

that two products in a product line are distinct. The choice probability for an individual 𝑖, 𝑃𝑖,𝑗𝑀𝑜𝑑𝑒𝑙 , is

computed from one of the estimated models, as summarized in Column 3, Table 4. We perform this

optimization for 21 brands separately. When a brand is optimizing the 10 products in the product line,

it does not take into account other brands’ product information, unlike in a multi-firm competitive

game.

In our optimization problem, the number of possible product lines grows rapidly as the number of

products increases. Moreover consideration models generate non-smooth choice probabilities over

continuous relaxations of the discrete feature selection problem, potentially problematic for branch-

and-bound like strategies for combinatorial optimization. Thus a genetic algorithm (GA) was used to

search for optimal attribute level combinations with a checking scheme to remove identical profile

pairs. During a trial, the GA terminated the search if it detected relative changes of both the best

19

fitness and the average fitness of the population to be less than 0.0001. Our approach ran 100 trials

with different initial genome population of size N = 50, with each genome representing an initial

product line portfolio. Among these trials, the solution with the highest objective value is reported.

While other more intelligent and tailored methods may exist for this problem, such as mixed-integer

programming, our purpose here is not to investigate the most efficient portfolio optimization strategies.

Although GAs offer limited a priori guarantees of global optimality, they can still perform quite well

in practice (Belloni et al., 2008; Gen & Cheng, 2000; Mitchell, 1998).

3.6. Performance Measures

In this subsection, we quantify metrics for performance measures that we will use to

compare Models A-D: predictive power, feature commonality, and profitability.

Predictive Power

We apply relative likelihood (RL) to quantify how much of the synthetic true behaviors can be

recovered by the estimated models. By definition, relative likelihood measures the likelihood of a

model relative to the likelihood of the actual distribution. Relative likelihood is also inversely

consistent with Kullback-Leibler divergence used previous consideration model literature (see Ding et

al., 2011; Dzyabura & Hauser, 2011; Hauser et al., 2010). In this study, the relative likelihood is given

by:

𝑅𝐿 = (∏ (∏(𝑃𝑗,𝑚

𝑀𝑜𝑑𝑒𝑙

𝑃𝑗,𝑚𝑇𝑟𝑢𝑒 )

𝑃𝑗,𝑚𝑇𝑟𝑢𝑒𝐽𝑚

𝑗=0

)

𝑀

𝑚=1

)

1/𝑀

(9)

Where 𝑃𝑗,𝑚𝑀𝑜𝑑𝑒𝑙 represents the predicted choice probability of alternative 𝑗 (with 𝑗 = 0 labeling the none

option) in question 𝑚 , and 𝑃𝑗,𝑚𝑇𝑟𝑢𝑒 represents the choice probability computed with the synthetic

behavior. The higher value of relative likelihood (closer to 1) indicates higher predictive power. RL is

computed for each model over 25 sets of validation choice surveys sharing the same features as the

calibration sets in the number of alternatives per question, the number of questions per survey, and the

survey generation methods.

20

We also quantify the predictive power in the consideration decisions using “receiver operating

characteristics”. As shown in Table 5, a "consider" response matched with a "consider" prediction is a

true positive (TP), and matched with "reject" is a false negative (FN). A "reject" response matched

with a “reject” prediction is a true negative (TN), and matched with a "consider" prediction is a false

positive (FP). Accuracy is total correct predictions, i.e. TP+TN, which does not fully capture the

prediction performance of the model. Because respondents consider only a small fraction of profiles,

even predicting “reject all profiles” can yield high accuracy under this metric. Therefore, we also

measure the true positive rate (TPR), TP/(TP+FN), indicating the fraction of “consideration” in the

responses that can be correctly identified by the model; and the false positive rate (FPR), FP/(TN+FP),

indicating the fraction of “rejection” in the responses that the model mistakenly predicts “consider”.

Under these measures, a good model not only yields high accuracy but also high TPR and low FPR.

Note that the choice models, Models C and D do not make prediction about considerations, as they

simply assumed that all options are considered. Therefore Models C and D always have FN = 0 and

TN = 0, and thus TPR = FPR = 100%.

Table 5. Measures of the predictive power in consideration decisions Prediction

Response “Consider” “Reject”

“Consider” True Positive (TP) False Negative (FN)

“Reject” False Positive (FP) True Negative (TN)

Accuracy=TP+TN

True Positive Rate (TPR) = TP/(TP+FN)

False Positive Rate (FPR) = FP/(TN+FP)

Feature Commonality

The degree of commonality index (DCI) (Collier, 1981) quantifies the number of shared parts and

components among products to assist product line planning and assembling. This study quantifies the

number of shared features in a product line with the most basic formulation of DCI, which is

implemented in our case as:

𝐷𝐶𝐼 =∑ 𝜙𝑘

𝐽+𝑑𝑘=𝐽+1

𝑑

(10)

21

Where each product is labeled as 1, … , 𝐽 and distinct attribute levels are labeled as 𝐽 + 1, … , 𝐽 + 𝑑,

with 𝐽 as the number of products in the product line (corresponding to the end items in the original

definition), and 𝑑 is the number of distinct attribute levels (referred as component items in the original

definition). 𝜙𝑘 is the number of immediate parents that attribute level 𝑘 has. DCI indicates the number

of common products that share the same attribute level averaged over all distinct attribute levels. For

example, in Figure 4, cars 1 and 2 are standard sedans, one with 30 MPG and one with 35 MPG. Item

#3 and #5 each has one immediate parent item, end products #1 and #2, where Item #4 has two parents.

Thus, we have 𝜙3 = 1, 𝜙4 = 2, 𝜙5 = 1 with distinct feature number 𝑑 = 3. The DCI in this example

is (1+2+1)/3 = 1.33. A set of identical products has a DCI value of 𝐽, and totally different/unique

products have a value of 1.

Figure 4. A two-product example of computing degree of commonality index.

Profitability

We measure the profit predicted by the estimated models (“predicted profit”) and the profit

obtained by “actually offering” a chosen portfolio to the synthetic population (“actual profit”). Given

an optimal product line design, the predicted profit is computed as the objective function value in

Eqn.(8), with the modeled probabilities:

𝑃𝑟𝑒𝑑𝑖𝑐𝑡𝑒𝑑 𝑝𝑟𝑜𝑓𝑖𝑡 = ∑ ∑ 𝑃𝑖,𝑗𝑀𝑜𝑑𝑒𝑙(𝑝1

∗, ⋯ , 𝑝𝐽∗, 𝒚1

∗ , ⋯ , 𝒚𝐽∗)(𝑝𝑗

∗ − 𝑐𝑜𝑠𝑡(𝒚𝑗∗) )

𝐼

𝑖=1

𝐽

𝑗=0

(11)

22

where 𝑝1∗, ⋯ , 𝑝𝐽

∗ are the optimal prices of products 1 to J, 𝒚1∗ , ⋯ , 𝒚𝐽

∗ are the optimal non-price attributes

of product 1 to J. 𝑃𝑖,𝑗𝑀𝑜𝑑𝑒𝑙 is the individual i’s choice probability of product j, formulated from one of

the estimated models that used to solve for such optimal product line. The actual profit is computed

using the “true” behavior as:

𝐴𝑐𝑡𝑢𝑎𝑙 𝑝𝑟𝑜𝑓𝑖𝑡 = ∑ ∑ 𝑃𝑗𝑇𝑟𝑢𝑒(𝑝1

∗, ⋯ , 𝑝𝐽∗, 𝒚1

∗ , ⋯ , 𝒚𝐽∗)(𝑝𝑗

∗ − 𝑐𝑜𝑠𝑡(𝒚𝑗∗))

𝐼

𝑖=1

)

𝐽

𝑗=0

(12)

where 𝑃𝑖,𝑗𝑇𝑟𝑢𝑒 is the choice probabilities formulated from the synthetic behavior. Ideally, if a model

accurately predicts profits will have small discrepancy between the predicted profit and actual profit.

4. Results and Discussion

This section compares the predictive power, design commonality, and profitability of Models A –

D under the two synthetic true behaviors of Scenarios I and II.

4.1. Predictive Power

We examine the predictive power in the choice decisions in terms of relative likelihood. Note that

higher relative likelihood indicates higher predictive power, with perfect prediction at 1. Figure 5

shows the relative likelihood for the four estimated models across 25 validation choice surveys. In

Scenarios I and II (aspirational and sub-set conjunctive screening rules), the aspirational model (Model

B) has the highest predictive power. In scenario I, the conjunctive model (model A) predicts better

than the multinomial logit model (model C) and the latent class logit model (model D). In scenario II,

model A predicts worse than models C and D.

23

Figure 5. Predictive power measured by relative likelihood in 25 validation sets. The solid center bar

marks the mean over all estimates validated in all validation sets. The shaded boxes indicate the maximum

and minimum values.

To investigate the importance of predicting the consideration decisions in the final choice

prediction, we further look at the true positive rate (TPR) and false positive rate (FPR) defined in

Table 5. Perfect predictions of the consideration decisions yield TPR =100% and FPR = 0% at the

same time. Table 6 and 7 summarize the consideration prediction statistics of Models A and B.

Because models C and D assume respondents consider all profiles, they are unable to predict any

rejection responses. The aspirational model (Model B) has TPR = 91.7%, FPR = 5% in Scenario I, and

TPR = 77.3%, FPR = 10.2% in Scenario II, indicating that Model B has the best prediction of

consideration decisions among four models. Recall that the aspirational model does not have

information in the choice stage, but it still has high predictive power in the final choice as shown in

Fig.5.Predicting consideration accurately is sufficient to result in high predictive power in the final

choice for Model B. The conjunctive model (Model A) has low TPR and low FPR; it over-rejects

profiles, but still performs better than Models C and D in Scenario I, but contrariwise in Scenario II.

24

Table 6. Consideration prediction statistics Models A and B in Scenario I Model A. Conjunctive model Model B. Aspirational model

Prediction

Response “Consider” “Reject”

Prediction

Response “Consider” “Reject”

“Consider” 18.6% 17.0% “Consider” 32.6% 3.0%

“Reject” 8.9% 55.5% “Reject” 3.2% 61.2%

Accuracy = 74.0%

True Positive Rate = 52.0%

False Positive Rate = 13.9%

Accuracy = 93.8%

True Positive Rate = 91.7%

False Positive Rate = 5%

Table 7. Consideration prediction statistics of Models A and B in Scenario II Model A. Conjunctive model Model B. Aspirational model

Prediction

Response “Consider” “Reject”

Prediction

Response “Consider” “Reject”

“Consider” 13.3% 22.1% “Consider” 27.4% 8.0%

“Reject” 5.4% 59.2% “Reject” 6.6% 58.0%

Accuracy=72.5%

True Positive Rate = 37.6%

False Positive Rate = 8.4%

Accuracy=85.4%

True Positive Rate = 77.3%

False Positive Rate = 10.2%

4.2. Product Line Feature Commonality

We use the four estimated models A-D in a product line optimization, as introduced in Section 3.5,

and also optimized a line directly from the "true behavior" of the simulated customers, without an

intermediate choice model. Figure 6 shows the degree of commonality index (DCI), or amount of

feature-sharing within the resulting optimal product lines. In this simulation, a brand optimizes 10

products independently of any competitors’ product information. We performed optimization

separately for 21 brands. Accordingly, DCI was calculated for each brand over its 10 products. The

two consideration models A and B have DCIs close to the DCI of the true behavior. On average,

the two choice Models C and D have higher DCIs than the two consideration models, which

indicates that the product lines of the choice models are more likely to have features in common

than those of consideration.

25

Figure 6. The degree of commonality indices (DCI) of the optimal product lines of four models and the true behavior. The central bar represents the mean value of DCI across 21 brands. The shaded box represents the minimum and maximum values.

Next, we interpret the DCI with the representative optimal product lines of the brand Ford. Figures

7 and 8 show the prices, body styles, powertrains, fuel economy and the number of cylinders of the

optimal product lines. Consistent with the high DCI of multinomial logit and latent class logit models

(Models C and D), the product lines of these two models tend to share certain features. For example, in

Scenario I, observe the all-gasoline-powertrain line and the “4 cylinder engine & standard sedan”

combination in four vehicles from the multinomial model (see Model C in Figure 7), the “hybrid

powertrain, 8 cylinder engine & small SUV” combination in five vehicles from the latent class model

(see Model D in Figure 7).

Even though the conjunctive model and the aspirational model have DCIs close to that of the true

behavior optimization, their product lines are diverse in different manners from the true behavior. In

Scenario I, the optimal product line of the true behavior covers all body styles except hatchback and

pickup truck. This product line also offers various price points, mpg levels, and powertrains among the

SUVs (see true behavior in Figure 7). The optimal line of the conjunctive model focuses more on

sports cars and hatchbacks (see Model A in Figure 7). The aspirational model has the widest coverage

of body styles of any model (see Model B in Figure 7). In Scenario II, the optimal line of the true

behavior offers all body styles except the standard sedan. Two compact sedans at different price

26

points are offered, and two crossovers with different powertrains are offered (see true behavior

in Figure 8). The conjunctive model suggests a product line without a compact sedan, small SUV

or pickup truck and offered two designs in the sports car, hatchback, standard sedan, and

crossover body styles (see Model A in Figure 8). The aspirational model includes the all body

styles except small SUV and pickup truck, offering three different designs in full-size SUV (see

Model B in Figure 8).

Figure 7. The representative optimal product line of Ford in Scenario I.

27

Figure 8. The representative optimal product line of Ford in Scenario II.

4.3. Profitability

As defined by the profitability metric in Eqn.(12), we access the actual profits (i.e. the profit

obtained by “actually offering” a portfolio to the synthetic population). The y-axis in Figure 9 reports

the actual profits of four models and true behavior. In Scenario I, the aspirational model and the latent

class logit model are the best two models to achieve actual profits, while the multinomial logit model

has the lowest actual profit. In Scenario II, the multinomial logit model and latent class logit model

outperform the conjunctive model and aspirational model. In both Scenarios, all four models show

robustness in achieving at least 80% of the actual profits of the true behavior. The x-axis in Figure 9

represents the predicted profits (i.e. the profits predicted by the estimated models, as quantified in

Eqn.(11) ). In both scenarios, the multinomial logit model and latent class logit model predict that the

28

profits would be significantly higher than the aspirational model and the conjunctive model. The

conjunctive model has the lowest predicted profit among the four models.

By comparing the actual profits against the predicted profits, we can investigate the profit

prediction accuracy of four models. The diagonal line in Figure 9 represents 𝑝𝑟𝑒𝑑𝑖𝑐𝑡𝑒𝑑 𝑝𝑟𝑜𝑓𝑖𝑡 =

𝑎𝑐𝑡𝑢𝑎𝑙 𝑝𝑟𝑜𝑓𝑖𝑡. Any points on this line indicate perfectly accurate prediction, as the true behavior

shows in the figure. Any points above the line indicate under-prediction, i.e. 𝑝𝑟𝑒𝑑𝑖𝑐𝑡𝑒𝑑 𝑝𝑟𝑜𝑓𝑖𝑡 <

𝑎𝑐𝑡𝑢𝑎𝑙 𝑝𝑟𝑜𝑓𝑖𝑡; in contrast, any points below the line indicate over-prediction i.e. 𝑝𝑟𝑒𝑑𝑖𝑐𝑡𝑒𝑑 𝑝𝑟𝑜𝑓𝑖𝑡 >

𝑎𝑐𝑡𝑢𝑎𝑙 𝑝𝑟𝑜𝑓𝑖𝑡. From this visualization, we observe that the conjunctive model and aspirational model

give prediction closer to the actual profits than the multinomial logit and latent class logit model (as

show in the Figure 8, Model A and B are closer to the diagonal line than Model C and D). The

multinomial logit and latent class logit significantly over-predict their actual profits (Model C and D

display below and far from the accurate prediction line). Among four models, only conjunctive model

under-predicts the profits.

Note that the reported profits are subject to the limitation of the simplified cost model simulation

(see Section XX) and thus for the purpose of comparison only. All products were assigned a single

cost regardless their designs, leading to higher profits higher than reality.

29

Figure 9. Actual profit achieved by the optimal designs versus predicted profit, summed over 500 respondents, compared across four estimated models and true behavior. Circles represent mean values across 21 brands, and bars represent the range between minimum and maximum values.

5. Discussion

As summarized in Table 4, we generated a synthetic population who made “consider-then-

choose” decisions. We compared the uses of four consumer models (conjunctive, aspirational,

multinomial logit and latent class logit) and provided three metrics (predictive power, product

line feature commonality and profitability) to assess their performances. The assumptions behind

the models drove the survey experiments towards different methods and led the consumer to give

different aspects of responses. Thus, different models captured and delivered different information for

design optimizations, and eventually led to different design strategies. We discuss each model on the

connection between their specifications and their performances in design as the follows.

The conjunctive model (Model A in Table 4) estimated heterogeneous conjunctive screening rules

from each respondent via Bayesian adaptive “consider/reject” questions, specifying non-compensatory

“must-have” attribute levels for each respondent. Given these rules were heterogeneous, it is less likely

that products with attributes in common could satisfy a variety of individuals. This model trait was

apparent in the degree of commonality in the optimal product line, in that it has the lowest average

degree of commonality. A conjunctive screening rule is strict about “considering” a product, as all

attributes have to be acceptable for a product to be considered. Yet, in the true behavior, the

consideration decisions were relaxed – in Scenario I, a respondent considered a product as long as a

certain number of attributes are acceptable under the subset-conjunctive rule, and in Scenario II, a

respondent considered a product as long as the overall utility of a product exceeds some limit. Thus,

the conjunctive model under-predicted “consider” responses and profitability, as shown in the results

in Section 4.1 and 4.3.

The aspirational model (Model B) estimated heterogeneous aspirational screening rules from each

respondent via support vector machine. From Section 4.1, we observed that even without information

30

in the choice stage, the aspirational model yielded the highest predictive power in the choice responses.

The aspirational model also predicted profits more accurately than the multinomial logit and latent

class logit models. An interesting finding is that the use of support vector machine in Scenario I

resulted in good performance, even though respondents in Scenario I actually used subset-conjunctive

screening rather than aspirational screening. The explanation anchors in the mathematical formulation

of the separating hyperplane in Eqn.(3). Note that this formulation has similar structure as in the

subset-conjunctive screening rule in Eqn.(1). The values of the coefficients are 0 and 1 in the screening

rule vector, and some integer as the number of attributes. In Eqn.(3), the coefficients are defined in the

real number space, i.e. including the binary and integer space. Thus, the support vector machine that

uses formulation in Eqn.(3) is generalizable to estimate subset-conjunctive screening in Scenario I.

The multinomial logit model (Model C) assumed homogeneous utility and estimated homogenous

utility partworths for the population as a whole. Consistent with the homogeneity assumption, the

optimal product line suggested by multinomial logit tends to have more feature in common, resulting

in a higher DCI than Models A and B. Also, because the model does not estimate consideration, it

under-predicted product rejections, which corresponded to over-predicting the optimal profit of the

product line.

The latent class logit model (Model D) estimated latent-class-specific utility partworths by

aggregating choice responses from the whole population. This produced similar results to the

multinomial logit model, that is, a product line with high feature commonality and over-prediction of

profit. It should be noticed that latent class model embraced slightly higher heterogeneity than the

multinomial logit model. Correspondingly, the product line of latent class model had slightly lower

degree of commonality than the multinomial logit model.

The results of this study subject to some limitations. First, the screening rules in the synthetic

population do not reflect real-world screening rules. For example, the tolerance of the high price levels

in the synthetic population is higher than in real world. This leads to the result that the highest price

level dominates the optimal product line. Also, it is possible for screening rules to exist in certain

31

correlated patterns. For example, the consideration of a particular brand may correlate to the

consideration of certain body styles and price levels. No such correlations are included here.

Second, the optimal product line includes a simplistic engineering and cost model. More complex

engineering constraints and cost functions may reshape the optimal combinations of the attributes, as

the engineering constraints may reduce the feasible design space and the cost functions may counteract

the choice probability in some regions of the design space. Finally, a synthetic population, despite our

best attempts to base behavior from real-world data and introduce noise, will always behave slightly

differently than real respondents in the real world; this could affect findings.

6. Managerial Implications

This work was motivated by the fact that the field of consumer choice model selection has

plenty of options, but lacks of generalizable guidelines for selection of a particular model for a

particular application. The work presented here creates such guidelines for choice and

consideration models when applied to product optimization. A summary of recommendations is

presented in Table 8.

Table 8. Summary of recommendations

Model Type Characteristics Data Collection

Predictive Ability Optimz. Notes

Conjunctive

Capture heterogeneous “must-have” attribute levels in consideration decisions

Need to adaptively collect individual-specific data.

No known pre-packaged survey software available.

Requires significant computational power to ensure short generation time between survey questions.

Robust in predicting choices and profits.

Prevents over-prediction.

Generates diverse optimal products in a line.

Aspirational

Capture overall aspirational utility criteria in consideration decisions

Most accurate in predicting choice and profits.

Multinomial logit Capture homogeneous trade-offs in choice decision

Multiple pre-packaged survey software

options exist.

Requires less computational time.

Lower predictive power if consumers use sub-set conjunctive screenings.

Over-predicts profits.

Generates products with more features in common.

Latent class logit Capture trade-offs as sorted classes in choice decision

32

Before this work, the main indicator of model success was predictive power (for choice).

Here we demonstrate that accuracy in predictive power (for choice) does not necessarily

translate to accurately predicting profits. Over-predicting profit may be a crucial managerial

concern in model selection. Strategically speaking, over-predicting profits may be worse than under-

predicting profits: when realizing actual profits in a real market, over-predicted profits will appears as

“shortfalls” while under-predicted profits may appear as “windfalls”. Despite the robustness in

achieving close-to-ideal actual profits through optimal product line design, the choice models probably

provided the best lines over all in terms of profitability relative to the ideal but were the weakest

predictors of actual profits (Figure 9). That is, a designer using them would expect to obtain

significantly higher profit than might be actually realized, which may be viewed poorly by their

organization when the line is actually introduced and sold. In contrast, while the conjunctive model

may miss some highly profitable attribute combinations due to its simplified and strict screening rule

structure, it under-predicts profits. A designer using this model, while they may suggest strong

although not the most profitable lines, might appear prescient and be rewarded by their organization

for a “windfall”. Product line designers should balance the strengths and weakness of consideration

and choice models by, for example, valuing not only the compensatory trade-offs information during

profit optimizations, but also the non-compensatory screening information that might avoid over-

ambitious forecasts.

We also recommend that a firm needs to account for product line diversity when selecting models.

Product diversity can mitigate risk. If a prediction is inaccurate, focusing a line on too-close a set of

features, such as with a logit model, can lead to unforeseen failure. If the firm values risk mitigation,

then conjunctive or aspirational model, which suggests products with a larger variety of features,

should be used. Additionally, if a firm uses multinomial logit or latent class logit model, the

consequential product feature commonality may benefit the sharing of manufacturing platforms

between products, but may increase the chance of cannibalization within the product line.

33

The selection of model may also take into account the data and computation resources. If a

firm is limited by the computation resources and the ability to program its own surveys, a

multinomial logit or latentl class logit can be a convenient choice, since they enable the use of

pre-generated choice surveys and only take one estimation to infer the partworths from

aggregated choice responses. Collecting data for a conjunctive or aspirational model not only

requires an estimation to generate each question, but also requires that the computational time

between questions is short enough to avoid respondents’ waiting (especially when fielding

multiple responses at once). A firm should decide whether this more-involved data collect

procedure is worthwhile, considering if it will benefit from the predictive ability and optimal

product design benefits discussed above.

7. Conclusions

This study investigated the performance of consideration models and choice models in product

line design. Consideration models explicitly capture real-world decision processes where some “must-

have” product attributes (such as vehicle body types that have capacity for children) are non-

compensatory; while choice models assume trade-offs happen between all product attributes. Our

simulation estimated consideration models and choice models from adaptive consideration surveys and

discrete choice surveys, respectively. The research further applied the estimated models to a single-

firm product line optimization problem and compared the design outcomes along the metrics of

predictive power, feature commonality, and profitability.

We provided the insight that the use of consideration models not only cogently reflected real-

world decision process, but also changed design decisions. Capturing non-compensatory

heterogeneous screening rules resulted in offering more diverse optimal product lines overcoming the

weakness of over-predicting profits in the choice models, and slightly under-predicted potential profits.

We demonstrated that the assumptions behind the consumer models permeated the survey

experiment and the estimation, eventually flowed into the design optimization. An important lesson for

the designers and managers is that even given the same target population, one will end up with

34

different design strategies that depend on analysis approaches. It is also important for designers to

assess the performances of consumer models in different managerially relevant angles, such as the

diversity of product line and the accuracy of predicting profit.

References Abramson, C., Andrews, R. L., Currim, I. S., & Jones, M. (2000). Parameter bias from

unobserved effects in the multinomial logit model of consumer choice. Journal of Marketing Research, XXXVII(November), 410–426.

Andrews, R. L., Ainslie, A., & Currim, I. S. (2008). On the recoverability of choice behaviors with random coefficients choice models in the context of limited data and unobserved effects. Management Science, 54(1), 83–99. doi:10.1287/mnsc.1070.0749

Andrews, R. L., & Manrai, A. K. (1998). Simulation experiments in choice simplification: The effects of task and context on forecasting performance. Journal of Marketing Research, 35(2), 198–209.

Belloni, A., Freund, R., Selove, M., & Simester, D. (2008). Optimizing Product Line Designs: Efficient Methods and Comparisons. Management Science, 54(9), 1544–1552. doi:10.1287/mnsc.1080.0864

Ben-Akiva, M., & Lerman, S. (1985). Discrete Choice Analysis: Theory and Applcation to Predict Travel Demand. The MIT press.

Besharati, B., Azarm, S., & Kannan, P. K. (2006). A decision support system for product design selection: A generalized purchase modeling approach. Decision Support Systems, 42(1), 333–350. doi:10.1016/j.dss.2005.01.002

Besharati, B., Luo, L., Azarm, S., & Kannan, P. K. (2006). Multi-Objective Single Product Robust Optimization: An Integrated Design and Marketing Approach. Journal of Mechanical Design, 128(4), 884. doi:10.1115/1.2202889

Bettman, J. R., Luce, M. F., & Payne, J. W. (1998). Constructive Consumer Choice Processes, 25(December).

Bishop, C. M. (2006). Pattern Recognition and Machine Learning. New York: Springer. Boser, B. E., Guyon, I. M., & Vapnik, V. N. (1992). A Training Algorithm for Optimal Margin

Classifiers. In Proceedings of the Fifth Annual ACM Workshop on Computational Learning Theory (pp. 144–152). doi:10.1.1.21.3818

Bröder, A. (2000). Assessing the empirical validity of the “Take-the-best” heuristic as a model of human probabilistic inference. Journal of Experimental Psychology: Learning, Memory, and Cognition, 26(5), 1332–1346.

Chen, K. D., & Hausman, W. H. (2000). Technical Note: Mathematical Properties of the Optimal Product Line Selection Problem Using Choice-Based Conjoint Analysis. Management Science, 46(2), 327–332. doi:10.1287/mnsc.46.2.327.11931

Chiang, J., Chib, S., & Narasimhan, C. (1999). Markov chain Monte Carlo and models of consideration set and parameter heterogeneity. Journal of Econometrics, 89(1), 223–248.

Collier, D. A. (1981). The measurement and operating benefits of component part commonality. Decision Sciences, 12(1), 85–96.

Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273–

35

297. doi:10.1007/BF00994018 Ding, M., Hauser, J. R., Dong, S., Dzyabura, D., Yang, Z., Su, C., & Gaskin, S. P. (2011).

Unstructured direct elicitation of decision rules. Journal of Marketing Research, 48(1), 116–127.

Dzyabura, D., & Hauser, J. R. (2011). Active machine learning for consideration heuristics. Marketing Science, 30(5), 801–819. doi:10.1287/mksc.1110.0660

Gen, M., & Cheng, R. (2000). Genetic algorithms and engineering optimization. Gigerenzer, G., & Gaissmaier, W. (2011). Heuristic decision making. Annual Review of

Psychology, 62, 451–482. Gilbride, T. J., & Allenby, G. M. (2004). A choice model with conjunctive, disjunctive, and

compensatory screening rules. Marketing Science, 23(3), 391–406. Greene, D. L. (2001). TAFV alternative fuels and vehicles choice model documentation. Oak

Ridge: Oak Ridge National Laboratory, Center. Greene, W. H., & Hensher, D. A. (2003). A latent class model for discrete choice analysis:

contrasts with mixed logit. Transportation Research Part B: Methodological, 37(8), 681–698. doi:10.1016/S0191-2615(02)00046-2

Hanson, W., & Martin, K. (1996). Optimizing mulitnomial logit profit functions. Management Science, 42(7), 992–1003.

Hauser, J. R. (1978). Testing the accuracy , usefulness , and significance of probabilistic choice models: An information-theoretic approach. Operations Research, 26(3), 406–421.

Hauser, J. R., Ding, M., & Gaskin, S. P. (2009). Non-compensatory (and compensatory) models of consideration-set decisions. In Proceedings of the Sawtooth Software Conference. Delray Beach, FL.

Hauser, J. R., Toubia, O., Evgeniou, T., Befurt, R., & Dzyabura, D. (2010). Disjunctions of conjunctions, cognitive simplicity, and consideration sets. Journal of Marketing Research, 47(3), 485–496.

Hauser, J. R., & Wernerfelt, B. (1990). An Evaluation Cost Model of Consideration Sets. Journal of Consumer Research, 16(4), 393–408. doi:10.1086/209225

Hoyle, C., Chen, W., Wang, N., & Koppelman, F. S. (2010). Integrated Bayesian Hierarchical Choice Modeling to Capture Heterogeneous Consumer Preferences in Engineering Design. Journal of Mechanical Design, 132(12), 121010.

Huang, D., & Luo, L. (2015). Consumer preference elicitation of complex products using fuzzy support vector machine active learning. Forthcoming: Marketing Science, (Special Issue: “Big Data”).

Huber, J., & Zwerina, K. (1996). The importance of utility balance in efficient choice designs. Journal of Marketing Research, 33(3), 307–317.

Jedidi, K., & Kohli, R. (2005). Probabilistic subset-conjunctive models for heterogeneous consumers. Journal of Marketing Research, 42(4), 483–494.

Jedidi, K., Kohli, R., & DeSarbo, W. S. (1996). Consideration sets in conjoint analysis. Journal of Marketing Research, 33(3), 364–372.

Johnson, F. R., Lancsar, E., Marshall, D., Kilambi, V., Mühlbacher, A., Regier, D. A., … Bridges, J. F. P. (2013). Constructing Experimental Designs for Discrete-Choice Experiments: Report of the ISPOR Conjoint Analysis Experimental Design Good Research Practices Task Force. Value in Health, 16(1), 3–13. doi:10.1016/j.jval.2012.08.2223

36

Kohli, R., & Jedidi, K. (2007). Representation and inference of lexicographic preference models and their variants. Marketing Science, 26(3), 380–399.

Kohli, R., & Sukumar, R. (1990). Heuristics for Product-Line Selection Using Conjoint Analysis. Management Science, 36(12), 1464–1478. doi:10.1287/mnsc.36.12.1464

Kuhfeld, W. F. (2010). Marketing Research Methods in SAS. Kullback, S., & Leibler, R. A. (1951). On information and sufficiency. The Annals of

Mathematical Statistics, 22(1), 79–86. Liu, L., & Dukes, A. (2013). Consideration Set Formation with Multiproduct Firms : The

Case of Within-Firm and Across-Firm Evaluation Costs. Management Science, 59(8), 1871–1886. doi:10.1287/mnsc.1120.1659

Long, M., & Morrow, W. R. (2015). Should optimal designers worry about consideration? Journal of Mechanical Design, 137(July), 071411–1–16. doi:10.1115/1.4030178

Luo, L. (2010). Product line design for consumer durables: an integrated marketing and engineering approach. Journal of Marketing Research, Forthcoming, XLVIII(February), 128–139. Retrieved from http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1717940

Macdonald, E., Whitefoot, K., Allison, J. T., Y., P. P., & Gonzalez, R. (2010). An investigation of sustainability, preference, and profitability in design optimization. In Proceedings of the ASME 2010 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference (pp. 1–14). Montréal, Quebec, Canada.

Manrai, A. K., & Andrews, R. L. (1998). Two-stage discrete choice models for scanner panel data: An assessment of process and assumptions. European Journal of Operational Research, 111(2), 193–215. doi:10.1016/S0377-2217(98)00145-3

Martignon, L., & Hoffrage, U. (2002). Fast, frugal, and fit: Simple heuristics for paired comparison. Theory and Decision, 52, 29–71.

McFadden, D. (1974). The measurement of urban travel demand. Journal of Public Economics, 3, 303–328.

Michalek, J. J., Papalambros, P. Y., & Skerlos, S. J. (2004). A Study of Fuel Efficiency and Emission Policy Impact on Optimal Vehicle Design Decisions. Journal of Mechanical Design, 126(6), 1062.

Mitchell, M. (1998). An introduction to genetic algorithms. MIT press. Morrow, W. R., & Skerlos, S. J. (2011). Fixed-Point Approaches to Computing Bertrand-

Nash Equilibrium Prices Under Mixed-Logit Demand. Operations Research, 59(2), 328–345.

Payne, J. W. (1976). Task complexity and contingent processing in decision making: An information search and protocol analysis. Organizational Behavior and Human Performance, 16(2), 366–387.

Rieskamp, J., & Hoffrage, U. (2008). Inferences under time pressure: how opportunity costs affect strategy selection. Acta Psychologica, 127(2), 258–276.

Roberts, J. H., & Lattin, J. M. (1997). Consideration: review of research and prospects for future insights. Journal of Marketing Research, 34(3), 406–410.

Sawtooth. (2013). The CBC system for choice-based conjoint analysis. Shin, J., & Ferguson, S. (2015). Modeling noncompensatory choices with a compensatory

model for a product design search. In Proceedings of the ASME 2015 International Design Engineering Technical Conferences & Computers and Information in

37

Engineering Conference (pp. 1–13). Boston. Siddarth, S., Bucklin, R. E., & Morrison, D. G. (1995). Making the cut: Modeling and

analyzing choice set restriction in scanner panel data. Journal of Marketing Research, 32(3), 255–266.

Simon, H. A. (1956). Rational choice and the structure of the environment. Psychological Review, 63(2), 129–38. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/13310708

Simon, H. A. (1972). Theories of bounded rationality. In Decision and organization (Vol. 1, pp. 161–176). North-Holland Publishing Company.

Swait, J. (1994). A structural equation model of latent segmentation and product choice for cross-sectional revealed preference choice data. Journal of Retailing and Consumer Services, 1(2), 77–89.

Swait, J. (2001a). A non-compensatory choice model incorporating attribute cutoffs. Transportation Research Part B: Methodological, 35(10), 903–928.

Swait, J. (2001b). Choice set generation within the generalized extreme value family of discrete choice models. Transportation Research Part B: Methodological, 35(7), 643–666. doi:10.1016/S0191-2615(00)00029-1

Train, K. E. (2009). Discrete Choice Methods with Simulation (2nd ed.). Cambridge university press.

Tversky, A. (1972). Elimination by aspects: A theory of choice. Psychological Revieweview, 79(4), 281–299.

Vapnik, V., Golowich, S. E., & Smola, A. (1996). Support vector method for function approximation, regression estimation, and signal processing. Annual Conference on Neural Information Processing Systems (NIPS), 281–287. doi:10.1007/978-3-642-33311-8_5

Wang, M., & Chen, W. (2015). A Data-Driven Network Analysis Approach to Predicting Customer Choice Sets for Choice Modeling in Engineering Design. Journal of Mechanical Design, 137(July), 1–11. doi:10.1115/1.4030160

Wassenaar, H. J., & Chen, W. (2003). An Approach to Decision-Based Design With Discrete Choice Analysis for Demand Modeling. Journal of Mechanical Design, 125(3), 490–497.

Yee, M., Dahan, E., Hauser, J. R., & Orlin, J. (2007). Greedoid-based noncompensatory inference. Marketing Science, 26(4), 532–549.

Nomenclature 𝒙𝑗 Feature vector of a product profile 𝑗

𝐶𝑖(. ) Consideration set of individual 𝑖 𝒗 Partworths in the aspirational screening model

𝛾 Aspiration limit

𝜹𝑖 Heterogeneous binary aspect-coded subset conjunctive rules for

individual 𝑖 𝐾 The least number of acceptable attributes a profile has to have to

be considered in the subset conjunctive models

𝜷𝑖 Heterogeneous partworths of the logit behavior for individual 𝑖 in

the synthetic population

𝜖𝑖,𝑘 Random term of the logit behavior of alternative 𝑘 for individual 𝑖

38

in the synthetic population

𝑃𝑗𝑀𝑁𝐿 Choice probability of product 𝑗 in multinomial logit model

𝑃𝑗|𝑞𝐿𝐶𝐿 Choice probability of product 𝑗 conditional on class 𝑞 in the latent

class model

𝑃𝑗𝐿𝐶𝐿 Choice probability of product 𝑗 in the latent class model

𝛼𝑞 Probability of latent class 𝑞

ℎ(. ) Support vector machine decision hyperplane function

𝑦𝑛 Response to profile 𝑛, 1 indicates “consider” and -1 indicates

“reject”

𝒑𝑗 Binary coded price level vector of product 𝑗

�̅�𝑗 Numeric value of price mapped from 𝒑𝑗

𝒚𝑗 Binary coded non-price features vector of product 𝑗

𝜙𝑘 The number of immediate parents that feature level 𝑘 has

𝑑 The number of distinct feature levels