Copyright © 2010 Pearson Education, Inc.
Publishing as Prentice Hall
Conducting Research and Utilizing Tests to Measure Performance
Chapter 10
Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall
Please Note:
Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall
Opening Vignette
Omaha Steaks
International
Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall
Research Serves Direct Marketers
Fact-Finding Information Gathering Problem-Solving Decision-Making
Need for Marketing Research Managers must have good, accurate,
timely information with which to make decisions.
Marketing research helps to gather the needed information.
The results of research can be quantitative and/or qualitative.
Valid research measures results … not opinions.
Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall
Problem Structure How much advertising is needed? How will the direct marketing mix
be selected? How will our resources be utilized?
Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall
The Nature of ResearchSurveys vs. Experiments:
A survey looks at things the way they are to measure product preferences or determine future buying intentions
An experiment or test measures the effect of change
Which one is used more widely by direct marketers?
Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall
Databased Research and Analysis“My mind’s made up. Don’t confuse me with
facts.”
A characteristic of database-driven and directed marketing is measurement and accountability for actions. Decisions are based on facts, not opinions.
Direct marketers build databases from facts, relying not so much on responses derived from survey, but more on conclusions derived from experimentation.
Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall
Products and Services Media Offers/Propositions Copy Platforms Creative Formats Timing/Seasonality
What Do Direct Marketers Test?
Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall
TEST THE BIG THINGS
Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall
Sourcing & Collecting Information
Secondary Data
Primary Data
Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall
Secondary Data Sources The Organization’s Internal Records Government Sources: Federal, State & Local
U.S. Department of Commerce - Bureau of the Census U.D. Department of Labor - Bureau of Labor Statistics U.S. Department of Agriculture Other U.S. Government Sources: President’s Office,
Congress, Treasury Department, Interior Department, and Health and Human Services Department
State and Local Governments: economic surveys, license registrations, and tax records
Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall
Secondary Data Sources Trade, Technical, Professional and
Business Associations Private Research Organizations Foundations, Universities and Other
Nonprofits Libraries, Public and Private Advertising Media Financial Institutions and Utilities
Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall
Behavior
Intentions
Knowledge
Socioeconomic Status
Attitudes and Opinions
Motivations
Psychological Traits
Primary data, collected via survey, can yield information about:
Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall
PRIMARY DATA COLLECTION METHODSSurvey Method Advantages Disadvantages
Personal Interview More complete and accurate sample, more complete information, greater flexibility in structuring questions, high response rate
Cost, possibility of interview bias, and need for extensive interviewer supervision and control
Telephone Interview
Economy, speed, representative sampling, minimal nonresponse, simple callbacks, ability to make interview coincide with other activities
Limited availability of information at time of contact, excludes those without phone service, increasing public resistance to phone calls, inability to use questions requiring demonstration or visualization.
E-Mail Survey Ability to reach hard-to-reach consumer segments, speed, economy, respondent choice in when to reply
Representative sample issues, higher degrees of nonresponse bias
Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall
PRIMARY DATA COLLECTION METHODS
Survey Method Advantages Disadvantages
Mail Questionnaire Great versatility, low cost, no interviewer bias, no field staff needed, respondent choice in when to reply, respondent confidentiality
High rate of nonresponse, need for follow-up, response bias factors
On-line Surveys Ability to reach potential respondents at most opportune time, speed and convenience
Similar to e-mail surveys - Representative sample issues, higher degrees of nonresponse bias
Observation Removes respondent bias More costly and time consuming than some of the other methods
Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall
How to Design An Experiment
Control
Randomization
Statistically-valid sample size
Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall
How to Track Responses
Key Codes
Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall
Response rate & break-even analysis
Control vs. experimental packages
Direct marketers test or experiment with different offers and campaign themes to determine which one generates the greatest response rate
Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall
Promotion Cost
------------------------ = Break-even Number of Sales
Unit Profit per Sale
Break-even analysis
Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall
Samples & Estimations Random sample designs
Simple Random Samples Systematic Random Samples Stratified Random Samples Cluster Samples Replicated Samples Sequential Samples
Determination of sample size
Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall
Using Statistical Evaluation to Determine the Sample Size
Confidence level - the number of standard deviations from the mean in a normal distribution (for example a 95% confidence levels corresponds to 1.96 standard deviations from the mean)
Limit of error - the number of percentage points by which the researcher can miscalculate the actual response rate
Expected (actual) response rate - the number of positive responses expressed as a percentage of the total
Sample size - the number of observations in an experiment
Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall
Measurement of Differences
Hypothesis testing
Types of errors in hypothesis testing
Statistical evaluation of differences
Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall
Hypothesis TestingHypotheses are typically stated in negative terms; that
is, a null hypothesis (H0) versus an alternative hypothesis (Ha) in a form such as the following:
H0: Direct mail response from the test promotion is at or below direct mail response from the control promotion.
Ha: Direct mail response from the test promotion is above direct mad response from the control promotion.
The null hypothesis, then, states that direct mail response will not be better than the control. Our measurement sets out to disprove this null hypothesis.
Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall
Types of Errorin Hypothesis TestingType One: Results when the decision-maker rejects the
null hypothesis even though it is, in fact, true ... i.e., taking an action when one shouldn't
Type Two: Results when the decision-maker accepts the
null hypothesis when, in fact, not true ... i.e., not taking an action when one should.
Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall
Assume that a sample has been properly selected and is of an adequate size. Assume further that an experiment has been designed and implemented in a valid manner. It now remains for the direct marketer to be able to recognize the difference in the response rate from a test and that from a control, with some degree of confidence and within an acceptable limit of error.
Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall
Test Control Totals Response A C A + C Non-response B D B + D Total mailed A + B C + D A + B + C + D = N
The statistic 2 is computed as follows:2 = N [ (A x D) - (C x B) - N/2]2
(A+B) x (C+D) x (A+C) x (B+D) Here is a sample calculation:
Test Control Totals Response 200 100 300 Non-response 800 900 1700 Total mailed 1000 1000 2000
2 = 2,000 x [ 180,000 - 80,000 - 1,000]2
1,000 x 1,000 x 300 x 1,700
2 = 38.4 ... which is significant at the 99++% level since it exceeds the critical value in the X2 table for one degree of freedom for a significance level of 0.001, given as 10.83
Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall
STRUCTURING and EVALUATING AN EXPERIMENT
State the hypothesis Develop, by a priori analysis, the
assumptions required and compute the appropriate sample size
Structure and perform the experiment Develop, by a posteriori analysis,
statistics for judging hypothesis validity
Make the decision
Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall
CHAPTER CASE
THE AMERICAN HEART
ASSOCIATION
Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall