How to unlock the value hidden away in your customer databases using

statistical segmentation methods

Nigel Marriott, MSc, CStat, CSci Statistical Consultant

www.marriott-stats.com

May 2014

• Traditional customer segmentation is often based around WHO the customer is e.g. demographics, ACORN, etc.

• Of far greater value are segments based on what customers are DOING i.e. their behaviour.

• To do this, we need data that answers these questions: 1. WHAT are our customers buying? i.e. product info

2. WHEN are our customers buying? i.e. date & time info

3. WHERE are our customers buying? i.e. location info, online/offline, etc

4. HOW are our customers buying? e.g. payment method, usage of offers, etc

• Many organisations have databases with such behavioural data.

• The key is being able to link each transaction to a specific customer. – This tends to be online/mail order/phone order based retailers.

– Can be high street retailers via high usage of loyalty cards or other identifier.

Database Value is about Customer Behaviour

www.marriott-stats.com Statistical Segmentation – ExIStA May 2014

• Using multivariate statistical methods, I can allocate each customer to a SEGMENT which describes a particular kind of behaviour.

• Subsequently (if desired) I can then analyse WHO will behave in these ways based on their demographics.

• In effect, traditional methods focus on P( Behaviour | Who )

• I argue that the focus should be on P( Who | Behaviour ).

• In fact the two can be combined using Bayes Rule i.e.

P( Behaviour | Who ) = K x P( Who | Behaviour ) x P( Behaviour )

• We can then extend our list of questions from the previous slide. 5. WHO are our customers? May need a survey to answer this.

6. WHY are our customers buying? Definitely needs a survey to answer this!

Statistical Behavioural Segmentation is…

www.marriott-stats.com Statistical Segmentation – ExIStA May 2014

1. 3 Examples of customer database segmentation I did in 2013. a. Supermarket Online Shopping Channel

b. Online Clothing retailer

c. Online Pet Food & Medications retailer

2. What kind of behavioural data might you find in customer databases? a. WHAT are our customers buying?

b. WHEN are our customers buying?

c. WHERE are our customers buying?

d. HOW are our customers buying?

3. Which statistical techniques can we use to segment customers?

What will I talk about today

www.marriott-stats.com Statistical Segmentation – ExIStA May 2014

3 Examples from 2013

www.marriott-stats.com Statistical Segmentation – ExIStA May 2014

Pet Food & Medications Retailer

The 6 segments shown here have significant implications for corporate strategy. 1. Nearly half of all sales comes from just 1 in 8 customers who place on average 7 orders a

year compared to less than 2 for most other segments.

2. The Bulk Food segment requires handling of large bulky items which has implications for shipping costs and warehouse layouts.

3. The largest segment in terms of customers (Non-Standard Orders) are far less likely to use the website & credit cards. The cost of servicing such customers must be high but they do buy more Horse products which is a rapidly growing market.

www.marriott-stats.com Statistical Segmentation – ExIStA May 2014

On-Line Clothing Retailer

1. Only 1 in 7 customers effectively order more than once a year but account for 40%+ of sales 2. Growth is coming from Own Label and Larger Size Brands shoppers but Own Label shoppers

significantly less likely to use a website which has cost implications. 3. Other Branded Products are in serious decline. 4. Email campaigns are an effective source of orders. 5. 11% of customers are making very high rate of returns. Could be “Wardrobers”?

www.marriott-stats.com Statistical Segmentation – ExIStA May 2014

Supermarket On-Line Shopping Channel

www.marriott-stats.com Statistical Segmentation – ExIStA May 2014

What kind of Behavioural Data can be found in Customer Databases?

www.marriott-stats.com Statistical Segmentation – ExIStA May 2014

Use the 4 Questions to identify suitable data 1. WHAT are our customers buying? 2. WHEN are our customers buying? 3. WHERE are our customers buying? 4. HOW are our customers buying?

By keeping these questions in mind, you will be able to identify suitable fields. It doesn’t matter if you can’t match a field to a question, the point of the Qs is prompt creative thinking. Remember that many derived calculations are possible. For example, knowledge of the date of each order per customer allows you to derive the 5 WHEN fields shown. Some fields will specific to each customer. Often these are WHAT fields relating to the products that are purchased. Do NOT include WHO & WHY fields! These are for secondary analysis, not for segmentation.

www.marriott-stats.com Statistical Segmentation – ExIStA May 2014

Statistical Techniques

www.marriott-stats.com Statistical Segmentation – ExIStA May 2014

• The choice of technique depends first on whether data is all numerical or whether some of the fields are categorical. – Remember that any binary categorical field can be made numerical by

replacing the categories with 0 & 1.

– Ordinal categories can be replaced with numbers subject to the usual caveats regarding ordinal data.

• If your data is all numerical then these techniques are suitable. – AHC (agglomerative hierarchal clustering)

– K-Means Clustering

– MDS (Multi-Dimensional Scaling)

– PCA (Principal Components Analysis)

• If your data contains some categorical fields, then my preference is: – MCA (Multiple Correspondence Analysis)

Statistical Techniques for Segmentation

www.marriott-stats.com Statistical Segmentation – ExIStA May 2014

• Distance Metric: This measures how different each customer is from each other in terms of their behaviour. – Key to AHC, MDS & K-Means Clustering.

– Euclidean Distance is the most common choice but don’t ignore other possibilities.

• Standardisation of Variables: Distance metrics & covariance matrices (in PCA) are sensitive to your choice of scales. Do you want this to be the case?

• Manual Clustering: This is often overlooked. Sometimes it is obvious from the data how customers should be segmented or more meaningful to override the results of your chosen method.

• Aggregation of Categories: If the # of customers that fall in a certain category is low, may be worth combining category with another.

Don’t Forget These Issues!

www.marriott-stats.com Statistical Segmentation – ExIStA May 2014

AHC in a Nutshell The segments in the 3 examples shown in this presentation were all built using AHC which is probably the most common statistical technique used. The output is known as a DENDROGRAM.

www.marriott-stats.com Statistical Segmentation – ExIStA May 2014

5 Customer

Cohorts Found

Customer Segmentation Dendrogram

A B D E C

Think of this as an upside down tree with the leaves at the bottom. Think of the customers as ants who are living on this tree. Customers who share the same behaviour will live on the same leaf. Customers who are similar in behaviour will live on neighbouring leaves. Customers who differ in behaviour significantly will live on opposite sides of the tree. The distance between leaves is measured by the length & number of branches it takes for an ant to walk from one leaf to the other. By cutting the tree at a major branch, we will be left with a cluster of leaves which we call a SEGMENT.

Example: Pet Food & Medications Retailer

www.marriott-stats.com Statistical Segmentation – ExIStA May 2014

I selected a random sample of 50,000 customers and split these into 5 random sub samples. The 5 dendrograms all gave a 6 cluster solution at the same height (branch length). The size & shapes of each cluster is roughly similar in each dendrogram but not perfect.

Follow up analysis of the means of each variable for each segment showed that a 6 segment solution was meaningful.

• Remember that your client needs to be able to understand & use your segmentation. – Too many segments will be difficult to handle .

– Too few will limit the potential insight.

– In my experience 4 to 12 segments is optimal.

• Use the sub-sample Dendrograms to find the probable # of segments but look at using more segments as well. – E.g. the previous slide suggested a 6 cluster solution which is what I settled on

but I also explored a 10 cluster solution.

– Analysis of the means for the 10 cluster solution showed that with 1 exception, there was no real need to split the 6 clusters further.

• Keep an eye out for “uninteresting” clusters. – In my experience, this is more a problem with satisfaction surveys rather than

customer databases where people tend to give the same response to all Qs.

How Many Segments?

www.marriott-stats.com Statistical Segmentation – ExIStA May 2014

• Once you have decided on your segmentation, you may need to develop rules for allocating existing & new customers to 1 of your N segments. – Generally not necessary if your analysis is primarily to inform strategy.

– Essential if your segmentation will be used for planning purposes.

• Classification rules need to be easily understood by non-statisticians who will have to implement. Therefore do not be afraid of developing a manual solution that mimics your statistical segmentation.

Develop Your Classification Rules

www.marriott-stats.com Statistical Segmentation – ExIStA May 2014

Key Points

www.marriott-stats.com Statistical Segmentation – ExIStA May 2014

• Any organisation that can link their transactions with an individual person (or some other discrete unit) can segment their customers based on what they are doing rather than who they are.

• Use the 4 questions (WHAT, WHEN, WHERE & HOW) to identify & derive the behavioural fields in your database.

• In most cases, your behavioural data set will be numerical in nature which gives you a lot of choice in terms of statistical methodology. – Don’t blindly use the default option or what has been used before!

– Always think through the details such as distance metrics, etc.

• Ensure that your segmentation can be used by your clients! – Identify the optimal # segments for strategic & operational planning purposes.

– Are the classification rules easily understood by the client?

– Don’t be afraid to manually override your results to ensure usability.

Customer Databases are there to be Tapped!

www.marriott-stats.com Statistical Segmentation – ExIStA May 2014