Post on 14-Aug-2015
transcript
Merchandise Planning
Lessons learnt in real world applications, examples shown only (data made up).
This presentation covers:
• Value in accurate planning
• Real world data challenges
• The approach
• The technology
• The process
Ben Post Analytics Client Success Professional
Version 1, 23rd June 2015
What’s not covered
Affinity analysis and Lift analytics to optimise merchandise assortment, pricing, placement and promotions. 2 min video on that here: https://www.youtube.com/watch?v=XEe7fquFvBI
Smarter Merchandise Planning Benefits
Reduce lost sales
due to stock outs
Maximise Inventory Turnover
Reduce write-downs
Store/department level participation
Less time and hassle spent preparing a forecast
Model with all important effects from social to weather
The Forecasting Challenge
Time series techniques such as ARIMA work well with fast moving items that have clear seasonal and trend patterns BUT most SKUs in large retail are relatively slow moving.
300,000+ SKUs Binned by TRX_Count and Total Sales Revenue
High Volume Items (easier to forecast)
Low Volume Items(harder to forecast)
This means your series are often sparse and noisy
Sparsity = lots of zeros in your series
Window is the time period that you aggregate POS data to e.g. Week
Common time series models like ARIMA or Holt-winters need nicely prepared baselines (>3 cycles of history free of noise).
Time series models propagate error so are best for short term. Merchandise Planning requires short to medium term due to supply chain constraints!
Holt-Winters model types and features
Time series models often miss due to real world factors
Causation: for two events c (cause) and e (effect), c causes e if (1) c and e both occur and (2) if c had not occurred and all else remained the same, then e would not have occurred
Trend + Seasonality + Employee but, sometimes a miss Factors
Can we spot the residuals and, determine causation Trend + Seasonal + State Space
What factors typically affect demand?
1. Is it on the shelf? Shrinkage, smart rolling stock checks
2. Price, Promotion, Placement
3. Staff Coverage
4. Availability and price of substitute items in store
5. Competing retailer/online offers
6. Emerging trends (social media analysis)
7. Seasonal & Events
8. Econometric Factors
9. Weather
The Forecasting Challenge
Techniques like Dynamic Linear models using key factors can outperform common time series techniques for most SKUs. i.e. value in sophisticated forecasting techniques (less error)
Each combination of SKU and location will require a model for accuracy. That translates into a lot of compute for large retailers requiring a scalable solution.
The SPSS platform and an appropriate database such as Netezza or a SPARK cluster delivers the ease of use and speed to deliver results.
Merchandise Planning Challenges and the TM1 Engine
Large dimensions e.g. Product can be in excess off 1M elements. More than a spreadsheet can handle and relational databases don’t aggregate well.
Calculating and analysing sales plans is a multi-dimensional problem. SKU by Time window by Scenario by your measures. Spreadsheets are two dimensional (rows and columns) and relational databases are not modeling tools.
Collaboration with store managers and merchandisers requires enterprise planning capabilities. Spreadsheets are single user desktop tools and relational databases are just databases.
IBM Cognos TM1 is the leading OLAP (cube) based business modeling tool. It is used by many leading retailers for Merchandise Planning and Analysis. It is scalable & fast.
Merchandise Planning Process (method)
Summarise POS• Aggregate by SKU, day of week and
week and store• Variance analysis forecast vs actual –
top 50
Classify Series• Volume/Seasonal• Slow moving• New like existing• Substitutes (rules & adjustments)• New line (rules & adjustments)
Plans & Assumptions• Inventory (receipts & balances)• Events calendar• Promotions• Weather• Economic • Staff
Baseline • Remove the effects of promotions • Impute probable demand for stock
outs (lost sales) or exclude these periods from training set
• Remove anomalies e.g. large one off orders from baseline
• Targeted rolling stock take
Forecast• High turnover items e.g. ARIMA• Low turnover lines e.g. Dynamic
Linear Models
Adjust, Agree and Execute• Store/Department review and adjust• Purchasing review supplier lead times• Purchasing review and adjust• Generate purchase orders and reports
Weather and Events
Include nulls as 0s (sparse)export 7 day fields for each week and factor
IBM TM1 Concert - guided merchandise planning process/tasks with social
https://www.youtube.com/watch?v=KqRBS6p2KwI3min demonstration of product features
Summarise POS
Sample 52 week
TRX Count
Demonstrate Analysis speed with Cognos Analysis for Excel over TM1
Accuracy
TM1 CAFÉ report highlights the largest
misses for investigation as top and bottom 10 Actual – Forecast $
Merchandise Planning Process (method)
Summarise POS• Aggregate by SKU, day of week and
week and store• Variance analysis forecast vs actual –
top 50
Classify Series• Volume/Seasonal• Slow moving• New like existing• Substitutes (rules & adjustments)• New line (rules & adjustments)
Plans & Assumptions• Inventory (receipts & balances)• Events calendar• Promotions• Weather• Economic • Staff
Baseline • Remove the effects of promotions • Impute probable demand for stock
outs (lost sales) or exclude these periods from training set
• Remove anomalies e.g. large one off orders from baseline
• Targeted rolling stock take
Forecast• High turnover items e.g. ARIMA• Low turnover lines e.g. Dynamic
Linear Models
Adjust, Agree and Execute• Store/Department review and adjust• Purchasing review supplier lead times• Purchasing review and adjust• Generate purchase orders and reports
Weather and Events
Include nulls as 0s (sparse)
Classify series for forecasting
TM1 calculates statistics on each series like when it was last sold, # of weeks it sold in the last year and avg trx count
TM1 classifies the series based on these properties (which you can override) and applies a default forecast method/profile (which you can override)
Demonstrate example using spread profiles to enter a forecast (smartco)
Note this example is simplified; there are other attributes which feed into the inventory planning such as shipping costs, minimum order quantities etc. used in the rules governing order creation.
Merchandise Planning Process (method)
Summarise POS• Aggregate by SKU, day of week and
week and store• Variance analysis forecast vs actual –
top 50
Classify Series• Volume/Seasonal• Slow moving• New like existing• Substitutes (rules & adjustments)• New line (rules & adjustments)
Plans & Assumptions• Inventory (receipts & balances)• Events calendar• Promotions• Weather• Economic • Staff
Baseline • Remove the effects of promotions • Impute probable demand for stock
outs (lost sales) or exclude these periods from training set
• Remove anomalies e.g. large one off orders from baseline
• Targeted rolling stock take
Forecast• High turnover items e.g. ARIMA• Low turnover lines e.g. Dynamic
Linear Models
Adjust, Agree and Execute• Store/Department review and adjust• Purchasing review supplier lead times• Purchasing review and adjust• Generate purchase orders and reports
Weather and Events
Include nulls as 0s (sparse)export 7 day fields for each week and factor
Plans & Assumptions in TM1
Staff cover by department, summarised into categories
Inventory plans based on prior forecast
Plans & Assumptions in TM1
Weather forecast This doesn’t have to held in TM1 but it’s a convenient place to store categories by day
Events calendar by DayUse several fields e.g. School holidays this week, School holidays next week. Special Events etc. IBM Social Media Analytics can help spot major events. Watson can read the paper.
Promotions PlanningPromotions being run in the forecast week and weeks prior by department
Economic FactorsIf it add value (accuracy to your models) then you can also compile economic factors
Merchandise Planning Process (method)
Summarise POS• Aggregate by SKU, day of week and
week and store• Variance analysis forecast vs actual –
top 50
Classify Series• Volume/Seasonal• Slow moving• New like existing• Substitutes (rules & adjustments)• New line (rules & adjustments)
Plans & Assumptions• Inventory (receipts & balances)• Events calendar• Promotions• Weather• Economic • Staff
Baseline • Remove the effects of promotions • Impute probable demand for stock
outs (lost sales) or exclude these periods from training set
• Remove anomalies e.g. large one off orders from baseline
• Targeted rolling stock take
Forecast• High turnover items e.g. ARIMA• Low turnover lines e.g. Dynamic
Linear Models
Adjust, Agree and Execute• Store/Department review and adjust• Purchasing review supplier lead times• Purchasing review and adjust• Generate purchase orders and reports
Weather and Events
Include nulls as 0s (sparse)
Merging baseline and other plans and assumptions from TM1 in SPSS
Baseline SKU volume, price, inventory plan, substitutes and other attributes, promotions, weather and event plans and assumptions are pulled from TM1, transformed and used for model training, evaluation and forecasting in SPSS
Demonstrate SPSS Modeler data preparation and modeling
Stock showing in inventory but no sales?
Use analytics to identify categories most at risk of being out of stock
Implement a targeted rolling stock take
Use data mining to identify shrinkage specific factors e.g. store, employee, shift, department, line, location, time, events, school holidays..
Merchandise Planning Process (method)
Summarise POS• Aggregate by SKU, day of week and
week and store• Variance analysis forecast vs actual –
top 50
Classify Series• Volume/Seasonal• Slow moving• New like existing• Substitutes (rules & adjustments)• New line (rules & adjustments)
Plans & Assumptions• Inventory (receipts & balances)• Events calendar• Promotions• Weather• Economic • Staff
Baseline • Remove the effects of promotions • Impute probable demand for stock
outs (lost sales) or exclude these periods from training set
• Remove anomalies e.g. large one off orders from baseline
Forecast• High turnover items e.g. ARIMA• Low turnover lines e.g. Dynamic
Linear Models
Adjust, Agree and Execute• Store/Department review and adjust• Purchasing review supplier lead times• Purchasing review and adjust• Generate purchase orders and reports
Weather and Events
Include nulls as 0s (sparse)export 7 day fields for each week and factor
Forecast Generation, Behind the scenes
The SPSS platform:
Automatically re-trains a model, evaluates accuracy and pushes a forecast for each SKU/Store (on a schedule) to TM1
In-database scale (SPARK, Netezza, DB2, SQL etc.) read/write from TM1
Automation with Python, R integration for the more exotic models
Powerful and easy to use data-mining GUI
Demonstrate SPSS Modeler merging data from TM1 and pushing forecast back to TM1
Accuracy, feature and model selection
The forecast modeling process involves:
Transforming data for the best results
Selecting fields/features that matter
Selecting a model produces the least error (ARIMA, GLM, Dynamic Linear, Dynamic Baysian etc.)
Example test data set Forecast SKU unit sales vs Actual.Factors used; weather, holidays, price but missing everything else. The tighter the dots around the line, the better the model
Agree and close (execute purchase orders)
Forecasts are now ready for review in TM1
TM1 inventory planning model uses rules to minimise stock outs and minimise probably stock outs using:
Lead time
Carrying costs
Minimum order quantities
Forecast demand
Purchase orders can be exported as a file to execute in your ERP
Adjust, Agree and Execute• Store/Department review and adjust• Purchasing review supplier lead times• Purchasing review and adjust• Generate purchase orders and reports
Demonstrate Store/department review/sign off
Demonstrate TM1 merchandise planning solution