1© 2015 The MathWorks, Inc.
Machine Learning for Predictive Modelling
Rory Adams
2
Machine Learning
– What is Machine Learning and why do we need it?
– Common challenges in Machine Learning
Example: Human activity learning using mobile phone data
Example: Real-time object identification using images
Example: Load forecasting using weather data
Summary & Key Takeaways
Agenda
3
Machine Learning is Everywhere
Image Recognition
Speech Recognition
Stock Prediction
Medical Diagnosis
Data Analytics
Robotics
and more…
[TBD]
4
Machine Learning
Machine learning uses data and produces a model to perform a task
Standard Approach Machine Learning Approach
𝑚𝑜𝑑𝑒𝑙 = <𝑴𝒂𝒄𝒉𝒊𝒏𝒆𝑳𝒆𝒂𝒓𝒏𝒊𝒏𝒈𝑨𝒍𝒈𝒐𝒓𝒊𝒕𝒉𝒎
>(𝑠𝑒𝑛𝑠𝑜𝑟_𝑑𝑎𝑡𝑎, 𝑎𝑐𝑡𝑖𝑣𝑖𝑡𝑦)
Computer
Program
Machine
Learning
𝑚𝑜𝑑𝑒𝑙: Inputs → OutputsHand Written Program Formula or Equation
If X_acc > 0.5
then “SITTING”
If Y_acc < 4 and Z_acc > 5
then “STANDING”
…
𝑌𝑎𝑐𝑡𝑖𝑣𝑖𝑡𝑦= 𝛽1𝑋𝑎𝑐𝑐 + 𝛽2𝑌𝑎𝑐𝑐+ 𝛽3𝑍𝑎𝑐𝑐 +
…
Task: Human Activity Detection
𝑚𝑜𝑑𝑒𝑙: Predictors → Response
5
Different Types of Learning
Machine Learning
Supervised Learning
Classification
Regression
Unsupervised Learning
• Discover a good internal representation
• Learn a low dimensional representation
• Response is a continuous number
(temperature, stock prices).
• Response is a choice between classes
• (True, False) (Red, Blue, Green)
6
Example: Human Activity Learning Using Mobile Phone Data
Machine
Learning
Data:
3-axial Accelerometer data
3-axial Gyroscope data
7
“essentially, all models are wrong,
but some are useful”
– George Box
8
Steps Challenge
Access, explore and analyze
dataData diversity
Numeric, Images, Signals, Text – not always tabular
Preprocess dataLack of domain tools
Filtering and feature extraction
Feature selection and transformation
Train modelsTime consuming
Train many models to find the “best”
Assess model performanceAvoid pitfalls
Over Fitting
Speed-Accuracy-Complexity tradeoffs
Iterate
Challenges in Machine LearningHard to get started
Steps Challenge
Access, explore and analyze
dataData diversity
Numeric, Images, Signals, Text – not always tabular
Preprocess dataLack of domain tools
Filtering and feature extraction
Feature selection and transformation
Train modelsTime consuming
Train many models to find the “best”
Assess model performanceAvoid pitfalls
Over Fitting
Speed-Accuracy-Complexity tradeoffs
Iterate
9
PREDICTION
Machine Learning Workflow
Train: Iterate till you find the best model
Predict: Integrate trained models into applications
MODELSUPERVISED
LEARNING
CLASSIFICATION
REGRESSION
PREPROCESS
DATA
SUMMARY
STATISTICS
PCAFILTERS
CLUSTER
ANALYSIS
LOAD
DATAPREPROCESS
DATA
SUMMARY
STATISTICS
PCAFILTERS
CLUSTER
ANALYSIS
NEW
DATA
PREPROCESS
DATA
SUMMARY
STATISTICS
PCAFILTERS
CLUSTER
ANALYSIS
MODEL
10
Machine Learning
– What is Machine Learning and why do we need it?
– Common challenges in Machine Learning
Example: Human activity learning using mobile phone data
Example: Real-time object identification using images
Example: Load forecasting using weather data
Summary & Key Takeaways
Agenda
11
Example 1: Human Activity Learning Using Mobile Phone Data
Objective: Train a classifier to classify
human activity from sensor data
Data:
Approach:
– Extract features from raw sensor signals
– Train and compare classifiers
– Test results on new sensor data
Predictors 3-axial Accelerometer and
Gyroscope data
Response Activity:
(Classification)
12
PREDICTIONMODEL
Machine Learning Workflow for Example 1
Train: Iterate till you find the best model
Predict: Integrate trained models into applications
MODELSUPERVISED
LEARNING
CLASSIFICATION
REGRESSION
PREPROCESS
DATA
SUMMARY
STATISTICS
PCAFILTERS
CLUSTER
ANALYSIS
LOAD
DATA
PREPROCESS
DATA
SUMMARY
STATISTICS
PCAFILTERS
CLUSTER
ANALYSIS
TEST
DATA
1. Mean
2. Standard
deviation
3. PCA
Classification
Learner
1. Mean
2. Standard
deviation
3. PCA
13
Machine Learning
– What is Machine Learning and why do we need it?
– Common challenges in Machine Learning
Example: Human activity learning using mobile phone data
Example: Real-time object identification using images
Example: Load forecasting using weather data
Summary & Key Takeaways
Agenda
14
Example 2: Real-time Toy Identification Using Images
Objective: Train a classifier to identify toy
type from a webcam video
Data:
Approach:
– Extract features using Bag-of-words
– Train and compare classifiers
– Classify streaming video from a webcam
Predictors Several images of cars:
Response CAR, HELICOPTER, PLANE, BIKE
(Classification)
15
PREDICTIONMODEL
Machine Learning Workflow for Example
Train: Iterate till you find the best model
Predict: Integrate trained models into applications
MODELSUPERVISED
LEARNING
CLASSIFICATION
REGRESSION
PREPROCESS
DATA
SUMMARY
STATISTICS
PCAFILTERS
CLUSTER
ANALYSIS
LOAD
DATA
PREPROCESS
DATA
SUMMARY
STATISTICS
PCAFILTERS
CLUSTER
ANALYSIS
WEBCAM
1. Build Bag-of-
features
2. Encode images
as new features
Classification
Learner
Encode images as
new features
16
Machine Learning
– What is Machine Learning and why do we need it?
– Common challenges in Machine Learning
Example: Human activity learning using mobile phone data
Example: Real-time object identification using images
Example: Load forecasting using weather data
Summary & Key Takeaways
Agenda
17
Example 3: Day-Ahead System Load Forecasting
Objective: Train a neural network to predict
the required system load for a zone
Data:
Approach:
– Extract additional features
– Train neural network
– Predict load
Predictors Temperature, Dew point, Month,
Day of week, Prior day load, Prior
week load
Response LOAD
(Regression)
18
PREDICTIONMODEL
Machine Learning Workflow for Example 1
Train: Iterate till you find the best model
Predict: Integrate trained models into applications
MODELSUPERVISED
LEARNING
CLASSIFICATION
REGRESSION
PREPROCESS
DATA
SUMMARY
STATISTICS
PCAFILTERS
CLUSTER
ANALYSIS
LOAD
DATA
PREPROCESS
DATA
SUMMARY
STATISTICS
PCAFILTERS
CLUSTER
ANALYSIS
TEST
DATA
Temp, Dew point
Day of week
Prior day load
Prior week load
Neural Network
Temp, Dew point
Day of week
Prior day load
Prior week load
19
Machine Learning
– What is Machine Learning and why do we need it?
– Common challenges in Machine Learning
Example: Human activity learning using mobile phone data
Example: Real-time object identification using images
Example: Load forecasting using weather data
Summary & Key Takeaways
Agenda
20
Steps Challenge
Accessing, exploring and
analyzing dataData diversity
Preprocess data Lack of domain tools
Train models Time consuming
Assess model
performance
Avoid pitfallsOver Fitting,
Speed-Accuracy-Complexity
Iterate
Challenges in Machine Learning
21
MATLAB Strengths for Machine Learning
Challenge Solution
Data diversityExtensive data support
Import and work with signal, images, financial,
Textual, geospatial, and several others formats
Lack of domain toolsHigh-quality libraries
Industry-standard algorithms for Finance, Statistics, Signal,
Image processing & more
Time consuming Interactive, app-driven workflows
Focus on machine learning, not programing
Avoid pitfallsOver Fitting,
Speed-Accuracy-Complexity
Integrated best practicesModel validation tools built into app
Rich documentation with step by step guidance
Flexible architecture for customized workflowsComplete machine learning platform
22
Consider Machine Learning when:
– Hand written rules and equations are too complex
Face recognition, speech recognition, recognizing patterns
– Rules of a task are constantly changing
Fraud detection from transactions, anomaly in sensor data
– Nature of the data changes and the program needs to adapt
Automated trading, energy demand forecasting, predicting shopping trends
MATLAB for Machine Learning
Key Takeaways
23
Additional Resources
Documentation:Machine Learning with MATLAB:
24
Q & A
Topic of interest Session / Demo Station
Working with IoT data Session: MATLAB and the Internet of Things (IoT): Collecting and
Analysing IoT Data
Accessing, analysing and
visualising data
Session: Analysis of Experimental and Test Data
Working with big data sets Session: Tackling Big Data with MATLAB
Deploying machine learning
algorithms
Demo: Building MATLAB Apps to Visualise Complex Data
Machine learning with computer
vision
Demo: Identification of Objects in Real-Time Video