Introduction to Machine Learning with Python and scikit-learn

transcript

Python AtlantaNov. 14th 2013

Matt Hagymatt@liveramp.com

Slide #2 Intro to Machine Learning with Python matt@liveramp.com

Machine Learning (ML):• Finding patterns in data

• Modeling patterns

• Use models to make predictions

ML can be easy*• You already have ML applications!

• You can start applying ML methods now with Python & scikit-learn

• Theoretical knowledge of ML not needed (initially)*

*Gaining more background, theory, and experience will help

Simple Example

Simple Model

import numpy as npfrom sklearn.linear_model import LinearRegression

x,y = np.load('data.npz')x_test = np.linspace(0, 200)

model = LinearRegression()model.fit(x[::, np.newaxis], y)y_test = model.predict(x_test[::, np.newaxis])

Variance/Bias Trade Off

• Need models that can adapt to relationships in our data

• Highly adaptable models can over-fit and will not generalize

• Regularization – Common strategy to address variance/bias trade off

import numpy as npfrom sklearn.svm import SVRfrom sklearn.pipeline import Pipelinefrom sklearn.preprocessing import StandardScaler

x,y = np.load('data.npz')x_test = np.linspace(0, 200)

model = Pipeline([ ('standardize', StandardScaler()), ('svr', SVR(kernel='rbf', verbose=0, C=5e6, epsilon=20)) ])model.fit(x[::, np.newaxis], y)y_test = model.predict(x_test[::, np.newaxis])

regularizationterm

Supervised Learning

031342934

Input, X

1637931767

Output, Y Modeling relationship between inputs and outputs

Multiple Inputs

Input, X

031342934

231689123

103127542

470291321

1637931767

Output, Y

Example: Image Classification

• Classify handwritten digits with ML models

• Each input is an entire image

• Output is digit in the image

9Input, X Output, Y

import numpy as npfrom sklearn.ensemble import RandomForestClassifier

with np.load(’train.npz') as data: pixels_train = data['pixels'] labels_train = data['labels’]with np.load(’test.npz') as data: pixels_test = data['pixels']

# flattenX_train = pixels_train.reshape(pixels_train.shape[0], -1)X_test = pixels_test.reshape(pixels_test.shape[0], -1)

model = RandomForestClassifier(n_estimators=50)model.fit(X_train, labels_train)labels_test = model.predict(X_test)

Trains on 50,000 images in roughly 20 seconds.96% accurate !!

Kaggle Data Science Competition

• Given 6 million training questions labeled with tags

• Predict the tags for 2 million unlabeled test questions

www.users.globalnet.co.uk/~slocks/instructions.htmlstackoverflow.com/questions/895371/bubble-sort-homework

Predicting the tags of Stack Overflow questions with machine learning

Text Classification Overview

Raw Posts Vector Space Machine Learning Model

Feature Extraction & Selection

Model Selection & Training

Term Frequency Feature Extraction

“Why is processing a sorted array faster than processing an array this is not sorted?”

Characterize text by the frequency of specific words in each text entry

Example Title:

whyprocessing

sorted

faster

1 2 2 2 1

Term Frequencies

Ignore common words (i.e. stop words)

Frequency of key terms is anticipated to be correlated with the tags of the question

processing

sorted

faster

homework

Title 1 1 2 2 2 1 0 0 0 0

Title 2 0 0 0 0 0 1 1 1 1

Title 3 0 0 1 1 0 0 1 0 1

Example Model Coefficients

ML can be easy*• You already have ML problems!

• You can start applying ML methods now with Python & scikit-learn

• Theoretical knowledge of ML not needed (initially)*

scikit-learn.org

github.com/scikit-learn

Check out: liveramp.com/careers

Helping companies use their marketing data to delight customers

Opportunities•Backend Engineers•Data Scientists•Full-Stack Engineers

Tools•Java•Hadoop (Map/Reduce)•Ruby

Build and work with large distributed systems that process massive data sets.

Introduction to Machine Learning with Python and scikit-learn

Education