Sentiment Analysis Using Apache MXNet and Gluon - AWS Online Tech Talks

transcript

Vikram Madan, Sr. Product Manager

November 14, 2017

Sentiment Analysis Using

Apache MXNet and Gluon

Sentiment analysis

Sentiment analysis is a natural language processing (NLP) task for classifying

text or speech as some specified sentiment

Brand Sentiment Political Sentiment Customer Satisfaction

Performing sentiment analysis

Labelled Data

for Training

Embedding

Train Deep

Neural Network

Deploy Model

for Prediction

Focus of Today’s Discussion

IMBD dataset for movie reviews

Positive

“The impetuous pace of the film

is at one with its moral

shamelessness, and, without

thinking, we sign up for both”

“What a magical movie”

“It is a film so awe-inspiringly

wooden that it’s basically a fire-risk”

“An explosion in a stupid factory”

“Love him, hated Hur”

Negative

25,000 movie reviews: 12,500 positive & 12,500 negative

Converting words to vectors

Two Major Approaches: word2vec & GloVe

Training Co-Occurrence Data Output Vectors

Example: 3-Dimensional Word Vectors

GloVe: Global Vectors for Word Representation

Deep Neural Networks

Recurrent Neural Networks (RNNs)

Long Short-Term Memory (LSTM) networks

Illustrative Example (Part 1 of 2)

Edwin Chen Blog (http://blog.echen.me/)

Illustrative Example (Part 2 of 2)

Edwin Chen Blog (http://blog.echen.me/)

Gluon API Specification

Gluon is a technical specification providing a programming

interface that: (1) simplifies development of deep learning models,

(2) provides greater flexibility in building neural networks, and (3)

offers high performance

Simplicity Flexibility Performance

Gluon API is Available in Apache MXNet

Others…

Development of the Gluon API specification was

a collaboration between AWS and Microsoft

Available Now

Coming Soon

Gluon API is implemented in Apache

MXNet and soon will be in CNTK

Advantages of the Gluon API

Simple, Easy-to-

Understand Code

Flexible, Imperative

Structure

Dynamic Graphs

High Performance

Neural networks can be defined using simple, clear, concise code

Plug-and-play neural network building blocks – including predefined layers,

optimizers, and initializers

Eliminates rigidity of neural network model definition and brings together

the model with the training algorithm

Intuitive, easy-to-debug, familiar code

Neural networks can change in shape or size during the training process to

address advanced use cases where the size of data fed is variable

Important area of innovation in Natural Language Processing (NLP)

There is no sacrifice with respect to training speed

When it is time to move from prototyping to production, easily cache neural

networks for high performance and a reduced memory footprint

Simple, Easy-to-Understand Code

Use plug-and-play neural network building blocks, including

predefined layers, optimizers, and initializers

# First step is to initialize your model

net = gluon.nn.Sequential()

# Then, define your model architecture

with net.name_scope():

net.add(gluon.nn.Dense(256, activation="relu")) # 1st layer (256 nodes)

net.add(gluon.nn.Dense(256, activation="relu")) # 2nd hidden layer

net.add(gluon.nn.Dense(num_outputs)) # Output layer

Flexible, Imperative Structure

Prototype, build, and debug neural networks more easily with a

fully imperative interface

epochs = 10

for e in range(epochs):

for i, batch in enumerate(train_data):

with autograd.record(): # Start recording the derivatives

output = net(data) # the forward iteration

loss = softmax_cross_entropy(output, label)

loss.backward()

trainer.step(data.shape[0])

Dynamic Graphs

Build neural networks on the fly for use cases where neural

networks must change in size and shape during model training

def forward(self, F, inputs, tree):

children_outputs = [self.forward(F, inputs, child)

for child in tree.children]

#Recursively builds the neural network based on each input sentence’s

#syntactic structure during the model definition and training process

High Performance

Easily cache the neural network to achieve high performance

when speed becomes more important than flexibility

net = nn.HybridSequential()

with net.name_scope():

net.add(nn.Dense(256, activation="relu"))

net.add(nn.Dense(128, activation="relu"))

net.add(nn.Dense(2))

net.hybridize()

Sentiment Analysis Using Apache MXNet and Gluon - AWS Online Tech Talks

Documents