Sentiment Analysis Using Apache MXNet and Gluon - AWS Online Tech Talks

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Vikram Madan, Sr. Product Manager

November 14, 2017

Sentiment Analysis Using

Apache MXNet and Gluon


Sentiment analysis

Sentiment analysis is a natural language processing (NLP) task for classifying

text or speech as some specified sentiment

Brand Sentiment Political Sentiment Customer Satisfaction


Performing sentiment analysis

Labelled Data

for Training

Word

Embedding

Train Deep

Neural Network

Deploy Model

for Prediction

Focus of Today’s Discussion


IMBD dataset for movie reviews

Positive

“The impetuous pace of the film

is at one with its moral

shamelessness, and, without

thinking, we sign up for both”

“What a magical movie”

“It is a film so awe-inspiringly

wooden that it’s basically a fire-risk”

“An explosion in a stupid factory”

“Love him, hated Hur”

Negative

25,000 movie reviews: 12,500 positive & 12,500 negative


Converting words to vectors

Two Major Approaches: word2vec & GloVe



Training Co-Occurrence Data Output Vectors



Example: 3-Dimensional Word Vectors


GloVe: Global Vectors for Word Representation


Deep Neural Networks


Recurrent Neural Networks (RNNs)


Long Short-Term Memory (LSTM) networks


Illustrative Example (Part 1 of 2)

Edwin Chen Blog (http://blog.echen.me/)


Illustrative Example (Part 2 of 2)

Edwin Chen Blog (http://blog.echen.me/)


Gluon API Specification

Gluon is a technical specification providing a programming

interface that: (1) simplifies development of deep learning models,

(2) provides greater flexibility in building neural networks, and (3)

offers high performance

Simplicity Flexibility Performance


Gluon API is Available in Apache MXNet

Others…

Development of the Gluon API specification was

a collaboration between AWS and Microsoft

Available Now

Coming Soon

Gluon API is implemented in Apache

MXNet and soon will be in CNTK


Advantages of the Gluon API

Simple, Easy-to-

Understand Code

Flexible, Imperative

Structure

Dynamic Graphs

High Performance

Neural networks can be defined using simple, clear, concise code

Plug-and-play neural network building blocks – including predefined layers,

optimizers, and initializers

Eliminates rigidity of neural network model definition and brings together

the model with the training algorithm

Intuitive, easy-to-debug, familiar code

Neural networks can change in shape or size during the training process to

address advanced use cases where the size of data fed is variable

Important area of innovation in Natural Language Processing (NLP)

There is no sacrifice with respect to training speed

When it is time to move from prototyping to production, easily cache neural

networks for high performance and a reduced memory footprint


Simple, Easy-to-Understand Code

Use plug-and-play neural network building blocks, including

predefined layers, optimizers, and initializers

# First step is to initialize your model

net = gluon.nn.Sequential()

# Then, define your model architecture

with net.name_scope():

net.add(gluon.nn.Dense(256, activation="relu")) # 1st layer (256 nodes)

net.add(gluon.nn.Dense(256, activation="relu")) # 2nd hidden layer

net.add(gluon.nn.Dense(num_outputs)) # Output layer


Flexible, Imperative Structure

Prototype, build, and debug neural networks more easily with a

fully imperative interface

epochs = 10

for e in range(epochs):

for i, batch in enumerate(train_data):

with autograd.record(): # Start recording the derivatives

output = net(data) # the forward iteration

loss = softmax_cross_entropy(output, label)

loss.backward()

trainer.step(data.shape[0])


Dynamic Graphs

Build neural networks on the fly for use cases where neural

networks must change in size and shape during model training

def forward(self, F, inputs, tree):

children_outputs = [self.forward(F, inputs, child)

for child in tree.children]

#Recursively builds the neural network based on each input sentence’s

#syntactic structure during the model definition and training process


High Performance

Easily cache the neural network to achieve high performance

when speed becomes more important than flexibility

net = nn.HybridSequential()

with net.name_scope():

net.add(nn.Dense(256, activation="relu"))

net.add(nn.Dense(128, activation="relu"))

net.add(nn.Dense(2))

net.hybridize()

Date post:	21-Jan-2018
Category:	Documents
Upload:	amazon-web-services
View:	529 times
Download:	0 times

Sentiment Analysis Using Apache MXNet and Gluon - AWS Online Tech Talks

Documents