Introduction to Neural Nets in TensorFlow, with Application

Jackson Barkstrom (with advisor Michael Lee)

Example Task: Image Classification:Using the MNIST Databaseof Handwritten Digits

[0,0,0,1,0,0,0,0,0…………….….1,0,0,0,0,0]

Process:Reshaping Data and Implementing A Neural NetNote: we could look at the data in two dimensions and use a Convolutional Neural Network, and TensorFlow

can handle as many as four dimensions. But for our simplified example, we converted the data to one

dimension.

Introduction:What is a Neural Net?: The Basic Framework

0.1

0.8

1

8

Mathematical Code Walkthrough

Specifying Input (x) and Output (y)

The Neural Net in Code

The Math, explained

Softmax Activation Function: Great for One-Hot Encoding

The sum of all the entries adds to one, and the largest entry will be nearly one while the smaller ones will be nearly zero.

Gradient Descent Optimization

Cross entropy error function

Results: 96% Accuracy After Feeding in Data

Note: We could probably improve these results by using a Convolutional Neural Network

An Application: Predicting Fires in Charlottesville, Virginia

Note: It’s best to use neural networks when we have a LOT of data, and for this example we didn’t

have enough data to make the neural network the best approach. However, using a neural net for

this is still an interesting application as far as learning is concerned.

How I Made the Data

● Started with four datasets: one of all commercial homes, one of all residential homes, one of all fires before 2016, and one of all fires after 2018.

● Modified addresses, cleaned things up a lot, and merged fires to the commercial and residential datasets by addresses.

● One-hot encoded all categorical variables, and normalized all numerical variables.

We have to create a “dataset” object with TensorFlow to feed data into, and we have to specify how we want iterate

through the data. Other than that, it’s almost exactly the same code as before!

100%Of addresses were usually predicted NOT to have fires. This is not surprising! But don’t worry, we still have something...

Our neural net outputs its confidence in its predictions, and we can use this to generate arbitrary “risk levels.”

~8%Of commercial addresses in our highest risk category had fires within the next 2 years! Around 5% of

homes in our middle risk category, and around 2.5% of homes the lowest risk category also had fires

within the next two years.

Insight: we can tell that some houses have triple the fire

risk of others! This is useful!

~6%Of residential addresses in our highest risk category had fires within the next 2 years. Around 4% of

homes in our middle risk category, and around 2.5% of homes the lowest risk category also had fires

within the next two years. But this doesn’t make sense… we have more data for residential homes both in

number and in detail, so shouldn’t we do better here?

Overfitting is a likely culprit here. It would be useful to use a validation set.

Possible Improvements

● Train/test splitting by time (checking if our model can help us understand what will happen in the future)

● Methodically eliminate less useful columns to prevent overfitting● Improve how we normalize columns--the difference between 1 and 20

in a column could be very meaningful, but if we have a gigantic value in the column it could this difference look like nothing to the neural net.

● Change an activation function? Add more layers?

We Can Do a Lot with Neural Nets

Both regression and classification!

(as long as we have a lot of data and we figure out a good way to format the input and the output)

Thanks for Listening!