Estimating Calorie Content in Food Dishes using Transfer Learning

Neural Networks

Ali Abul Hawa

2

Difficulty in this project/dataset: many food dishes look very similar

Beef noodleRamen noodle

3

4

Low-levelfeatures

Higher-levelfeatures

Output Classification

Layers

1.2M images1000 classes

16 Layers

VGG16, 16 layers: 3 weeks on 4 GPUs Titan Black ~$6000

My computer (~ g2.2xlarge AWS) : ~ 30 weeks

TRANSFER LEARNING

5

Low-levelfeatures

Higher-levelfeatures

CustomOutput

Classification Layers

15 Layers

100Food

classifier

Train custom classification layer ~52%

TRANSFER LEARNING

6

Low-levelfeatures

Higher-levelfeatures

CustomOutput

Classification Layers

15 Layers

100Food

classifier

Train classification layer + highest level feature layer

~ 70% top-1, 92% top-5 accuracyin 30 hours 30 weeks

TRANSFER LEARNING

7

TRANSFER LEARNING

Why transfer learning?Time

Works well even on small datasets. However, need to use image augmentations to prevent overfitting!

8

Live Demo …

… Thanks!