Post on 25-Mar-2020
transcript
Deep Learning System Stack
Pooyan JamshidiUofSC
The Deep Learning Systems Juggle
Typical Deep Learning System Stack
Typical Deep Learning System Stack
Example: Logistic Regression
Logistic Regression in Numpy
Logistic Regression in Numpy
Logistic Regression in TinyFlow (TensorFlow like API)
Logistic Regression in TinyFlow
Logistic Regression in TinyFlow
Logistic Regression in TinyFlow
Logistic Regression in TinyFlow
The Declarative Language: Computation Graph
Computational Graph Construction by Step
Computational Graph Construction by Step
Computational Graph Construction by Step
Computational Graph Construction by Step
Execution only Touches the Needed Subgraph
Discussion: Computational Graph
Discussion: Numpy vs TF Program
Typical Deep Learning System Stack
Computation Graph Optimization
Parallel Scheduling
Typical Deep Learning System Stack
GPU Acceleration
Typical Deep Learning System Stack
Supporting More Hardware backends
Each Hardware backend requires a software stack
New Trend: Compiler based Approach