Post on 29-Dec-2015
transcript
SU YUXINJAN 20, 2014
Petuum: An Iterative-Convergent Distributed
Machine Learning Framework
Outline
Introduction
Implementation
Questions
Demo
Introduction to Petuum
Bulk Synchronous Parallel
Asynchronous
Parameters read / update at any time
Stale Synchronous Parallel
Convergence
Programming
read(table, row, col)
inc(table, row, col, value)
iteration()
Implementation
Overview in Logic
Overview in the Real
Main Components
Table
ConsistencyController::DoGet()
ConsistencyController::iterate()
Server::GetRow()
Least-Recently-Used(LRU) Strategy
Questions
Is Lock-Free Possible ?
Data exchange in real-time ? next …
Is Auto-Rescheduling Possible ?
sub-centralized server
reduce communication cost
Is Auto-Partition Possible ?
Run ML algorithms like that in a single thread
A Solution for all ML algorithms
In-Memory or In-Storage ?
Data capacity is greater than memory size.
Memory should be a cache for disk storage.
Solution for disk storage: Hadoop Spark ….
New Schema to Reduce the Upper Bound?
STRADS Scheduler
Variable Correlations Auto-Parallelization
Dynamic Prioritization Monitor the contribution of variables to objective
function
Load-Balancing in Task
Demo
Switch to my laptop …