Date post: | 12-Apr-2017 |
Category: |
Data & Analytics |
Upload: | jen-aman |
View: | 63 times |
Download: | 2 times |
RISELab: Enabling Intelligent Real-time Decisions
Ion StoicaFebruary 8, 2017
Berkeley’s AMPLab (2011-2016)
2
Algorithms
Machines People
Goal: Next generation of open sourcedata analytics stack for industry & academia
Berkeley Data Analytics Stack (BDAS)
Berkeley’s AMPLab (2011-2016)
3
Algorithms
Machines People
Goal: Next generation of open sourcedata analytics stack for industry & academia
Berkeley Data Analytics Stack (BDAS)
…
RISE: Real-time Intelligent Secure Execution
From batch data to advanced analytics
AMPLab
5
From live data to real-time decisions
RISELab
RISE Lab (2017-2022)
12 faculty across AI, systems, security, and architectures
11 Founding sponsors
6
Why?Data only as valuable as the decisions it enables
7
Why?
What does this mean?• Faster decisions better than slower decisions• Decisions on fresh data better than decisions on stale data• Decisions on personalized data better than on aggregate data
8
Data only as valuable as the decisions it enables
Goal
Real-time decisions
on live data
with strong security
9
decide in ms
the current state of the environment
privacy, confidentiality, integrity
Typical decision system
Decision SystemQuery
Decision
Environment+
sensors & actuators
Observations, Feedback
Preprocess Intermediatedata
DecisionEngine
Decision
QueryDecisionEnginePreprocess Intermediate
data
Environment+
sensors & actuators
Typical decision system
Decision System
Observations, Feedback
LiveUpdate latency(e.g., ~1 seconds)
Decision
QueryDecisionEnginePreprocess Intermediate
data
Environment+
sensors & actuators
Typical decision system
Decision System
Observations, Feedback
LiveUpdate latency(e.g., ~1 seconds)
Secure
Real-timedecision latency
(e.g., ~10 ms)
Example of decision systems
Decision SystemQuery
DecisionTraining
Models(diff. tradeoffs
complexity/accuracy)
ModelServing
FeedbackObservations, Feedback
ML Pipeline(e.g., Clipper +
Spark/Tensorflow)
Decision SystemObs.
Action
Update Policy
Policyobs àaction
QueryPolicy
Observations, Rewards
ReinforcementLearning Systems
(e.g., Ray)
Pre-process
Interm-ediateData
DecisionEngine
What else do we want from decisions?
Intelligent: complex decisions in uncertain environments
Robust: handle complex noise, unforeseen inputs, failures
Explainable: ability to explain non-obvious decisions
Goal
Develop open source platforms, tools, and algorithms for
intelligent real-time decisions on live-data
Some Proposed Research
Secure Real-time Decisions Stack (SRDS) • Open source platform to develop of RISE apps• Secure from ground up• Reinforcement Learning (RL) as one of key app patterns
Learning control hierarchies: speedup learning, training
Shared learning: learn over confidential data
16
Secure Real-time Decisions Stack (SRDS) • Open source platform to develop of RISE apps• Secure from ground up• Reinforcement Learning (RL) as one of key app patterns
Learning control hierarchies: speedup learning, training
Shared learning: learn over confidential data
17
Some Proposed Research
Secure Real-time Decision Stack (SRDS)
scheduler object store
RISE μkernel
Ray Clipper …
Ground (data context service)
Tim
e M
achi
ne
scheduler object store
RISE μkernel
Ray Clipper …
Ground (data context service)
Tim
e M
achi
ne
Minimalist execution engine:• Support both data flow and task-parallel execution models• High-throughput, low-latency: ~ 1M tasks/sec @ ms latency
Secure Real-time Decision Stack (SRDS)
scheduler object store
RISE μkernel
Ray Clipper …
Ground (data context service)
Tim
e M
achi
ne
Central repository for models, APIs to capture the context in which data gets used and producedStatus: ongoing project with industry partners
Secure Real-time Decision Stack (SRDS)
scheduler object store
RISE μkernel
Ray Clipper …
Ground (data context service)
Tim
e M
achi
ne
Replaying of apps at fine granularity• Simplify development, debugging• Robustness: replay against perturbed inputs• Explainability: identify inputs causing decision• Security: confirm vulnerabilities, test security
patches, compliance auditing
Secure Real-time Decision Stack (SRDS)
scheduler object store
RISE μkernel
Ray Clipper …
Ground (data context service)
Tim
e M
achi
ne
Dramatically simplify development of RISE applications• Apache Spark: improve latency and security• Clipper: model serving for Apache Spark, Scikit learn, etc• Ray: framework for RL applications
Secure Real-time Decision Stack (SRDS)
Improving Apache SparkDrizzle• Decrease latency of Structured Streaming and ML algorithms by ~10x• Techniques: group scheduling, shared variables
23
Streaming Latency: YCSB benchmark
24
0
500
1000
1500
2000
1 2 4 8 12 16 20 24
Med
ian
Even
t Lat
ency
(m
s)
Throughput (Million events/s)
Spark Drizzle Flink Drizzle-Opt
Drizzle-Opt: Reduce-by on mapper side
Streaming Latency: YCSB benchmark
25
0
500
1000
1500
2000
1 2 4 8 12 16 20 24
Med
ian
Even
t Lat
ency
(m
s)
Throughput (Million events/s)
Spark Drizzle Flink Drizzle-Opt
Drizzle-Opt: Reduce-by on mapper side
Streaming Latency: YCSB benchmark
26
0
500
1000
1500
2000
1 2 4 8 12 16 20 24
Med
ian
Even
t Lat
ency
(m
s)
Throughput (Million events/s)
Spark Drizzle Flink Drizzle-Opt
Drizzle-Opt: Reduce-by on mapper side
Streaming Latency: YCSB benchmark
27
0
500
1000
1500
2000
1 2 4 8 12 16 20 24
Med
ian
Even
t Lat
ency
(m
s)
Throughput (Million events/s)
Spark Drizzle Flink Drizzle-Opt
Drizzle-Opt: Reduce-by on mapper side
Streaming Latency: YCSB benchmark
28
0
500
1000
1500
2000
1 2 4 8 12 16 20 24
Med
ian
Even
t Lat
ency
(m
s)
Throughput (Million events/s)
Spark Drizzle Flink Drizzle-Opt
Drizzle-Opt: Reduce-by on mapper side
15x
MLlib: SGD Performance
29
0
20
40
60
4 8 16 32 64 128
Tim
e / i
ter(
ms)
Machines
Spark Drizzle
MLlib: SGD Performance
30
0
20
40
60
4 8 16 32 64 128
Tim
e / i
ter (
ms)
Machines
Spark Drizzle
6x
Improving Apache SparkDrizzle• Decrease latency of Structured Streaming and ML algorithms by ~10x• Techniques: group scheduling, shared variables• Some of these techniques will make their way to Apache Spark
Opaque• Full data encryption, authentication, and verification (Intel’s SGX)• Oblivious mode: hide data access pattern• Support most SparkSQL functionality• See Wenting’s talk later
31
RISELab
Already promising results
Expect much more over the next five years!
32
Goal: Develop open source platforms, tools, and algorithms for intelligent real-time decisions on live-
data
Thank you