Real-Time Cloud Computing

ChenyangLuCyber-PhysicalSystemsLaboratoryh7p://www.cse.wustl.edu/~lu/

InternetofThings

Convergence of Ø  Miniaturized hardware: integrate processor, sensors and radios.Ø  Low-power wireless: connect millions of devices to the Internet.

Ø  Data analytics: make sense of sensor data.

Ø  Cloud: scalable computing.

R.Dor,G.Hackmann,Z.Yang,C.Lu,Y.Chen,M.KollefandT.C.Bailey,ExperienceswithanEnd-To-EndWirelessClinicalMonitoringSystem,ConferenceonWirelessHealth(WH'12),October2012.

InternetofThings

Convergence of Ø  Miniaturized hardware: integrate processor, sensors and radios.Ø  Low-power wireless: connect millions of devices to the Internet.

Ø  Data analytics: make sense of sensor data.

Ø  Cloud: scalable computing.

R.Dor,G.Hackmann,Z.Yang,C.Lu,Y.Chen,M.KollefandT.C.Bailey,ExperienceswithanEnd-To-EndWirelessClinicalMonitoringSystem,ConferenceonWirelessHealth(WH'12),October2012.

IoT-DrivenControl

Ø  Internet of Things à large-scale real-time controlq  Smart manufacturing, smart transportation, smart grid…

q  Need real-time cloud computing for IoT!

Ø  Example: Intelligent Transportationq  Collect data from cameras and roadside detectors.

q  Analyze traffic dynamics and optimize traffic signals.

q  Control traffic signals.

q  SCATS @ Sydney: controlling 3,400 signals at 1s round-trip latency.

4

EmbeddedSystemVirtualiza=on

Ø Consolidate 100 ECUs à ~10 multicore processors.Ø  Integrate multiple systems on a common platform.

q  Infotainment on Linux or Android

q  Safety-critical control on AUTOSAR

Ø  Preserve real-time performance on a virtualized platform!

2/25/17 5

Source:h7p://www.edn.com/design/automoYve/4399434/MulYcore-and-virtualizaYon-in-automoYve-environments

Ø  Virtualization platforms provide no guarantee on latencyq  Xen: credit scheduler, [credit, cap]

q  VMware ESXi: [reservation, share, limitation]

q  Microsoft Hyper-V: [reserve, weight, limit]

Ø Clouds lack service level agreement on latencyq  Amazon, Google, Microsoft cloud services: #VCPUs

6

Cloudisreal-=metoday

Current clouds provision resources, not latency!

TowardsReal-TimeCloudØ  Support real-time applications in the cloud.

q  Latency guarantees for tasks running in virtual machines (VMs).

q  Real-time performance isolation between VMs.

q  Resource sharing between real-time and non-real-time VMs.

Ø Multi-level real-time performance provisioning.q  RT-Xen à real-time VM scheduling on a virtualized host.

q  VATC à real-time network I/O on a virtualized host.q  RT-OpenStack à real-time cloud resource management.

7

VATC: Real-Time Communication

RT-O

penS

tack

Xen

Ø Xen: type-1, baremetal hypervisor q  Domain-0: drivers, tool stack to control VMs.

q  Guest Domain: para-virtualized or fully virtualized OS.

Ø  Scheduling hierarchyq  Xen schedules VCPUs on PCPUs.

q  Guest OS schedules threads on VCPUs.

q  Xen credit scheduler: round-robin with proportional share.

8

PCPUs

OS Sched

Xen Scheduler

OS Sched OS Sched

VCPUReal-TimeTask

RT-Xen

Ø  Real-time schedulers in the Xen hypervisor.Ø  Provide real-time guarantees to tasks in VMs.

Ø  Incorporated in Xen 4.5 as the real-time scheduler.

9

RT-Xen

h7ps://sites.google.com/site/realYmexen/

S. Xi, M. Xu, C. Lu, L. Phan, C. Gill, O. Sokolsky and I. Lee, Real-Time Multi-Core Virtual Machine Scheduling in Xen, ACM International Conference on Embedded Software (EMSOFT'14), October 2014.

Composi=onalScheduling

Ø Analytical real-time guarantees to tasks running in VMs.Ø  VM resource interfaces

q  A set of VCPUs each with resource demand <period, budget >

q  Hides task-specific informationq  Computed based on compositional scheduling analysis

10

Resource Interface Resource Interface

Resource Interface

Hypervisor

Virtual Machines

Workload Workload

Scheduler

Scheduler

Scheduler

Real-TimeSchedulerDesignØ  Global scheduling

q  Allow VCPU migration across cores

q  Work conserving – utilize any available cores

q  Migration overhead and cache penalty

Ø  Partitioned schedulingq  Assign and bind VCPUs to cores

q  Cores may idle when others have work pending

q  No migration overhead or cache penalty

Ø  Enforce resource interface through budget managementq  Periodic server vs. deferrable server

Ø  Priority: Earliest Deadline First vs. Deadline Monotonic

11

RT-Xenvs.Xen

12

•  Xen misses deadlines at 22% of CPU capacity.

•  RT-Xen delivers real-time performance at 78% of CPU capacity.

VirtualizedNetworkI/O

Ø Xen handles all network traffic through Dom0Ø  Real-time and non-real-time traffic share Dom0

q  CPU and network contention

Ø  Long delays for real-time traffic in virtualized hosts

XenHypervisor

NetworkComponents

Non Real-Time

App

Dom2

CPUMemory Storage

Real-Time App

Dom1Dom0

NIC

13

NetworkI/OinVirtualizedHostsØ  Linux Queueing Discipline

q  Rate-limit and shape flows

q  Prioritization or fair packet scheduling

Ø  Priority inversion in virtualization componentsq  between transmissionsq  between transmission and reception

Ø  VATC: Virtualization-Aware Traffic Controlq  Process packets in prioritized kernel threadsq  Dedicated packet queues per priority NIC

QueueingDiscipline

Dom0

Real-Time App

Dom1Non- Real-Time App

Dom2

Virtualiza=onComponents

14

C. Li, S. Xi, C. Lu, C. Gill and R. Guerin, Prioritizing Soft Real-Time Network Traffic in Virtualized Hosts Based on Xen, IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS'15), April 2015.

Real-TimeTrafficLatency

VATC reduces priority inversion à lower latency for real-time traffic.

15

10 16 32 64 128 256 512 1024 Dyn Cons Dyn0

0.5

1

1.5

2

2.5

3

3.5

4

Roun

d−tri

p La

tenc

y ( m

s)

Interrupt Interval (µs)

Prio, Dom0−3.18FQ_CoDel, Dom0−3.18VATC

•  Medianround-triplatencyofreal-Ymetraffic.•  CPUcontenYonfromtwosmall-packetinterferingstreams.

VirtualizedHostàCloud

Ø  Provide real-time performance to real-time VMsØ Achieve high resource utilization

16

OpenStackLimita=ons

Ø  Popular open-source cloud management system

Ø  VM resource interfaceq  Number of VCPUsq  Not real-time

Ø  VM-to-host mappingq  Filtering (admission control)

•  VCPU-to-PCPU ratio (16:1), max VMs per host (50)

•  Coarse-grained admission control for CPU resources

q  Ranking (VM allocation)

•  Balance memory usage

•  No consideration of CPU resources

17

Manager

Host Host Host

VM VM VM VM VM

RT-OpenStack

Ø Co-hosting real-time VMs with non-real-time VMs

Ø Deliver real-time performanceq  Support RT-Xen resource interfaceq  Real-time-aware VM-to-host mapping

Ø Achieve high resource utilizationq  Co-locate non-real-time VMs with real-time VMsq  Non-real-time VMs consume remaining resources without affecting the

real-time performance of real-time VMs

18

S. Xi, C. Li, C. Lu, C. Gill, M. Xu, L. Phan, I. Lee, O. Sokolsky, RT-OpenStack: CPU Resource Management for Real-Time Cloud Computing, IEEE International Conference on Cloud Computing (CLOUD'15), June 2015.

RT-OpenStack:VM-to-HostMapping

Ø Admission control: RT-Filterq  Accept real-time VMs based on schedulability and memory

q  Consider only accepted real-time VMs

Ø  VM allocation: RT-Weigherq  Balance CPU utilization

q  Consider only accepted real-time VMs

19

ResourceInterface AdmissionControl VMAlloca=on

Real-TimeVMs {<period,budget>} Schedulability+Memory CPUUYlizaYon

Non-Real-TimeVMs BestEffort Memory Memory

OpenStack

20

13%

36%

31% 61% 37%

75%30%29%

47% 32%

73%HadoopfinishYme:314seconds

Ø  Overload four hosts with real-time VMs à deadline misses. Ø  Two hosts running non-real-time VMs only.

Ø  Unbalanced distribution of real-time domains.

RT-OpenStack

21

0%

0%0%

0%

0% 0%

0%

0%0% 0%

0%

HadoopfinishYme:435seconds

Ø  Schedulability guarantees for real-time VMs à no deadline miss.Ø  Distribute real-time VMs across hosts.

Ø  Hadoop makes progress using remaining CPU resources.

Real-=meCloudStack

Ø  RT-Xen: real-time VM scheduling in virtualized hosts.Ø  VATC: real-time network I/O in virtualized hosts.

Ø  RT-OpenStack: real-time cloud resource management.

22

VATC: Real-Time Communication

RT-O

penS

tack

Latency guarantees

End-to-EndReal-TimeforIoT

Ø Miniaturized hardware à real-time embedded systemsØ  Low-power wireless à real-time wireless

Ø Data analytics à real-time analyticsØ Cloud à real-time service chains from edge to cloud

23

Large-Scale IoT-driven Control�à Smart Manufacturing, City, Grid…