Boomerang: Real-Time I/O Meets

Legacy Systems

Ahmad Golchin, Soham Sinha, Richard West

Boston University

2

Introduction

● Key challenges:– How to extend real-time system with legacy functionality?– How to extend legacy system with real-time capabilities?– How to isolate tasks and services of different criticality

levels?– How to guarantee I/O processing is real-time

● Input → Processing → Output

● Contribution:– Boomerang: I/O system built using Quest-V partitioning

hypervisor and Quest RTOS– Composable tuned pipes

3

Spatial and Temporal Isolation

● Spatial

– Ensure one software component cannot alter another’s private code or data, or interfere with control of its devices

● Temporal

– Ensure a software component cannot affect when another component accesses a resource

4

Spatial and Temporal Isolation

● Lack of isolation leads to timing and functional failures

– Potentially catastrophic for high-criticality tasks

● Enforce isolation using separate hardware for each functional component

– e.g., separate ECU in an automotive system

– Not scalable or cost-effective

5

Spatial and Temporal Isolation

● Single hardware solution

– Use machine virtualization for spatial isolation of tasks of different criticalities

– Separate cores statically mapped to different virtual machines

– Different cores used for timing-critical tasks● Use a partitioning hypervisor to manage system

– Quest-V

6

Conventional vs Boomerang I/O

LEGACY OSRTOSLEGACY OS(e.g. Linux)

CoreMemory Devices

12

3

(a) Conventional (b) Boomerang

Core(s)

I/Ohandling

...

Tasks

Memory DevicesCore(s)

Real-TimeTasks + I/Ohandling

...

Memory DevicesCore(s)

Low-criticality

Tasks + I/Ohandling

...

● Boomerang uses Quest-V partitioning hypervisor to support tuned pipes between RTOS & legacy OS

– Allows legacy OS to contribute to real-time I/O without interference from low-criticality device interrupts

7

Tuned Pipes

● Like POSIX pipes but guarantee throughput and delay on communication

● Boomerang I/O subsystem supports real-time I/O (Tuned Pipes) across Quest RTOS and legacy OSes

– Empowers legacy OSes (Linux, Android) with real-time capabilities

– Leverages Quest RTOS VCPU scheduling of tasks and interrupts

8

VCPU Scheduling in Quest

Main VCPUs

I/O VCPUs

Threads

PCPUs (Cores)

Address Space

9

Boomerang: Tuned Pipes

VCPU

InputEndpoint

tpipe

OutputEndpoint

Task /Handler

tpipe_id_t tpipe (ep_t *inp[], int n_inp, ep_t *outp,qos_t spec, tpipe_task_t func, void* arg);

Boomerang automatically calculates VCPU parameters to meet QoS spec when all inputs and outputs are connected

10

Tuned Pipes Buffering Semantics

● Pipeline composition of two or more tuned pipes

● RT_ASYNC – Simpson’s 4-slot asynchronous communication● RT_FIFO – Ring-buffered semi-asynchronous communication

VCPU

Task

VCPU

Task

tpipe1 tpipe2

11

Tuned Pipes Buffering Semantics

● Pipeline composition of two or more tuned pipes

● Device Pipe – Interrupt handling (IO VCPU) + scatter/gather processing (Main VCPU)

● Task Pipe – Data processing (Main VCPU only)

DeviceEndpoint

Buffer

IO VCPU

Handler

MAINVCPU

Scatter,Gather

Device Pipe

VCPU

Task

Task Pipe

RT_ASYNC

PipeBuffer

12

Example Pipeline Composition

[∗](A | B), C | D | E, F [e2e_tput | loss_rate, e2e_delay]

● Parallel pipelines separated by commas● Asterisk enforces ring-buffered lossless communication● e2e_tput: min msgs/time exiting final pipe (for lossless comms)● e2e_delay: applies to longest path

A B

C

D

E

F

13

Boomerang: Pipeline Constraints

● Automatically establish VCPU (C,T) tuned pipe parameters satisfying:

● For FIFO and 4-slot:

1)

2)

● For FIFO only:

3)

4)

● For 4-slot only:

3)

∑i∈l

T i≤e2 e _ delay for longest path l

min∀ i{miT i

}≥e2 e _ tput , for mi≥1messages transferred by tpipei every C i

All FIFObuffersare sized toensure noadditional blockingdelays

All task scheduling constraints aremet

max{ 1−T pT c

}≤loss _ rate , for allT p≤T c

14

Boomerang: Experimental Setup

LINUXQuest RTOS 21

Memory Core 0 Memory Core 1 Devices

USB-CANInterfaceUSB-CANInterface

Devices

ECU1ECU1

ECU2ECU2

ECU3ECU3

ECU4ECU4

(CAN1)

(CAN2-3)

(CAN4)

(CAN5)

Real-Time Tasks & I/O

Handling

... ...

Low-criticalitytasks & non-real-time I/O

SSD NICNIC

UP S

quare

d S

BC

● Boomerang tuned pipe path (1) spans Quest + Linux + USB-CAN ● Boomerang tuned pipe path (2) spans Quest + USB-CAN

15

Boomerang: Experimental Setup

LINUX SMPTasks and IRQs partitioned

SHM2

1

Memory Core 0 Memory Core 1 Devices

USB-CANInterfaceUSB-CANInterface

Devices

ECU1ECU1

ECU2ECU2

ECU3ECU3

ECU4ECU4

(CAN1)

(CAN2-3)

(CAN4)

(CAN5)

Real-Time Tasks & I/O

Handling

... ...

Low-criticalitytasks & non-real-time I/O

SSD NICNIC

UP S

quare

d S

BC

● Boomerang tuned pipe path (1) spans Quest + Linux + USB-CAN ● Boomerang tuned pipe path (2) spans Quest + USB-CAN

16

Boomerang: Asynchronous Results

● Boomerang meets communication timing guarantees● A Linux SMP (multicore) OS with real-time extensions cannot perform I/O

predictably

Pipeline 1 (Asynchronous, no loss) Pipeline 2 (Asynchronous, no loss)

17

Boomerang: Loss-tolerant Results

● Boomerang guarantees Pipeline 1 (3.5% loss) and Pipeline 2 (0% loss)● Linux SMP fails Pipeline 1 (55% loss) and Pipeline 2 (50% loss)

Pipeline 1 (Asynchronous, ≤20% loss) Pipeline 2 (Asynchronous, ≤20 % loss)

18

Boomerang: Synchronous Results

Pipeline 1 (FIFO buffering) Pipeline 2 (FIFO buffering)

19

ACRN: Partitioning Hypervisor

vLAPIC

Service OS

ACRN hypervisor

User OS

Virtio Front End

Driver

TCP Stack

OtherDeviceDrivers

Kernel Space

User Space

Socket API

Virtio API

Remote MMAP

Notification

VMExit

ACRN Service Module

Remote MMAP

Notification

Hardware

Device Drivers

Virtio BEDriver

TCP Stack Virtio Back EndService (VBS-K)

Virtio Back End Service (VBS-U)

ACRN Device Model

Socket API

App App App

SharedMemoryBuffer

20

ACRN: Results

Pipeline 1 (Asynchronous, no loss) Pipeline 2 (Asynchronous, no loss)

● ACRN generally worse than Linux SMP● Neither as good as Boomerang (previously shown)

21

Conclusions

● Boomerang I/O system built for Quest-V partitioning hypervisor

– Supports composable tuned pipes between guests

– Empowers Non-RT OS with RT capabilities

– Automatically tunes VCPU parameters

– Guarantees E2E throughput, delay & loss

– Outperforms Linux SMP (RT-PREEMPT + DEADLINE) & ACRN