“Unlimited” Event Channels

David Vrabel

24 October 2013

What are Event Channels?

I Paravirtual interrupts

I Edge triggered

I Bidirectional

I Directed at a single VCPU

24 October 2013 “Unlimited” Event Channels 2 / 13

How do Event Channels Work?

Domain BDomain A

Shared MemoryNotify

Set Pending Upcall

1

2 3

4 Call Handlers

Xen

24 October 2013 “Unlimited” Event Channels 3 / 13

Original (2-Level) Design

Masked

Pending ...

... ...

... ...

...

...Selector

Shared Memory

Per-VCPU Shared Memory

4096 bits

64 bits

64 bits

1

0 1 0 0 0 1 0

24 October 2013 “Unlimited” Event Channels 4 / 13

Problems

I Too few

I No priorities

I Unfair

24 October 2013 “Unlimited” Event Channels 5 / 13

Additional Requirements

I Identical ABI between 32-bit and 64-bit guests

I Reasonable memory usage

I Easily extensible

24 October 2013 “Unlimited” Event Channels 6 / 13

New (FIFO-based) Design

P M L LINK

1

1

1

1

1

1

6

0

3

Up to 132,072 words

1

...

HEAD

0

0

0

READY

Per-VCPU shared memory Shared memory

Port 0(Reserved)

1

2

3

4

5

6

7

Bit 15 01

Priority 0

1

15

1Bit 31 030 29 16

...

24 October 2013 “Unlimited” Event Channels 7 / 13

Raising an Event

function raise(q, p)

E[p].pending = 1 if not E[p].masked and not E[p].linked E[p].linked = 1 if T[q] != p linked = link(T[q], p) else linked = false if not linked C.head[q] = p T[q] = p

1. Set pending

2. Unmasked and not in list? Add to list

3. Set LINK of tail or Set HEAD in control block

4. Advance tail

24 October 2013 “Unlimited” Event Channels 8 / 13

Handling an Event

function handle_one_event(q)

p = H[q] if p == 0 p = C.head[q] link = unlink(p) H[q] = link if E[p].pending and not E[p].masked handle(p)

1. Get local head

2. List was empty? Get new HEAD from control block

3. Clear LINKED and LINK

4. Advance local head

5. Pending and unmasked? Handle IRQ

24 October 2013 “Unlimited” Event Channels 9 / 13

Event Channel Fairness

0

500

1000

1500

2000

2500

3000

75 80 85 90

Late

ncy

(ns)

Event Channel Port

2-Level

FIFO-based

24 October 2013 “Unlimited” Event Channels 10 / 13

Current Status

I Xen support in 4.4

I Linux support in 3.14

24 October 2013 “Unlimited” Event Channels 11 / 13

Further Work

I Investigate how priorities may be used

I Add support for limiting number of event channels to other toolstacks (XAPI,libvirt)

I Add support in other OSes

I Xen event lock scalability

I Linux PIRQ setup/teardown refactoring

24 October 2013 “Unlimited” Event Channels 12 / 13

Questions?

I Design Documenthttp://xenbits.xenproject.org/people/dvrabel/event-channels-F.pdf

24 October 2013 “Unlimited” Event Channels 13 / 13