THINC: A Virtual Display Architecture for Thin-Client

Computing

Ricardo A. Baratto, Leonard N. Kim, Jason NiehNetwork Computing Laboratory

Columbia University

isolation...

Source: Internet Mapping Project (http://research.lumeta.com/ches/map/)

...connectivity

dis-integration of the computer

networkstorage

clusters and gridcomputing

remote display

display updates

input

benefits

ubiquitous access

remote collaboration

online help

stateless client

application processingand data

secure server room

thin clients

existing systems

existing performance problem

0% 20% 40% 60% 80% 100%

PC

ICA

RDP

GoToMyPC

X

NX

VNC

Sun Ray

audio/video quality

LAN WAN

THINC

virtual display architecture

high performance remote display

transparent operation

● system architecture● display protocol● translation● delivery

system architecture

applications

windowsystem

devicedriver

framebuffer

applications

windowsystem

devicedriver

framebuffer

high-level requests

interception and redirection

✗ stateful client hurts mobility

✗ app – window system synchronization

applications

windowsystem

framebuffer

devicedriver

raw pixels

high-level requests

interception and redirection

✗ lose semantics: difficult to encode

✗Bandwidth intensive

applications

windowsystem

virtual devicedriver

display updates

input events

virtual display architecture

Standard Interface

applications

windowsystem

virtual devicedriver

benefits

Leverageexisting

technology

applications

windowsystem

virtual devicedriver

benefits

virtual devicedriver

display updates

Simple, low-levelprotocol

benefits

virtual devicedriver

display updates

input events

Simple, statelessclient

benefits

display protocol

Inspired by Sun Ray protocol

2D Primitives● copy● solid and tile fill● bitmap fill● raw

two key problems

how do we translate from application commands

to the display protocol?

how and when do we send display updates?

translation

use and preserve semantic information for efficient translation

● use semantic information when doing translation

translation

use request semantics to generate update

req: fill window W, color C

window

system

req: fill [x,y,w,h] color C

THINCupdate: solid fill [x,y,w,h] color C

✔use semantic information when doing translation

● preserve semantic information throughout the system

translation

preserving semantics: offscreen rendering

draw

offscreen regions

abcde copy abcde

display

offscreen rendering (cont)offscreen region

command log

merge, clip, and discard commands as needed

using and preserving semantics: video

● reuse existing hardware acceleration application interfaces

● YUV (luminance-chrominance) color space– format independence– client hardware acceleration (scaling

for free)

delivery

maximize interactive response of the system

delivery

● transmit updates as soon as possible● merge, clip, and discard updates as

needed

shortest remaining size first scheduler

client buffer

C1

C2

C3

...Cn

real time

.

.

.

queue 1

queue p

cmdsize

implementation

● X/Linux server– ongoing: windows server

● X/Linux, windows, PDA, Java clients

experimental results

● web and video performance– comparison to existing systems– Internet 2 sites around the globe

“ ”

LANWAN

802.11g

web browsing performance

0 0.5 1 1.5 2 2.5 3

PC

ICA

RDP

GoToMyPC

X

NX

VNC

Sun Ray

THINC

per web page latency (s)

Desktop LAN Desktop WAN 802.11g PDA

a/v playback quality

0% 20% 40% 60% 80% 100%

PC

ICA

RDP

GoToMyPC

X

NX

VNC

Sun Ray

THINC

audio/video quality

Desktop LAN Desktop WAN 802.11g PDA

NY

MAPA

MN

NMCA

CA IEFI

KR

PR

Internet2 web browsing performance

0

0.5

1

1.5

2

ny pa ma mn nm ca can pr ie fi kr

location

pa

ge

la

ten

cy (

s)

Internet2 a/v playback quality

0%

20%

40%

60%

80%

100%

ny pa ma mn nm ca can pr ie fi kr

location

pla

yb

ac

k q

ua

lity

conclusions

THINC:● virtual display architecture transparently

leverages existing display infrastructure● efficient translation by using and preserving

semantic information from display request● delivery mechanisms increase

responsiveness of the system

for more info...

http://www.ncl.cs.columbia.edu

backup

audio

applications

OS

virtual audiodriver

audiodaemon

audio data

audio data

Experimental Results

Web Browsing Data Transfer

0 50 100 150 200 250

PC

ICA

RDP

GoToMyPC

X

NX

VNC

Sun Ray

THINC

per web page data (K B)

Desktop LAN Desktop WAN 802.11g PDA

A/V Data Transfer

0 50 100 150 200 250

PC

ICA

RDP

GoToMyPC

X

NX

VNC

Sun Ray

THINC

audio/video data transferred (MB)

Desktop LAN Desktop WAN 802.11g PDA

?

server-resized updates

offscreen drawing

draw

offscreen regions

abcde copy abcde

display

offscreen region

command queue

command queues

client

queue

1

2

33

2

1

copy onscreen

how?

applications clienthardware

caps

video

YUV

● Standard hardware interface● Format independence● Hardware acceleration (fullscreen for

free!)

how we deliver updates

displayupdates

client buffer

C1

C2

C3...C

n

future work

● 3D and high-end user interfaces● remote device access

http://www.ncl.cs.columbia.edu

old slides

THINC

virtual display architecturefor high performance

remote display

implementation

● server: X/Linux (Windows in progress)● clients: Linux (Windows, PDA, Java in

progress)

Web Browsing Performance

Audio/Video Performance

LBX

X proxies

                                            

... and a PC

Configurations

LAN Desktop

WAN Desktop

802.11g PDA

PlanetLab

applications

windowsystem

displaydriver

framebuffer

● ICA, RDP: Rich display protocol● SunRay: Ultra thin-client

Studies have shown performance problems

system architecture as important

as protocol and encoding

goals

● minimize latency● simple and portable● transparent operation

experimental results

● up to 4.8 times better web browsing performance

● up to two orders of magnitude better audio/video playback quality

applicationrequests

translate

commands

deliver

displayupdates

THINC

applications

display pipeline

windowsystem

displaydriver

framebuffer

applications

windowsystem

displaydriver

framebuffer

client/server partitioning

✗ stateful client hurts mobility✗ app – window system synchronization

applications

windowsystem

displaydriver

framebuffer

✗ “Blind encoding”

basic static translation

DrawAPI

standarddevicedriver

commands

THINCcommands

video: first-class citizen

THINC

● high performance remote display● LAN and WAN environments● transparent operation in exisiting desktop

systems● full screen, full motion audio/video playback

CA IE

FI

KR

PR