Scalability of network environments
Esau Libo and DamonEsau Libo and Damon Dept. of Computer ScienceDept. of Computer Science University College London University College London
Purpose of this seminar Describe the overall problem of scalability in virtual Describe the overall problem of scalability in virtual
environment and two approaches (MASSIVE & RING)environment and two approaches (MASSIVE & RING) in some detail. Discuss some applications.in some detail. Discuss some applications.
The overallproblem
ApproachesMASSIVE & RING
Discussion
Introduction
Introduction
ReviewComputers and networks strengthen an Computers and networks strengthen an Collaborative Virtual Environment (CVE)Collaborative Virtual Environment (CVE)
Computer have limitations and differing Computer have limitations and differing capabilitiescapabilities Network technologies like Ethernet, ATM and Network technologies like Ethernet, ATM and wirelesswireless
Each technology with bandwidth, delay Each technology with bandwidth, delay characteristics and reliability.characteristics and reliability.
Where we are
Approaches
Discussion
The overall problem
Scaling
Goals of CVE
Goals of CVEGoals of CVE
Introduction
ReviewSharing the use of information is the primary goal.Sharing the use of information is the primary goal.
Design to run over a set of heterogeneous Design to run over a set of heterogeneous computers and networks.computers and networks.
Maximise responsiveness and scalability while Maximise responsiveness and scalability while minimising latency (Roberts)minimising latency (Roberts)
Where we are
Approaches
Discussion
The overall problem
Scaling
Goals of CVE
ScalingScaling
Scaling
Introduction
ReviewScaling allows the amount of information in the Scaling allows the amount of information in the environment with number of users, to increase, environment with number of users, to increase, without reducing the fidelity of experience to any without reducing the fidelity of experience to any one user.one user.
Achieved by balancing each individuals need for Achieved by balancing each individuals need for information with what can be achieved.information with what can be achieved.
Where we are
Approaches
Discussion
The overall problem
Goals of CVE
Space structuring
Approaches
Discussion
The overall problem
。。。。。。。。。
Review
Space structuring
Division of virtual space
a) Separate Serversb) Uniform geometrical structurec) Free geometrical structured) User-centered dynamic structure
Where we are
Approaches
Discussion
The overall problem
。。。。。。。。。
Review
Network Topologies
Peer-to-peer UnicastPeer-to-peer MulticastClient/ServerMulti-Server Topology
Topologies
Space structuring
Where we are
Approaches
Discussion
The overall problem
。。。。。。。。。
Review
Definition of Scalability
Scalability
Space structuring
Topologic
Definition
…
…
• Scalability Scalability – How well a solution to some problem will How well a solution to some problem will
work when the size of the problem increases.work when the size of the problem increases.– For example…For example…complexity of O(N), O(N^2)complexity of O(N), O(N^2)
Where we are
Approaches
Discussion
The overall problem
。。。。。。。。。
Review
Scalability
Scalability
Space structuring
Topologic
…
…
Factors
Determining factors:
• graphical complexity • behavioral complexity• number of simultaneous participants
Where we are
Approaches
Discussion
The overall problem
。。。。。。。。。
Review
Scalability
Scalability
Space structuring
Topologic
…
…
Bottlenecks
Bottlenecks:
• Network Server• Dial-up Modems• PCs of users
Where we are
Approaches
Discussion
The overall problem
。。。。。。。。。
Review
Scalability
Scalability
Space structuring
Topologic
…
Solution:• limited information exchange
Example:• audio: objects too far away to hear• visual: objects behind the user
…
Solution
Where we are
Two approaches (MASSIVE & RING)
The overallproblem
ApproachesMASSIVE & RING
Discussion
• facilitating scalabilityfacilitating scalability – it is based on the concept of aura.it is based on the concept of aura.
• controlling spatial interactioncontrolling spatial interaction – it deals with the control of interaction or it deals with the control of interaction or
communication between two objects once communication between two objects once they become aware of each other through they become aware of each other through aura collision .aura collision .
…
Spatial model
…
MASSIVE
Where we are
Discussion
The overallproblem ApproachesSpatial model of interaction
-major components
Key concepts: Key concepts: • medium medium : communication type: communication type• aura aura : subspace in which interaction can occur: subspace in which interaction can occur• awarenessawareness:: quantifies one object’s significance to another object (in a quantifies one object’s significance to another object (in a
particular medium) particular medium) • focus focus : represents an observing object’s interest: represents an observing object’s interest• nimbus nimbus : represents an observed object’s wish to be seen: represents an observed object’s wish to be seen• adapters adapters : can modify an object’s auras, focus, and nimbus : can modify an object’s auras, focus, and nimbus
Awareness and the Spatial Model of Interaction (1)
Where we are
Discussion
The overallproblem Approaches
…
Spatial model
…
MASSIVE
Awareness and the Spatial Model of Interaction (2)
Where we are
Discussion
The overallproblem Approaches
…
Spatial model
…
MASSIVE
A example about watching TV
• implicitlyimplicitly through spatial actions such as through spatial actions such as moving and turning, e.g. when turning moving and turning, e.g. when turning around my focus and nimbus follow me, around my focus and nimbus follow me, resulting in an adjustment of my awareness resulting in an adjustment of my awareness of nearby objects of nearby objects
• ExplicitlyExplicitly by choosing from among by choosing from among different shapes and sizes, e.g. switching different shapes and sizes, e.g. switching between narrow or wide focus between narrow or wide focus
• via via adapter objectsadapter objects which transform which transform combinations of aura, focus and nimbus combinations of aura, focus and nimbus
Where we are
Discussion
The overallproblem Approaches
…
Spatial model
…
MASSIVE
Spatial model of interaction- Three Three manipulationmanipulation ways ways
• MASSIVE MASSIVE is the “Model, Architecture and System for is the “Model, Architecture and System for Spatial Interaction in Virtual Environments”, an Spatial Interaction in Virtual Environments”, an experimental distributed virtual reality system intended to experimental distributed virtual reality system intended to support collaborative activity support collaborative activity
• It has been driven by key requirement:It has been driven by key requirement: Scale - supporting as many simultaneous users as possibleScale - supporting as many simultaneous users as possible
• It uses a spatial model of interactionIt uses a spatial model of interaction
Where we are
Discussion
The overallproblem Approaches
…
MASSIVE
Functionality
…
MASSIVE functionality- IntroductionIntroduction
User-level featuresUser-level features• It is a multi-user distributed V.R. system. It is a multi-user distributed V.R. system. • There are textual, graphical and audio client programs There are textual, graphical and audio client programs
(usable in any combination) allowing users to communicate (usable in any combination) allowing users to communicate by graphical gestures, typed messages or real-time by graphical gestures, typed messages or real-time packetised audio. packetised audio.
• Text users may interact with graphical users and vice-versa. Text users may interact with graphical users and vice-versa. • All media are controlled by aura, focus and nimbus. All media are controlled by aura, focus and nimbus. • It includes adaptor objects (e.g. conference table, podium). It includes adaptor objects (e.g. conference table, podium). • There may be any number of worlds with portals to move There may be any number of worlds with portals to move
between worlds. between worlds. • Each process obtains and uses only (spatially) local Each process obtains and uses only (spatially) local
information. information.
• New media can be added without affecting the core system.New media can be added without affecting the core system. Where we are
Discussion
The overallproblem Approaches
…
MASSIVE
Functionality
…
MASSIVE functionality- key feature for scalabilitykey feature for scalability (1)(1)
Lower-level featuresLower-level features• keyword-based interface trading for coordinating keyword-based interface trading for coordinating
client programs for different media. client programs for different media. • spatial interface trading based on aura collisions. spatial interface trading based on aura collisions. • medium-independent peer connections medium-independent peer connections
implementing focus, nimbus and awareness. implementing focus, nimbus and awareness. • integrated support for focus, nimbus and aura integrated support for focus, nimbus and aura
adaptors.adaptors.
MASSIVE functionality- key feature for scalabilitykey feature for scalability (2)(2)
Where we are
Discussion
The overallproblem Approaches
…
MASSIVE
Functionality
…
MASSIVE functionality- A example of virtual conferenceA example of virtual conference
Graphics user’s interface Text user’s interface
The interactions in MASSIVE are a mixture of The interactions in MASSIVE are a mixture of client-server (between objects and the trader or client-server (between objects and the trader or aura manager) and peer-to-peer (between objectsaura manager) and peer-to-peer (between objectsin the world). Most of the interactions are in the world). Most of the interactions are bidirectional, being symmetrical in the case ofbidirectional, being symmetrical in the case ofpeer connections and involving asynchronouspeer connections and involving asynchronous notifications in the case of client server notifications in the case of client server connectionsconnections
Where we are
Discussion
The overallproblem Approaches
MASSIVE
…
MASSIVE Architecture- IntroductionIntroduction
Architecture
…
• two conditionstwo conditions of interaction between of interaction between objects in the virtual world objects in the virtual world – compatible interfaces compatible interfaces – objects must become sufficiently proximate objects must become sufficiently proximate
as determined by their auras as determined by their auras
Where we are
Discussion
The overallproblem ApproachesMASSIVE Architecture
- Aura and spatial trading
MASSIVE
…
Architecture
…
Where we are
MASSIVE Architecture- how spatial trading operates Discussion
The overallproblem Approaches
MASSIVE
…
Architecture
…
RING Architecture(1)
• RING represents a virtual RING represents a virtual environment as a set of independent environment as a set of independent entities each of which has a entities each of which has a geometric description and a behavior.geometric description and a behavior.
Where we are
Discussion
The overallproblem Approaches
MASSIVE
Architecture
…
RING
……
…
…
• Entity:Entity: Multi-user interaction is supported Multi-user interaction is supported
by matching user actions to entity by matching user actions to entity updatesupdates
• Client-Server designClient-Server design Reliability, Redundancy, Load-Reliability, Redundancy, Load-
BalacingBalacing
Where we are
Discussion
The overallproblem Approaches
MASSIVE
……
…
RING Architecture(2)
Architecture
…
RING
…
• Entity, Client, RINGEntity, Client, RING
• Every RING entity is Every RING entity is managed by exactly one managed by exactly one client workstationclient workstation
• Communication between Communication between clients is managed by serversclients is managed by servers
Where we are
Discussion
The overallproblem Approaches
MASSIVE
RING
……
…
RING Architecture(3)
Architecture
…
…
RING Mechanism
• Message FilteringMessage Filtering Area of Interest (exploit communication locality)Area of Interest (exploit communication locality)
Regular subdivision (NPSNET), proximity (DIVE)Regular subdivision (NPSNET), proximity (DIVE) Pre-determined inter-cell occlusion (SPLINE)Pre-determined inter-cell occlusion (SPLINE) Visibility culling (RING)Visibility culling (RING)
Explicitly registering interest( NPSNET)Explicitly registering interest( NPSNET) Dead Reckoning (NPSNET, PARADISE) Dead Reckoning (NPSNET, PARADISE)
messages: O(Nmessages: O(N22) ) O(NP) O(NP)
• Network topology: IP-Multicast +Message Network topology: IP-Multicast +Message FilteringFiltering
Where we are
Discussion
The overallproblem Approaches
MASSIVE
RING
……
…
Architecture
Character
…
Partition Regions(1)
Partition Regions(2)
RING Algorithm
• Changing "periodic" update Changing "periodic" update messages when entities cross cell messages when entities cross cell boundaries.boundaries.
• Using cell visibility "look-ups“Using cell visibility "look-ups“ Each cell Each cell a multicast group a multicast group Sends messages to its “cell”Sends messages to its “cell” Receive messages from its interest zone Receive messages from its interest zone
(including neighbouring cells)(including neighbouring cells)
Where we are
Discussion
The overallproblem Approaches
MASSIVE
RING
……
…
Architecture
Character
…
Regions
VIE W PO INT
VIS IBLECELLS
INVIS IBLECELLS
Visibility Culling• Occlusion: indoor (rooms,buildings), outdoor(‘fog of war’)Occlusion: indoor (rooms,buildings), outdoor(‘fog of war’)• Visibility culling carried out with:Visibility culling carried out with:
– ‘‘Potentially visible sets’ of cells (pre-computed)Potentially visible sets’ of cells (pre-computed)– Temporal Bounding Volumes, Update Free RegionsTemporal Bounding Volumes, Update Free Regions
ClientServer
object A
object B
object C
viewpoint
Different RING systems
• 1. Peer to Peer 1. Peer to Peer O(NP), scale finitelyO(NP), scale finitely• 2. Client-Server ( single server, 2. Client-Server ( single server, staticstatic)) scale finitelyscale finitely• 3. Client-Server ( any server, 3. Client-Server ( any server, regionalregional)) scale infinitely scale infinitely region finite region finite• 4. Other systems4. Other systems
Where we are
Discussion
The overallproblem Approaches
MASSIVE
……
…
…
RING
…
Compare
SystemSystemDesignDesign
Number Number of of
ServicesServices
ClientClient ServerServerInputInput OutputOutput InputInput OutputOutput
From From ServerServer
To To ServerServer
From From ClientClient
From From ServerServer
TotalTotal To To ClientClient
To To ServerServer
TotalTotal
StaticStatic(A)(A)
22 4.254.25 0.500.50 46.846.8 44.044.0 90.890.8 290.5290.5 44.044.0 334.5334.5
44 5.095.09 0.460.46 24.124.1 64.464.4 88.588.5 148.6148.6 64.764.7 213.3213.3
88 3.293.29 0.450.45 12.012.0 70.470.4 82.382.3 71.371.3 71.571.5 142.8142.8
1616 4.164.16 0.370.37 6.16.1 74.774.7 80.880.8 37.237.2 75.875.8 113.0113.0
RegionalRegional(B)(B)
22 4.674.67 0.420.42 48.248.2 1.91.9 50.250.2 270.7270.7 2.02.0 272.7272.7
44 3.893.89 0.530.53 24.224.2 4.34.3 28.528.5 153.0153.0 4.34.3 157.3157.3
88 4.154.15 0.460.46 12.312.3 3.63.6 15.815.8 83.583.5 3.63.6 87.187.1
1616 4.414.41 0.430.43 5.95.9 3.43.4 9.39.3 37.537.5 3.53.5 41.041.0
Table: Average message processing rates (messages per second)
Experimental results
• Static system design:Static system design:total number of server-server total number of server-server messages increases, sub-linearmessages increases, sub-linear
• Regional system design:Regional system design:inter-visibility decreaseinter-visibility decrease server- server-server messaging reduceserver messaging reduce
Where we are
Discussion
The overallproblem Approaches
MASSIVE
……
…
…
RING
…
Compare
Experimental Conclusion
• Different network characteristics and Different network characteristics and different system designs can different system designs can significantly affect the message significantly affect the message processing rates requiredprocessing rates required
• Hierarchical system (regional) scales Hierarchical system (regional) scales better than hierarchical system better than hierarchical system (static)(static)
Where we are
Discussion
The overallproblem Approaches
MASSIVE
……
…
…
RING
…
Compare
RING system advantage
1. Less storage, processing, and 1. Less storage, processing, and network network bandwidth requirements;bandwidth requirements; very dynamic and complex message very dynamic and complex message
processing may processing may be performed by servers be performed by servers
2. High-level management2. High-level management
Where we are
Discussion
The overallproblem Approaches
MASSIVE
……
…
…
RING
…
Compare
RING system disadvantage
1. Extra latency1. Extra latency
2. Currently support only visual 2. Currently support only visual interactions interactions
Discussion
The overallproblem Approaches
MASSIVE
……
…
…
RING
…
Compare
Where we are
Discussion
The overallproblem
ApproachesMASSIVE & RING
Discussion
Discussion• Multiplayer computer games Multiplayer computer games
Discussion
The overallproblem Approaches
Quake includes a multi-player mode to play over LAN or the Internet with or against other humans. The network play uses a client/server model, where the actual game runs on the server only and all players "log in" there to participate. Depending on the client's specific route to the server, different clients will get different ping times. The lower your latency (ping time), the smoother your in-game motions, and the easier it is to accurately aim and score. Someone playing on the server PC gets a substantial advantage due to essentially zero lag
Discussion
• Will cyberspace as described by William Will cyberspace as described by William
Gibson or Neal Stephenson ever be builtGibson or Neal Stephenson ever be built
Discussion
The overallproblem Approaches