23/4/2223/4/22 11
Portlets and Web Services for Collaboration and Videoconferencing
PTLIU Laboratory for Community GridsGeoffrey Fox, Hasan Bulut, Bryan Carpenter,
Marlon Pierce, X. Qiu, Ahmet Uyar, Wenjun WuIndiana University, Bloomington IN 47404
NESC 16 July 2003
23/4/2223/4/22 22
Abstract We define interoperable framework called XGSP (XML
based General Session Protocol) based on Web services technology for creating and controlling sessions
We developed a common dynamic messaging environment (NaradaBrokering) for Grid and collaboration applications
Based on the web-services framework and NaradaBrokering messaging environment, we are developing Global Multimedia Collaboration System (Global-MMCS) with Beihang University China
We use MVC (Model View Control) paradigm to develop Web Service model/data – Messaging Control -- Jetspeed style aggregation – portlet style view
23/4/2223/4/22 33
Collaboration and Web Services Collaboration has
a) Mechanism to set up members (people, devices) of a “collaborative sessions”
b) Shared generic tools such as text chat, white boards, audio-video conferencing
c) Shared applications such as Web Pages, PowerPoint, Visualization, maps, (medical) instruments ….
b) and c) are “just shared objects” where objects could be Web Services but rarely are at moment
• We can port objects to Web Services and build a general approach for making Web services collaborative
a) is a “Service” which is set up in many different ways (H323 SIP JXTA are standards supported by multiple implementations) – we should make it a WS
23/4/2223/4/22 44
Shared Event Collaboration All collaboration is about sharing events defining state changes
• Audio/Video conferencing shares events specifying in compressed form audio or video
• Shared display shares events corresponding to change in pixels of a frame buffer• Instant Messengers share updates to text message streams• Microsoft events for shared PowerPoint (file replicated between clients) as in
Access Grid Finite State Change NOT Finite State Machine architecture Using Web services allows one to expose updates of all kinds as
messages• “Event service” for collaboration is similar to Grid notification service and we
effectively define SDE’s (service data elements) in OGSI Group (Session) communication service is needed for the delivery of
the update events• Using Event Messaging middleware makes messaging universal
23/4/2223/4/22 55
Portals and Web Services Web Services allow us to build a component model for resources. Each resource naturally has a user interface (which might be
customized for user) Web Service <--> Portlet Natural to use a component model for portal building displayed
web page from collection of portlets• So can customize each portlet and customize which portlets you want• Need to redirect portlets to cell-phones etc.
What is form of MVC (Model-View-Controller) paradigm What happens if “real applications” like PowerPoint are
formulated as Web Services Jetspeed uses HTML to build a portal – as well as Java, SVG
(Scalable Vector Graphics) is interesting
23/4/2223/4/22 66
XGSP MCU Architecture
SIP H323 Access Grid Native XGSPAdmire
Gateways convert to uniform XGSP Messaging
High Performance (RTP)and XML/SOAP and ..
Media ServersFilters
Session ServerXGSP-based Control
NaradaBrokeringAll Messaging
Use Multiple Media servers to scale to many codecs and manyversions of audio/video mixing
NB Scales asdistributed
WebServices
23/4/2223/4/22 77
Global-MMCS 2.0 (1) XGSP MCU We are building an open source protocol independent Web
Service “MCU” which will scale to an arbitrary number of users and provide integrated thousands of simultaneous users collaboration services.
We will deploy it globally and hope to test with later this year. The function of A/V media server will be distributed using
NaradaBrokering architecture.• Media Servers mix and convert A/V streams
Open XGSP MCU based on the following open source projects• openh323 is basis of H323 Gateway• NIST SIP stack is basis of SIP Gateway• NaradaBrokering is open source messaging from Indiana• Java Media Framework basis of Media Servers
23/4/2223/4/22 88
WSDisplay
WSViewer
WS Display
WS Viewer
Event(Message)
Service
Master
WSDisplay
WS Viewer
Web Service MessageInterceptor
Collaboration as a WSSet up Session with XGSP
Application orContent source
WSDL
Web Service
FI
U
O
FI
R
O
Shared Output Port Collaboration
OtherParticipants
Text ChatWhiteboardMultiplemasters
23/4/2223/4/22 99
WSDisplay
WSViewer
WS Display
WS ViewerEvent
(Message)Service
Master
WSDisplay
WS Viewer
Collaboration as a WSSet up Session with XGSP
WebServic
e
FI
U
O
FI
R
O
Shared Input Port (Replicated WS) Collaboration
OtherParticipants
WebServic
e
FI
U
O
FI
R
O
WebServic
e
FI
U
O
FI
R
O
23/4/2223/4/22 1010
XGSP Conference Control Framework Components
User session management • User session management supports user sign-in, user
create/terminate/join/leave/invite-into XGSP sessions. Application Session Management
• XGSP application session management provides the services to A/V and data application endpoints and communities, controlling multipoint A/V RTP and data channels.
Floor Control • Floor control manages the access to shared collaboration
resources.
23/4/2223/4/22 1111
XGSP Application Session Management XGSP signaling protocol for
• H.323 signaling protocols (H.225, H.245) • SIP signaling protocol ( Invite, Bye Message )• Access Grid ( Unicast VIC & RAT ) Join XGSP Session, Leave XGSP Session, Invite into XGSP Session, Expel from XGSP Session
Activate the XGSP session• the XGSP session server will link all the “rooms” in the session
together by connecting multipoint A/V and data channels from different communities to the XGSP A/V Media and Data Channel Services.
Link/Disconnect XGSP SubSession
23/4/2223/4/22 1212
XGSP Floor Control XGSP should provide:
• Floor control primitives, including: request floor, release floor, grant floor, cancel floor, remove floor request
• mediator-controlled floor control: to support the mediator control policy
• Collaboration applications have to define their own roles in the XGSP registration so that the mediator could assign the role of the application to each user.
for example, a shared PowerPoint application should define master/slave role.
23/4/2223/4/22 1313
NaradaBrokering Based on a network of cooperating broker nodes
• Cluster based architecture allows system to scale to arbitrary size
Originally designed to provide uniform software multicast to support real-time collaboration linked to publish-subscribe for asynchronous systems.
Now has five major core functions• Message transport (based on performance measurement) in
heterogeneous multi-link fashion• General publish-subscribe including JMS & JXTA and
support for RTP-based audio/video conferencing • Distributed XML data-base using P/S XPATH metaphor• Filtering for heterogeneous clients• Federation of multiple instances of Grid services as
illustrated by JXTA peer-group linkage
23/4/2223/4/22 1414vic and RealVideo views of multiple streams
23/4/2223/4/22 1515Polycom view of multiple video streams
23/4/2223/4/22 1616
Performance Test : GlobalMMCS1.0 We conducted extensive performance tests on audio
and video servers. Video:
• The test shows that our video server is capable of supporting 300 clients if there is only one video sender.
• Video Server Machine : 1.2GHz Intel Pentium III dual CPU, 1GB MEM, RedHat Linux 7.3
Audio: • Our tests show that audio server can support 5 concurrent
sessions (250 participants in total) without any packet droppings. • Audio Server Machine: 2.5GHz Pentium 4 CPU, 512MB memory,
Windows XP machine Scale with logarithmic Broker network
23/4/2223/4/22 1717
1
2
3
4
5
6
7
8
9
1000 1500 2000 2500 3000 3500 4000 4500 5000
Tran
sit D
elay
(M
illise
cond
s)
Message Payload Size (Bytes)
Transit delay for message samples in NaradaBrokering Different communication hops - Internal Machines
hop-2hop-3hop-5hop-7
Sender/receiver/broker - (Pentium-3, 1 GHz, 256 MB RAM). 100 Mbps LAN. JDK-1.3, Red Hat Linux 7.3
23/4/2223/4/22 1818
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
1000 1500 2000 2500 3000 3500 4000 4500 5000
Sta
ndar
d D
evia
tion
(Milli
seco
nds)
Message Payload Size (Bytes)
Standard Deviation for message samples in NaradaBrokering Different communication hops - Internal Machines
hop-2hop-3hop-5hop-7
23/4/2223/4/22 1919
0
50
100
150
200
250
300
350
400
450
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Del
ay (M
illis
econ
ds)
Packet Number
Average delays/packet for 12 (of the 400 total) video-clients. NaradaBrokering Avg=80.76 ms, JMF Avg=229.23 ms
NaradaBrokering-RTP JMF-RTP
Comparison between the performance of NaradaBrokering and JMF
23/4/2223/4/22 2020
0
5
10
15
20
25
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Jitte
r (M
illis
econ
ds)
Packet Number
Average jitter/packet for 12 (of the 400 total) video clients. NaradaBrokering Avg=13.38 ms, JMF Avg=15.55 ms
NaradaBrokering-RTP JMF-RTP
Comparison between the performance of NaradaBrokering and JMF
23/4/2223/4/22 2121
Global-MMCS 2.0 (2) Portlets Collaboration clients will be built into portlets by
creating Java Applet or ActiveX controls for the non-HTML clients and adding them into HTML pages.
A collaboration portlet opens local services for XGSP application session management and floor control. • Node Manager portlet invoke the service to control local portlets
Apache Jetspeed seems good open source technology supporting this model
Portlets such as Access Grid portlet (really a VIC portlet) can be reused by Grid Portal Developers
23/4/2223/4/22 2222
Unicast AG Portlet
23/4/2223/4/22 2323
Multicast Multi-stream AG Portlet Java applet supports
multicast AG with multiple streams
In Jetspeed, easiest to have fixed size but this doesn’t fit well natural range of 1-20 separate streams
23/4/2223/4/22 2424
Collaborative SVG Web Service SVG is W3C 2D Vector Graphics standard and is interesting for
visualization and as a simple PowerPoint like application• Further SVG is built on W3C DOM and one can generalize results
to all W3C DOM-based applications (“all” in future?) Apache Batik SVG is Java and open source and so it is practical
to modify it to explore• Real Applications as a Web Service• Collaboration as a Web Service• MVC model and web services with implications for portlets
We intend to use NaradaBrokering and XGSP to control collaboration; support PDA Cell-phone and desktop clients; restructure Batik as MVC Web Service• Good progress in all areas see • http://www.svgarena.org for SVG Games• http://grids.ucs.indiana.edu/ptliupages/projects/carousel/ for PDA
04/22/2304/22/23 2525
Collaborative SVG Chess Collaborative SVG Chess Game in Batik BrowserGame in Batik Browser
Players
Observers
04/22/23 26Figure 3 Architecture of collaborative SVG browser on PC
XGSPSession control
Server
Nar
adaB
roke
ring
Even
t (M
essa
ge) S
ervi
ce In
fras
truc
ture
•••
Master clientSVG browser 1F
I
R
O
Other clientSVG browser 2F
I
R
O
Other clientSVG browser nF
I
R
O
Control to/from
all SVG browsers in the collaborative session
Data from master client
Control to/from XGSP
Data to other clients
Control to/from XGSP
Data from master client
Control to/from XGSP
Figure 3 Architecture of collaborative SVG browser on PC
XGSPSession control
Server
XGSPSession control
Server
Nar
adaB
roke
ring
Even
t (M
essa
ge) S
ervi
ce In
fras
truc
ture
•••
Master clientSVG browser 1F
I
R
O
Master clientSVG browser 1F
I
R
O
Other clientSVG browser 2F
I
R
O
Other clientSVG browser 2F
I
R
O
Other clientSVG browser nF
I
R
O
Other clientSVG browser nF
I
R
O
Control to/from
all SVG browsers in the collaborative session
Control to/from
all SVG browsers in the collaborative session
Data from master client
Control to/from XGSP
Data from master client
Control to/from XGSP
Data to other clients
Control to/from XGSP
Data to other clients
Control to/from XGSP
Data from master client
Control to/from XGSP
Data from master client
Control to/from XGSP
Architecture of collaborative SVG browser on PC
04/22/2304/22/23 2727
Classic MVC ParadigmClassic MVC ParadigmModel View Controller
Model
Controller
ViewMouse eventKeyboard events
Figure MVC Model
Display
04/22/23 28
Model View Controller
a. MVC Model
Controller
View
Display
Model
Messages contain control information
Decomposition of SVG Browser
b. Three-stage pipeline
High Level UI
Raw UIDisplay
Rendering as messages
Events as messages
Semantic
Events as messages
Rendering as messages
Figure 1 Reformulation of SVG to message based MVC in a Web Service Model
Input port Output port
Model View Controller
a. MVC Model
Controller
View
Display
Model
Model View ControllerModel View Controller
a. MVC Model
Controller
View
DisplayDisplay
Model
Messages contain control information
Decomposition of SVG Browser
b. Three-stage pipeline
High Level UI
Raw UIDisplayRaw UIDisplay
Rendering as messages
Events as messages
Semantic
Events as messages
Rendering as messages
Figure 1 Reformulation of SVG to message based MVC in a Web Service Model
Input port Output port
Reformulation of SVG to message based MVC in a Web Service Model
04/22/23 29
Figure 4 Architecture of collaborative Web Services drawn for particular case of Internet multiplayer game with SVG
Nar
adaB
roke
ring
Even
t (M
essa
ge) S
ervi
ce In
fras
truc
ture
•••
XGSPSession control
Server
SVG WS 1
Internet Game
SVG WS 2
SVG WS n
•••SVG display 1
SVG display 2
SVG display n
Control to/from
SVG WS1,2, …, n
Control to/from XGSP, SVG display 2
Rendering to SVG display 2
Control to/from
SVG WS1,2, …, n
Rendering from SVG WS 2
Control to/from SVG display 2
Figure 4 Architecture of collaborative Web Services drawn for particular case of Internet multiplayer game with SVG
Nar
adaB
roke
ring
Even
t (M
essa
ge) S
ervi
ce In
fras
truc
ture
•••
XGSPSession control
Server
XGSPSession control
Server
SVG WS 1
Internet Game
Internet Game
SVG WS 2
SVG WS n
•••SVG display 1
SVG display 2
SVG display n
•••SVG display 1SVG display 1
SVG display 2SVG display 2
SVG display nSVG display n
Control to/from
SVG WS1,2, …, n
Control to/from
SVG WS1,2, …, n
Control to/from XGSP, SVG display 2
Rendering to SVG display 2
Control to/from
SVG WS1,2, …, n
Control to/from
SVG WS1,2, …, n
Rendering from SVG WS 2
Control to/from SVG display 2
Web Services Messaging Displays
Architecture of multiplayer game
with SVG
04/22/2304/22/23 3030
Application as a Web serviceApplication as a Web service
Participating Client
RenderingRendering
User Interface
W3C DOM Events
From Master
FromCollaborationAs a WS
Events
Application as a Web serviceApplication as a Web service
Master Client
RenderingRendering
User Interface
W3C DOM Events
To Collaborative Clients
FromCollaborationAs a WS
Events
Control flow for collaborative SVG clients
Figure 3 Control flow for collaborative SVG clients
Collaborative SVG As A Web ServiceCollaborative SVG As A Web Service
04/22/23 31
Three among the different ways of decomposing SVG between client and Web Service component
Shared SVG Browser on PDA
b. Decomposed WS optimized for thin clients
Rendering as messages
Events as messages
Messages contain control information
SemanticHigh Level UI
R F I O
U F I O
Web Service
Event (Message) Service
Raw UIDisplay
Shared SVG Browser on PC
a. Non-decomposed collaborative SVG requiring minimal changes to the original source code
Messages contain control information
Event (Message) Service
R F I O
SVG Browser
SemanticHigh Level UI
Raw UIDisplay
Collaborative Events and Web Service messages
Internet Game
c. Decomposed WS optimized for performance
Messages contain control information
Semantic
R F I O
U F I O
Web Service
Event (Message) Service
High Level UIRaw UIDisplay
Figure 2 Three among the different ways of decomposing SVG between client and Web Service component
Input port Output portRendering as messages
Events as messages
Input port Output port
Collaborative Events and Web Service messages
Collaborative Events and Web Service messages
Shared SVG Browser on PDA
b. Decomposed WS optimized for thin clients
Rendering as messages
Events as messages
Messages contain control information
SemanticHigh Level UI
R F I O
U F I O
Web Service
Event (Message) Service
Raw UIDisplay
Shared SVG Browser on PC
a. Non-decomposed collaborative SVG requiring minimal changes to the original source code
Messages contain control information
Event (Message) ServiceEvent (Message) Service
R F I O
SVG Browser
SemanticHigh Level UI
Raw UIDisplay
Collaborative Events and Web Service messages
Internet Game
c. Decomposed WS optimized for performance
Messages contain control information
Semantic
R F I O
U F I O
Web Service
Event (Message) ServiceEvent (Message) Service
High Level UIRaw UIDisplay
Figure 2 Three among the different ways of decomposing SVG between client and Web Service component
Input port Output portRendering as messages
Events as messages
Input port Output port
Collaborative Events and Web Service messages
Collaborative Events and Web Service messages
04/22/23 32
Real-time Community Collaboration
Scalable Vector Graphics (SVG) via “Shared Web Service”
