Cop
yrig
ht ©
200
5 U
nive
rsit
y of
Gla
sgow
Talk Outline
• Real-time Collaborative Environments– Definition and Requirements
– Example: AccessGrid
– History and Current Status
– Underlying Technologies and Standards
– Capture and Analysis of Real-time Sessions
• Research Issues and Future Directions
…from the perspective of a developer of those applications, and of the relevant technical standards
Cop
yrig
ht ©
200
5 U
nive
rsit
y of
Gla
sgow
Original Picture ©1998 UCL
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Picture from http://www.accessgrid.org/
Real-Time Collaborative Environments
• Support collaborative work by scientists, industry and others– Both one-to-one and group communication
– Widely distributed participants; heterogeneity
• Provide high-quality audio-visual media
• Provide sense of community and presence– Venues as a rendezvous point; familiar virtual
meeting places
• Integrate with other computing resources– Visualization, shared state & data repositories,
computational resources, applications
• Secure and trusted infrastructure
Cop
yrig
ht ©
200
5 U
nive
rsit
y of
Gla
sgow
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Picture from http://www.accessgrid.org/
The AccessGrid
• Example: A widely used real-time collaborative environment– Supports many-to-many interactive meetings
– Typically room based, although not required by the underlying technology
• Components:– Venue client
– Media tools
– User experience
• A framework for initiating real-timecollaborative work sessions– Default media tools provided, can be
replaced
Cop
yrig
ht ©
200
5 U
nive
rsit
y of
Gla
sgow
The AccessGrid: Venue Client
• Venue client provides the rendezvous point to join a session
• Maintains state, participant list, links to other venues, etc.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Picture from http://www.accessgrid.org/
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Cop
yrig
ht ©
200
5 U
nive
rsit
y of
Gla
sgow
The AccessGrid: Typical Media Tools
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
• Joining a venue starts multicast audio and video tools on the controller display– Audio level and activity indicator
– Video thumbnails and statistics
• A separate display shows the main video windows…
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Cop
yrig
ht ©
200
5 U
nive
rsit
y of
Gla
sgow
The AccessGrid: User Experience
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Picture from http://www.accessgrid.org/
Tool development ongoing - theuser experience will change withtime…
Cop
yrig
ht ©
200
5 U
nive
rsit
y of
Gla
sgow
History and Standards Development
• Long development history• Strong open standards• Commercial interest
19851975 1995 2005
Mbone tools AccessGridvic
vat rat nte
wb sdr
Grid Computing
RTPNVP
ST ST-II SIP
H.323
NVP-II
H.320
3G phones
Video conferencing
IP Multicast
RFC 741
Cop
yrig
ht ©
200
5 U
nive
rsit
y of
Gla
sgow
Current Deployment Status
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
• Hundreds of nodes at sites worldwide• Large scale multicast video conferences• Standard, if primitive, media formats• Non-standard web service control infrastructure
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
• Tens of millions of mobile phones• Voice telephony and multicast streaming• Standard, advanced, media formats• Standard SIP-based control infrastructure
• Dedicated video conferencing• Wi-Fi and wired SIP-based VoIP• PC-based video streaming• Mixture of standard and proprietary
Others…
• Largely successful deployment of open standard media coding and transport protocols– With notable exceptions, and
continuing patent issues…
• Largely successful deployment of open standard session setup and control protocols– AccessGrid, although open, is
the main exception to the use of standards-based protocols
• Independent and interoperable implementations available and widely used
Cop
yrig
ht ©
200
5 U
nive
rsit
y of
Gla
sgow
Underlying Technologies
• Complex protocol stack, many options
• Two phase operation: control then data
Mediacodecs
Light weightsessions
IP
TCP UDP/UDPlite
SIP SAP RTP
SDP, SDPng
Session and call control
RTSP
DCCP
Web/Grid Services
STUN
ICE/TURN
NAT Traversal
SIP+SDP
Web services
RTP + media codecs
Cop
yrig
ht ©
200
5 U
nive
rsit
y of
Gla
sgow
Control Protocols: Concepts
• Many control protocols exist– For telephony and integration with the PSTN
– For streaming video, television and radio
– For collaborative work in a Grid environment
• Some common features:– Initial rendezvous point and user location
– Media format negotiation
– Address and port negotiation
– Session metadata• Dates and times
• Additional information
• Participant scheduling
• Shared state
– Negotiation of security parameters
v=0o=jdoe 2890844526 2890842807 IN IP4 10.47.16.5s=SDP Seminari=A Seminar on the session description protocolu=http://www.example.com/seminars/[email protected] (Jane Doe)c=IN IP4 224.2.17.12/127t=2873397496 2873404696a=recvonlym=audio 49170 RTP/AVP 0m=video 51372 RTP/AVP 99a=rtpmap:99 h263-1998/90000
Cop
yrig
ht ©
200
5 U
nive
rsit
y of
Gla
sgow
Control Protocols: Implementations
• Session Initiation Protocol (SIP)– Commercial video conferencing products
– Commercial telephony and voice-over-IP
• Real-Time Streaming Protocol– Video streaming (e.g. QuickTime)
• Web services– AccessGrid
All evolutions of HTTP designedto support negotiation and control
of real-time media sessions
Complex, flexible
Large toolkits
Initial negotiation
Ongoing control
Cop
yrig
ht ©
200
5 U
nive
rsit
y of
Gla
sgow
Data Transfer Protocols
• Once negotiation completed, media data transferred
• Complex due to nature of IP networks– A “best effort” packet delivery service
– Performance not guaranteed• Loss, delay, reordering, corruption, duplication
– Higher level protocols must compensate• Application complexity, simple network
• An application must correct errors, recover media timing, etc., in addition to processing audio/visual data
• Cannot use TCP, since require accurate timing
• Main standard is RTP: Real-time Transport Protocol
IP
EthernetADSL
PPP
Optical FibreTwisted Pair
TCP UDP
HTTP
FTP
SMTP RTP
SIP RTSP
HTML MIME Media codecs
Application programs
Wireless
Cop
yrig
ht ©
200
5 U
nive
rsit
y of
Gla
sgow
• A standard protocol for real-time collaborative environments on IP networks
• Provides features for:– Timing recovery and synchronisation
– Loss detection and recovery
– Media format identification
– Participant identification andpresence
– Reception quality statistics
but must be implemented inan application!
• A toolkit for real-time data transfer
Payload data
Padding
Synchronization source (SSRC) identifier
Timestamp
Sequence NumberPTMP XV CC
Contributing source (CSRC) identifiers
RTP: Real-time Transport Protocol
Cop
yrig
ht ©
200
5 U
nive
rsit
y of
Gla
sgow
Capture and Analysis of Media Sessions
• Media capture and decoding is complex– One cannot simply dump packets to a file!
– Need to understand transport protocol• Reconstruct timing, correct errors, log metadata
– Need to understand the signalling protocol• Metadata needed to find and understand the media
• Data storage format:– Standard file formats exist, but often lacking in metadata and context
– Formats based on the ISO multimedia file format better?
• Relevant? “RTP Payload Format for ETSI ES 201 108 Distributed Speech Recognition Encoding”, RFC 3557
Cop
yrig
ht ©
200
5 U
nive
rsit
y of
Gla
sgow
Underlying Technologies
• Flexible and sophisticated control protocols– Initial rendezvous and negotiation
– Ongoing control during a collaborative work session
• Capable data transfer protocols; range of media formats
• “Some assembly required”– Protocols are expected to be tightly coupled to applications
– The design rejects much of the traditional protocol layering
– Collaborative work environments use a wide range of protocols; often in different ways to commercial products
Cop
yrig
ht ©
200
5 U
nive
rsit
y of
Gla
sgow
Research Issues and Future Directions
• Continual evolution of protocols– New media formats
– Extension of the control frameworks• Emergency calls (E.911)
• Presence and messaging
• NAT traversal, seamless operation over present-day Internet
• Integration of AccessGrid with standard control protocols– SIP-based architecture? Desirable to move away from “Grid” protocols, to
a more standard architecture, to leverage massive commercial development
• Evolution of IP multicast peer-to-peer overlays
Cop
yrig
ht ©
200
5 U
nive
rsit
y of
Gla
sgow
Summary
• Real-time Collaborative Environments– Definition and Requirements
– Example: AccessGrid
– History and Current Status
– Underlying Technologies and Standards
– Capture and Analysis of Real-time Sessions
• Research Issues and Future Directions
Any Questions?