Post on 21-Apr-2017
transcript
Creating rich WebRTC Applications with Kurento
Luis Lopezlulop@kurento.org
http://www.kurento.org - lulop@kurento.org2
Who I amAssociate Professor
Escuela Técnica Superior de Ingenieros de TelecomunicaciónUniversidad Rey Juan Carlos (south Madrid)
Researcher in the area of RTC
• http://www.nubomedia.eu• http://www.fiware.org
FOSS enthusiast
• Kurento project lead: http://www.kurento.org • NUBOMEDIA Community lead: http://www.nubomedia.eu
http://www.kurento.org - lulop@kurento.org3
Real-Time media Communications
Capture Encode Cipher Transport Transport Decipher Decode RenderNetwork
The media plane
The signaling plane
I wan to call you, do you accept?
Yes, do it in this way
http://www.kurento.org - lulop@kurento.org4
WebRTC to the rescue.
Before WebRTCFirst wave of
WebRTC technologies
Begin End• APIs• Standards• FOSS
Development experience
when working with real-time media
Common WebRTC application (p2p communications)
WebRTC video stream
http://www.kurento.org - lulop@kurento.org5
WebRTC infrastructures
Peer-to-Peer WebRTC Application (without media infrastructure)
WebRTC video stream
WebRTC Application based on media infrastructuremedia infrastructure
http://www.kurento.org - lulop@kurento.org6
WWW VS (Web)RTC applications
Control
Multimedia Application logic
(developers’ code)
Multimedia Capabilities
RecordingTranscoding
RoutingMixing
AnalyzingEtc.
MediaTraffic
RTC Media APIs
Multimedia Clients
Events
Application Signaling
Control
WWW Application logic
(developers’ code)
Database Capabilities
Data storageData recoveryData querying
Data processingEtc.
DD.BB. API
WWW Clients
Events
Application Signaling
http://www.kurento.org - lulop@kurento.org7
http://www.kurento.org - lulop@kurento.org8
Control
Multimedia Application logic
(developers’ code)
Multimedia Capabilities
RecordingTranscoding
RoutingMixing
AnalyzingEtc.
RTC Media APIs
Events
http://www.kurento.org - lulop@kurento.org9
Kurento: a WebRTC infrastructure an its APIs
Control
Multimedia Application logic
(developers’ code)
RecordingTranscoding
RoutingMixing
AnalyzingAdapting
MediaTraffic
RTC Media APIs
Multimedia Clients
Events
Application Signaling
Computervision
Augmentedreality
BlendingEtc.
Multimedia Capabilities
Kurento Media Server(KMS)
Kurento Client APIKurento Room APIKurento Tree API
http://www.kurento.org - lulop@kurento.org
10
Cooking Kurento
10
http://www.kurento.org - lulop@kurento.org11
The Kurento FOSS Community
Community support WebRTC worldwide reference
International awards More than 300 companies
Very active mailing list
First result in Google
http://www.kurento.org - lulop@kurento.org12
The Kurento FOSS project• http://www.kurento.org
– Main web site
• https://www.twitter.com/kurentoms – Main social channel
• https://groups.google.com/forum/#!forum/ kurento– Main mailing list
• https://github.com/kurento – Main repository
• https://www.youtube.com/channel/UCFtGhWYqahVlzMgGNtEmKug– Kurento Youtube channel
http://www.kurento.org - lulop@kurento.org13
Developing with KMS:The Kurento Client API
Media Element• Provides a specific media
functionality
› Send/receive media
› Process media
› Transform media• Exchange media through› Sources
› Sinks
Media pipeline• Chain of media elements
implementing the desired media logic.
• The Media API provides the capability of creating media pipelines by joining media elements of the toolbox
Media Element
Sink
SRC
http://www.kurento.org - lulop@kurento.org14
A modular API
• Modular in the sense of modularity*– Isolation
• Internal states of a module don’t affect the rest– Abstracion
• Internal states are hidden behind an interface– Composability
• Interface must enable module recombination and assembly– Reusability
• If a module has it, you can use it– Extensibility
• If a module does not have it, you can add it.
* Baldwin, Carliss Young, and Kim B. Clark. Design rules: The power of modularity. Vol. 1. MIT press, 2000
http://www.kurento.org - lulop@kurento.org15
The paradigm of modularity
http://www.kurento.org - lulop@kurento.org16
The Kurento Client API in one word
sourceElement.connect(sinkElement)
connect
http://www.kurento.org - lulop@kurento.org17
Kurento hello world
http://www.kurento.org - lulop@kurento.org18
//MediaPipeline is a holder of media elementsMediaPipeline pipeline = kurento.createMediaPipeline();
//Create your media elementsWebRtcEndpoint webRtcEndpoint = new WebRtcEndpoint.Builder(pipeline).build();
//Connect your media elementswebRtcEndpoint.connect(webRtcEndpoint);
Control
Multimedia Application logic
(developers’ code)
RecordingTranscoding
RoutingMixing
AnalyzingAdapting
Kurento Media APIs
Multimedia Clients
Events
Application Signaling
Computervision
Augmentedreality
BlendingEtc.
Multimedia Capabilities
//Where to render the local and remote streamsvar options = {
localVideo : videoInput, remoteVideo : videoOutput, onicecandidate : onIceCandidate }
//Start media capture and communicationswebRtcPeer = new kurentoUtils.WebRtcPeer.WebRtcPeerSendrecv(
options, function(error) { if (error) return console.error(error); webRtcPeer.generateOffer(onOffer); });
Application Server Code (Java)
Client Server Code
Application source for Spring (Java EE)Application source for Node.js (JavaScript)
http://www.kurento.org - lulop@kurento.org19
Recording
Sink
SRC
Sink
//MediaPipeline is a holder of media elementsMediaPipeline pipeline = kurento.createMediaPipeline();
//Create your media elementsWebRtcEndpoint webRtcEndpoint = new WebRtcEndpoint.Builder(pipeline).build(); RecorderEndpoint recorderEndpoint = new RecorderEndpoint.Builder(pipeline).build(“file:///myfolder/myfile.mp4”);
//Connect your media elementswebRtcEndpoint.connect(webRtcEndpoint);webRtcEndpoint.connect(recorderEndpoint);
Application source for Spring (Java EE)Application source for Node.js (JavaScript)
http://www.kurento.org - lulop@kurento.org20
Playing
Sink
SRC SR
C
//MediaPipeline is a holder of media elementsMediaPipeline pipeline = kurento.createMediaPipeline();
//Create your media elementsWebRtcEndpoint webRtcEndpoint = new WebRtcEndpoint.Builder(pipeline).build(); PlayerEndpoint playerEndpoint = new PlayerEndpoint.Buildr(pipeline).build(“file:///myfolder/myfile.mp4”); //RSTP and HTTP URIs also supported
//Connect your media elementsplayerEndpoint.connect(webRtcEndpoint);
Application source for Spring (Java EE)Application source for Node.js (JavaScript)
http://www.kurento.org - lulop@kurento.org21
Interoperating
Media Pipeline
WebRTC audiostreaming
Sink
SRC
SRC
Sink
Media Pipeline
Sink
SRC
SRC
Sink
WebRTC videostreaming
RTP audiostreaming
RTP videostreaming
Transparent transcoding!!
http://www.kurento.org - lulop@kurento.org22
Some Media processing modules• Augmented reality
– MarkerArModule– MarkerlessArModule
• Computer vision– FaceDetector– NoseDetector– EyeDetector– CrowdDetector– PointerDetector– MotionDetector– VirtualFence– Etc.
Sink
SRC
Sink
SRC
Sink
SRC
Sink
SRC
Sink
SRC
http://www.kurento.org - lulop@kurento.org23
Using media processing modules
Sink
SRC
Sink
SRC
CrowdEventsSR
C
Sink
SRC
CrowdDetector
RtspPlayer
Sink
SRC
See demo
See demo
http://www.kurento.org - lulop@kurento.org24
Implementing group communications: just connect
Media Pipeline
WebRTCstreaming
Sink
SRC
WebRTCstreaming
SRC
Sink
SRC
Sink
WebRTCstreaming
WebRtcEndpoint 1
WebRtcEndpoint 3
WebRtcEndpoint 2
SRC
Sink
WebRTCstreaming
WebRtcEndpoint 4
User 1 User 2
User 3
User 4
http://www.kurento.org - lulop@kurento.org25
Group communications: RTP topologies
Sink
SRC
SRC Sink
SinkSRC
SRC
Sink
• Media Mixing Mixer– Composite
• Media Switching Mixer– Connect primitive– Natural topology
• SFU– WebRtcSfu
• Simulcast
http://www.kurento.org - lulop@kurento.org26
Creating your own modules
KurentoIDL Compiler
JSON
Module IDL Description
Java Java-Script
cpphpp
Media API code Toolchain
Auto-generated code
ModuleImplementation
cmake
Interface
Developer
(1)
(2)
(3)
http://www.kurento.org - lulop@kurento.org27
Going up the API stack: the room API
createRoomdeleteRoom
joinRoomleaveRoom
publisMediaunpublishMedia
onParticipantJoinedonParticipantLeft
onMediaPublishedonMediaUnpublished
onMessage
See demo
http://www.kurento.org - lulop@kurento.org28
Going up the API stack: the tree API
createTreedeleteTree
joinAsPresenterjoinAsViewer
leave
http://www.kurento.org - lulop@kurento.org29
What’s next
http://www.kurento.org - lulop@kurento.org30
Where to start
• http://www.kurento.org/documentation – Installation and administration guide– Java tutorials for Spring and Java EE developers– JavaScript tutorials for Node.js developers– JSON-RPC protocol documentation
• https://github.com/kurento – Main repository
http://www.kurento.org - lulop@kurento.org31
Thanks
Luis Lopezlulop@kurento.org