Kurento Real Time Media Stream
Processing
Juan Ángel Fuentes
Software Developer. Stream Oriented GE
Introducing multimedia infrastructures
1
Peer-to-Peer Application (without media infrastructure)
WebRTC video stream
Application with media infrastructure
Media infrastructure
Multimedia infrastructures for the
Future Internet
2
Media is
hereMedia got
there
Media got
there
Analyse
Transform
Store
Transcoding
MCU
Recording
Enrich
Augment
Adapt
Sensors
Context
Events
Media is
here
Enrich, augment, adapt, analyse, transform,
store: what’s the problem?
3
Complexity
The Stream Oriented Generic Enabler –
SO GE
4
• Interoperable media exchange (multiplatform/multiprotocol)
• WebRTC, RTP, HTTP (video tag), etc.
• Standard capabilities
• Transcoding, MCU, recording
• Advanced capabilities
• Computer vision, augmented reality, mixing, blending, etc.
Multimedia infrastructure
• REST API
• JavaScript API
• Java API
APIs
• LGPL 2.1
Is distributed through a flexible FOSS license
Kurento: The equation
5
Future Internet
Multimedia Infrastructur
e
Simple Developmen
t APIs
5
General Architecture
6
77
Key Concepts: Media Elements
Functional unit performing a specific action on a media stream
Developers abstract from the low level implementation
Able to receive media form other elements
Able to send media to other elements
8
Key Concepts: Types of Media Elements
Input Endpoints: injects media
Filters: Transform or analyze media
Hubs: Manage multiple media flows
Output Endpoints: take the media stream out
9
Key Concepts: Media Pipeline
Media Pipeline: is a chain of media elements, where the output
stream generated by one element (source) is fed into one or more
other elements input streams (sinks)
10
Key concepts: media elements and
pipelines
11
Media Element
• Provides a specific media functionality
› Send/receive media. These are the Endpoints
› Process media
› Transform media
• Ready to be used
• New media elements can be added
Media pipeline
• Chain of media elements implementing the
desired media logic
• The Media Server provides the capability
of creating media pipelines by joining
media elements of the toolbox
Media Element
Sin
k
SR
C
Media Pipeline
1212
Agnostic Media adapter
In charge of making possible that Stream Oriented GE APIS allow
developers to combine media elements to create the desired
pipeline
Fully abstracts all the complexities of media codecs and formats
13
Developers create applications just connecting
elementsApplication 2
Application 3
Application 1
Toolbox of media elements
14
Media Server (KMS): The nucleus of
Kurento
15
•KMS is a middleware for media streams:
•Receives the stream
•Process the stream
• Issues the stream
Send
Receive
Analyze
Augment
Enrich
Transform
Transcode
Record
Process
Replicate
Media
SourceMedia
Sink
KMS
Media plane and Signaling plane
16
How to control the media server?
17
•REST API
•JSON RPC
•Websocket interface in KMS
API Implementations
18
For now…
Multimedia Application Architecture in SO GE
Web application three layers parallelism:
Presentation layer: Multimedia presentation and capturing
Application logic: Multimedia logic in charge of building the pipeline
Service Layer: Kurento Media Server
19
20
Kurento Protocol
JSON-RPC over WS
Media Traffic
Application code
Media Traffic
Client Kurento
Media ServerApplication
Server
Java Kurento
Client
JavaScript
Kurento
Client
Application codeApplication code
Kurento
Protocol
JSON-RPC
over WS
Signaling
protocol
Kurento
Media Server
Client
Kurento Applications
Application execution flow
21
Client
Code
Kurento API Media
Server (KMS)
I want this media in this type
Commands requesting
the creation of a pipeline
(JSON-RPC request)
What you want is here
Pipeline
creation
phase
Media
exchange
phase
1
2
Specific logic
(e.g. AAA)
Media
pipeline
creation
Media exchange between client and server
Campus Party Brazil
22
23
Campus Party Brazil
24
Let’s get started!
25
Tutorials
http://doc-kurento.readthedocs.io/en/stable/tutorials.html
What do we need?
26
An instance of the Kurento Media Server
•FIWARE-LAB
•Local installation
Maven
NPM
Bower
Your favorite IDE
27
Tutorial 1 - Hello world
28
Tutorial 1 - Hello world
Tutorial 1 - Hello world, server
29
Tutorial 1 - Hello world, client
30
31
Tutorial 1 - Hello world (Node.js)
Tutorial 1 - Hello world (Node.js)
32
Tutorial 2 - WebRTC magic mirror
33
Tutorial 2 - WebRTC magic mirror
34
35
Tutorial 2 - WebRTC magic mirror
Tutorial 3 - WebRTC one-to-one video call with
recording and filtering
36
Tutorial 3 - WebRTC one-to-one video call with
recording and filtering
37
Tutorial 3 - WebRTC one-to-one video call with
recording and filtering
38
Kurento and the FIWARE-LAB
39
• FIWARE-LAB
– Working instance of FI-WARE enabling free experimentation with
technology
– http://lab.fiware.org
• Creating a Kurento instance from an image
– Use latest version of Kurento images.
• Creating a Kurento instance using recipes
– Use Ubuntu 14.04 LTS clear image
– Use latest version of Kurento recipes
Kurento and the FIWARE-LAB
40
To learn more…
41
Thank you!
http://fiware.org
Follow @FIWARE on Twitter