Date post: | 21-Jan-2017 |
Category: |
Presentations & Public Speaking |
Upload: | dr-ranbijay-kumar |
View: | 296 times |
Download: | 0 times |
Multimedia Playback & Streaming
JAM 316
Sean McVeigh & Ranbijay Kumar
BlackBerry
5 – 6 February, 2013
What are we interested in?
Building multimedia apps for…
Consuming Audio/Video
Manipulating the media
Dealing with content providers
An application/client solution
A compatible platform
Protocols & formats supported
DRM options
2
Takeaways
Multimedia API
Overview
Cascades Framework APIs – QML/Qt
Multimedia Renderer API – C
Formats & Streaming protocols
Digital Rights Management (DRM)
Sample code snippets!
3
Core API capabilities
Audio/Video playback:
Local or remote resource (streaming)
Embed video window within user interface
UI Controls (buttons, progress slider bar….)
Playlist control (next track, previous track …)
Metadata query and setting (artist, album info…)
5
Audio/video routing:
Audio via headphones, speakers, TV etc
Second screen via HDMI
Core API capabilities (continued)
Audio volume control:
Through device buttons or API
Now Playing Area:
Set artist, album name, get next track/previous track commands
Invoke core media previewer (or any other ones)
In “previewer” mode or the “app” mode
Register as a media previewer
6
Native SDK Context
7
C API
• Multimedia Renderer API
• Multimedia Renderer Event API
• Great for complex multimedia Apps.
C++/Qt (QML) - Cascades API
• Wrapper Classes - high level
APIs such as MediaPlayer,
AudioRecorder etc.
• Easy to integrate with Cascades
Application
Basic components
bb/multimedia/*.hpp
MediaPlayer
AudioRecorder
NowPlayingConnection
Notifications
SystemSound
And other classes for enum types …
9
Let’s see some [code]
1. Play - Pause functionality
2. Using QML/Cascades
3. And using C++/Qt
10
Momentics
IDE
Attribution:http://www.flickr.com/photos/zooboing/4361038612/sizes/o/in/photostream/
VideoPlayerSample
http://github.com/blackberry/Cascades-Community-Samples
Another
sample…
Video Playback
Create a ForeignWindowControl
Attach the window to the MediaPlayer
Scale, resize, reposition
Great performance
Smooth and responsive on interaction
19
mm-renderer Overview
mm-renderer provides mechanisms for:
specifying the set of media to play
issuing playback control commands
retrieving the current playback status
requesting notifications when the status changes
providing dynamic metadata (such as position in a track or playlist) for some media content
23
mm-renderer Overview
API steps:
1. Configure mmr – connect & create context
2. Attach audio and video output
3. Attach media URI
4. Play media!
5. Deal with events
24
mm-renderer - Event Handling
Done through Multimedia Renderer Event API
The event information delivered by the API contains, but is not limited to:
the event type
the context state
a warning string, when applicable
detailed error information, when applicable
26
C API – some relevant header files
mm/renderer.h – Initialize contexts & start playback
mm/renderer/types.h – Data types
mm/renderer/event.h – Event handling
strm.h – Dictionary API to define mm-renderer context settings
Allows multiple processes to safely re-use and access same parameter information
27
OpenAL
OpenAL 1.1
Cross platform 3D audio API
Comes bundled with Native SDK
Great for Games and complex musical applications
Granular & low-level audio functionalities
Use OpenAL + OggVorbis to reduce audio memory footprint through compression
OggVorbis is an Open Source & Cross Platform API too.
Great samples available at http://blackberry.github.com
29
Formats
31
Video + Audio Audio Image
• 3GP, 3GP2
• M4A
• M4V
• MOV
• MP4
• MPEG-4
• MKV
• MP2PS, MP2TS
• AVI
• ASF, WMV, WMA
• F4V
• MP3
• MKA
• AAC
• AWB
• AMR
• WAV
• OGG
• FLAC
• BMP, WBMP
• JPG
• GIF
• PNG
• TIFF
• SGI
• TGA
Codecs
32
Video + Audio Audio
• H.263, H.264
• MPEG-2, MPEG-4
• MJPEG
• Xvid
• VC-1
• VP6
• Sorenson Spark
• AAC
• AC-3 (new: Dolby Digital plus)
• MP3
• PCM
• AMR-WB
• AMR-NB
• QCELP
• FLAC
• Windows Media Audio ( 9, 10 Professional, Lossless)
• Vorbis
• FLAC
• PCM GSM6.10 G729 G722
Recommended Formats
File format Component Codec
Local: MP4 For remote streaming: Any supported containers/streams
Video H.264
Audio AAC
33
BlackBerry 10 will support all the formats that
PlayBook supports AND more…
Supported Streaming Protocols
HTTP
HTTPS
HTTP Progressive Download
RTMPe
RTSP
Http Live Streaming (HLS) v3
34
HTTP Live Streaming
Supports live data (unbounded multimedia stream)
Of course, prerecorded content too
Client intelligently switches streams (bit rates)
Based on bandwidth availability
Allows encryption of media over HTTPS
Custom DRM schemes for content providers
35
Digital Rights Management
1) HLS with 128 AES
2) 3rd party DRM providers
3) Microsoft PlayReady
4) Adobe Flash Access (FAXS) – web only
36
Mainly related with content providers & studios
1) HLS with 128 AES (DRM continued)
On our platform, it is impossible for one app to gain access to the protected content buffers of another
Internally, it is safe; externally, clear text AES keys are being transmitted in M3U files.
If your app can protect the AES keys, then you can use this approach
More details on the platform’s security with NDA (for studios, content providers etc)
The security team can be part of the engagement as well
37
2) 3rd party DRM providers
BlackBerry has active engagements with some 3rd party DRM providers for supporting BlackBerry 10 for product launch
BlackBerry can link content providers to these companies
They will also entertain introductions to other similar companies (that studios might know or prefer)
A port to a new platform could take 6 - 9 months for these companies
Sooner the process starts, the better
38
3) Microsoft PlayReady
The client solution would be using BlackBerry’s keys on the device
Hence, legal partnership would be required
Limited to Progressive Download protocol
39
Other formats/protocols?
41 Application
Multimedia API
Decrypt and
transform to
HLS (or any
other supported
formats)
Local Proxy
server
Often use
services from
other companies
Media Previewer
Just want to play a file?
Invoke the Media Previewer!
Register your app to be a previewer!
43
Quickly, to the
BlackBerry 10
Device!
Bound invocation (Media Previewer)
Set the target as well as the URI
45
For setting the “app” mode specifically, use:
sys.mediaplayer.previewer.app
Leverage other apps
Share links, playlists, artist/album with others
BBM, Twitter, Facebook, etc.
Calendar events, Universal Search
Notifications + Universal Inbox
46
Leverage other devices
Second screen via HDMI
Other libraries for wireless communication
DLNA, UPnP
47
Don’t forget about the UX!
User Experience guidelines
Navigation Pane
Overflow Menu
Context Menu
Peek, Swipe, Gestures
Many more….
48
Ending thoughts ..
BlackBerry 10 SDK is all about options. Choose what fits for your app!
Questions?
Comments?
Concerns?
49
Don’t Forget…
Download the Mobile Conference Guide from BlackBerry World. Search for BlackBerry Jam Europe!
Complete your session surveys in your conference portal or on your BlackBerry 10 device using the Mobile Conference Guide.
Fill out the conference survey at the Survey desk in the First Floor Lounge or on your BlackBerry 10 device using the Mobile Conference Guide and receive your gift*.
Thank you for attending BlackBerry Jam Europe! *limited quantities, while supplies last