Design in Motion:Video Production WorkflowDES05Ben WaggonerTechnical Evangelist,Core Media Processing TechnologiesMicrosoft Corporation

Agenda

Video and audio in WPF/EBest practices for processing video and audioBest practices for Windows Media encoding for WPF/EUsing the Expression Media Encoder

Video and audio in WPF/E

It's just Windows Media!Huge library of existing content already compatibleExisting URLs from Windows Media Services still workAlready supported by many hosting companiesLots of tools available for different markets

Video codec;Windows Media Video 7/8/9 and 9 Advanced Profile

Audio codec:Windows Media Audio "Standard"

Not Pro, Voice, or Lossless in this version

HD Video Playback

video

FLV v. WMV

What does WMV give content creators?

Higher quality video and audio Fewer bits needed for same quality

Much faster encoding3x faster for optimum quality on 4-core workstation15x faster with Tarari card for hardware acceleration

Much better TCO for video hostingMany more streams off a 2K3 Server than Flash serverCheaper per GB charges from hosting providers

Mature live streaming/encoding tools

Best Practices for Video Processing

CaptureGetting the content in higher quality

PreprocessingOptimizing the content for the codec itself

Best Practices: Capture

Capture native bitstream or uncompressedDV, DVCPRO HD as native bitstreamOther formats as uncompressedDon’t use DV bridge for analog-digital conversion

DV codec throws out a lot of image data, especially color

Access earliest generation dub as possibleMost VHS tapes didn’t get posted on VHS!Ripping from DVD also sub-optimal

Best to grab digital export from NLE

Preprocessing

Everything we do to the video and audio between the source and the codecThe biggest single factor in good-looking WPF/E video

The best the codec can do is give back what you gave it

For challenging projets, I spend ~80% of my keyboard-and-mouse time on preprocessing

Once the codec settings are nailed, you're good to go.

Unprocessed frame

Same Frame Preprocessed

Unprocessed @ 800 Kbps

Preprocessed @ 800 Kbps

Deinterlacing and Inverse Telecine

WPF/E uses progressive videoFor 60i/50i, deinterlace to progressiveFor video telecined from film, use inverse telecine to restore 24pIf it’s progressive, leave it alone!<example slide>

Crop

Video monitors don’t go to edgesComputers doCrop edge blankingAsymmetric okay to cut out noiseFor low rez, crop to safe areaCrop letterboxing as wellIf going to 16x16 resolution, can crop to final aspect ratioFor mixed video/computer playback, matte out noise in blanking

The Safe Area

Image adjustment

Brightness and ContrastComputers have broader range — use itBlacks should look identical to RGB=0,0,0 in browser

Noise ReductionMake image easier for compressorRemove grainSmooth sharp edges

Levels example

Levels in Action

After The Settings

Best Practices: Audio Normalization

Make loudest point of audio as loud as system beep-3 dB / 90%Not compressor/limiter - no change in dynamic range

Best Practices: Encoding

Progressive versus StreamingData RatesFrame SizeFrame RateAudio SettingsAdvanced Registry Key settings

Progressive v. Streaming

Two legitimate modes of delivering web videoNeed to know which one will be used before encoding

Progressive should use 2-pass VBRStreaming should use 2-pass CBR

Progressive Download

Off a web serverE.g., IIS in Windows 2003 Server

Most efficient with Variable Bit Rate (VBR) encodingRandom access in part of stream already downloadedGreat for movie trailers

Streaming

Files fromWindows Media ServicesIncluded in Windows 2003 Server

Needs Constant Bit Rate (CBR) encodingOffers random access throughout streamReduces need for local caching (big for mobile devices)Great for movies

Data Rates

We measure rates in power-of-ten numbersK=1000, not 1024M=1000000, not 1048576

StreamingNeed more sustained bandwidth to user more than data rate

ProgressiveA slower connection means user waits longer for playbackBut can use high bitrates for short clips

Frame Size

WPF/E wants square pixel videoSo need to convert to aspect ratioExample

DV is 720x4804:3 content becomes 640:48016:9 content becomes 832x480

Depends somewhat on data rate200-500 Kbps: 320x240 or 432x2401-2 Mbps: 640x480 or 832x240

Encoding modes

1-pass versus 2-pass1-pass reads the source in and encodes out at the same time2-pass does analysis pass first, calculating optimal bit-budget2-pass is always better if you've got the time

And it’s not 2x slower – the analysis pass runs quite a bit faster

CBR versus VBRConstant Bitrate

Optimum for streaming – quality varies, bitrate constant

Variable BitrateOptimum for progressive download – varies bitrate, quality constant

Frame Rates

Use frame rate that’s an integer division of source59.94 to: 59.94, 29.97, 15, 10, 7.5, 6, 550 to: 50, 25, 12.5, 10, 524 to: 24, 12, 8, 6

For anything over 200 Kbps, can use full frame rate of source

Expression Media Encoder

New compression tool targeting .WMV for WPF/EBunded with Expression Media ProductAccessible to both designers and video professionalsOffline and Live Encoding

But there's no secret sauce – you're welcome to use any WMV encoding tool with WPF/E

Using Expression Media Encoder

demo

Advanced Compression Settings

The codec has a bunch of registry key setting to further optimize encodingI fear regedit – you can use “WMV9 PowerToy”

Lots more options than I’ll discuss todayLink to full documentation at end

Tools will be integrating these controls later this year

WMV9 PowerToy

http://citizeninsomniac.com/WMV/#WMV9PowerToy

Number of Threads

Current codec can use up to four threads for encoding

Big speed improvement with multiple cores

Auto picks number of threads availableSees HyperThreading (Pentium 4) as 2 processors

But HT is less of a gain with compression than additional core

Want to have at least 64 pixels high per thread<128 height: 1-thread<256 height (so 320x240): 2-threads256+ height: 4-threads

Motion Vector Search Range

Motion Estimation copies moving elements between frames

Much more efficient to encode “motion vector” than second copy of element

Using a larger Motion Search rangeProvides better efficiency with faster motionSlows encodingCan reduce efficiency with slower motion

MV Range options

Available Options:Macroblock-adaptive128 pixels horizontally, 64 pixels vertically (default)256 pixels horizontally, 128 pixels vertically1024 pixels horizontally, 256 pixels vertically2056 pixels horizontally, 512 pixels vertically

Each progressive option slowest encoding by ~2xMacroblock-adaptive slightly faster than 1024x256

Use Macroblock Adaptive for non-realtime encoding

MV Range – Reference Frame

MV Range – Search Frame

Chroma Search

Finds where color changes but B&W channel doesn’tQuality improvements 5-20% (most with colorful content)

Crowd shots, motion graphics helped in particular

Modes:Macroblock-adaptive with nearest integer chroma (fast, good)Macroblock-adaptive with true chroma (speed/quality mix)Luma only (default)Luma with nearest-integer chromaLuma with true chroma (best, slowest)

B-Frames

Number of bi-directional (B-Frames) between I-Frames and P-FramesB-Frames based off previous and next frameB-Frames typically fewer bits for same qualityNo frames are based on B-frame

Don’t insert I-frame for flashes if they land on a B-frame

So dynamically insert B-frames when there are flashes!Big win for gunfights and discos

1 B-frame optimal for most web encoding

B-Frames Example

Loop Filter

Smooths out hard edge blocking artifacts“In-loop” - used as prediction for future frames

Only blurs content where an artifact would appearMuch less noticeable at higher resolutionsSoftness better if alternative is artifacts

Increased CPU decoder requirements somewhat

Overlap Filter

Softens edges where motion vectors overlapSoftens image somewhatBut reduces artifacts

RecommendationOn if you see artifactsOtherwise Off

Also increases decoder complexity some

Median Filter

Denoises image before calculating motion vectorsKeeps grain/noise from “swirling”Helps low-bitrate encodesBut can leave “trails”

RecommendationOnly use with grainy source, and QA carefully

Denoise filter

Simple noise reduction filterHelps when the source is noisyNot as good as offline noise reducersRecommendation

Good for realtime encoding in low lightBut better to not have noisy source…

Noise Edge Removal

Blanks Line 21, other noise around edge of frameHandy for real-time encoding of analog sourcesOnly use if source has edge noise

Lookahead

Specifies up to 16 frames forward for analysisOnly needed in 1-pass encoding

Used to detect scene changes, flash framesBig improvement for 1-pass CBR encodesCan dynamically insert B-frames with flashes

Default is Off (Does not look ahead)Adds to latency for live encoding

~½ second for Lookahead=15 @ 29.97 fps

Slightly slows encoding as Lookahead increases

DQuant

“Differential Quantization”Different macroblocks are compressed different amounts

Compress gradients less, to reduce banding

RecommendationI and P frames

Using on B-frames wastes bits that will be discarded

Lots of levels available“Regular” good default with most content

Perceptual Option

Like DQuant, optimizes bits based on imageUsed in addition to DQuantAdaptive Dead Zone 1 good default for film content

Experiment carefully with other kinds of sources

Optimum Settings: On demand

Highest quality encoding and reasonable performance

Up to 4x slower than default settings

Encoder complexity = 4/80 (second slowest)Motion Search Level = macroblock adaptive true chromaMotion Search Range = macroblock adaptiveMotion Match Method = Adaptive SAD/HadamardDQuant = RegularPerceptual Option = Adaptive Dead Zone 1 for filmForce LoopFilter = OnNumBFrames = 1At least 96 pixels high per thread (so 2 max for 320x240)

Optimum Settings: Webcasting

Optimized for reliable real-time encodingRemember dual-duals are almost 4x faster than single core

Motion Search Range = Biggest that doesn’t drop framesMotion Search Level = Luma + integer chromaNumBFrames = 1Force LoopFilter = 1 (on) Dquant Option = RegularPerceptual Option = Adaptive Dead Zone 1 for filmLookahead = 16 (Max – most important single option)

Designing Windows Applications with Expression Blend

2-day instructor-led training in over 20 US cities

Tools and Technologies UsedExpression Blend and Expression Design

NET Framework 3.0, Windows Presentation Foundation, XAML

Visual Studio 2005

Target AudienceUser interface designers with advanced experience with Flash or Director.  Understanding of Visual Studio 2005 or programming skills is not required. 

PrerequisitesDoes not require prior experience using Expression Blend or Windows Presentation Foundation.

We recommend:

Building an understanding of .NET Framework 3.0

Download Expression Blend http://www.microsoft.com/expression

Get inspired!  www.seewindowsvista.com

Learn More: http://tinyurl.com/3y6jg7

Q&A and Contact

Me:Ben.Waggoner@microsoft.com

My blog:http://on10.net/blogs/benwagg/

My class at Stanfordwww.digitalmediaacademy.org/courses/video-compression-training.html

Please fill out your eval

evaluation

© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions,

it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.