Page 1
CS148: Introduction to Computer Graphics and Imaging
Image Compositing
Colbert Challenge
CS148 Lecture 14 Pat Hanrahan, Fall 2009
Key Concepts
Optical compositing and mattes The alpha channel Compositing operators Premultipled alpha Matte extraction
Page 2
Optical Compositing
CS148 Lecture 14 Pat Hanrahan, Fall 2009
Image Composition
Defn: Combine foreground element with background
Examples: Graphics arts: Selections and masks, stencils,
friskets and masking tape Animation: cels, multiplane camera Film: optical printing, blue screen matting Video: chroma-keying Computer graphics: alpha channel
Page 3
CS148 Lecture 14 Pat Hanrahan, Fall 2009
Animation Cels
Appraised on Antiques Roadshow http://www.pbs.org/wgbh/roadshow/archive/200804A11.html
CS148 Lecture 14 Pat Hanrahan, Fall 2009
Multiplane Camera – Walt Disney
http://disneyandmore.blogspot.com/2007/09/walt-disney-multiplane-camera-and.html
Page 4
CS148 Lecture 14 Pat Hanrahan, Fall 2009
Optical Printing
From: “Special Optical Effects,” Zoran Perisic
From: “Industrial Light and Magic,” Thomas Smith (p. 181)
CS148 Lecture 14 Pat Hanrahan, Fall 2009
Composing Two Elements
Background
Foreground Traveling Matte
Holdout Matte
*
* +
+ =
=
Page 5
The Alpha Channel
CS148 Lecture 14 Pat Hanrahan, Fall 2009
The Alpha Channel
A alpha channel is an additional image that defines: The transparency or opacity of an image The presence or absence of imagery
Geometric coverage: soft-edge
Or both coverage and transparency
Alpha channels may be Binary masks: all or none Mattes: 0.0 to 1.0 (0 to 255)
Page 6
CS148 Lecture 14 Pat Hanrahan, Fall 2009
Fragment: Color + Coverage
Pixel
C=(R,G,B)
a = A = Coverage = Area = Opacity = 1 - Transparency A
Color c of pixel is an area-weighted average of C
c = A C
Area-weighted color = pre-multiplied (with A) color
Convention: capital, not multiplied; lower, multiplied
Image Composition
Page 7
CS148 Lecture 14 Pat Hanrahan, Fall 2009
OVER Operator
OVER
Foreground Background
=
Composite
aF CF
Composite color: c = AF CF + AB CB = (aF CF )+ (1-aF) (aB CB)
Composite alpha: a = AF + AB = aF + (1-aF ) aB
aB CB AF = aF AB = (1-aF ) aB
CS148 Lecture 14 Pat Hanrahan, Fall 2009
OVER Operator
OVER
Foreground Background
=
Composite
aF CF
Composite color: c = cF + (1-aF)cB
Composite alpha: a = aF + (1-aF ) aB
aB CB AF = aF AB = (1-aF ) aB
Page 8
CS148 Lecture 14 Pat Hanrahan, Fall 2009
Region Coverage
+ = 4 Regions
aB aA
CS148 Lecture 14 Pat Hanrahan, Fall 2009
Porter-Duff Compositing Algebra
How many ways can two pixels be combined?
4 Regions
Region 1: 1 possibility - 0
Region 2: 2 possibilities - A or 0
Region 3: 2 possibilities - B or 0
Region 4: 3 possibilities - A, B or 0
Operators: 12 total possibilities
1 2
34
Page 9
CS148 Lecture 14 Pat Hanrahan, Fall 2009
Porter-Duff Compositing Algebra
c = FAcA+FBcB
CS148 Lecture 14 Pat Hanrahan, Fall 2009
OpenGL Blending Modes
Specify src (foreground) and dst (background) F’s
Porter-Duff 0, 1, As, Ad, 1-As, 1-Ad,
Other useful modes min(As,1-Ad) – Why?
Page 10
CS148 Lecture 14 Pat Hanrahan, Fall 2009
Assumptions
Uncorrelated Correlated Anticorrelated
CS148 Lecture 14 Pat Hanrahan, Fall 2009
Premultiplied Alpha
Represent as c = aC = (ar,ag,ab,a)
Advantages Closure
Recovering C from c would require divide by a Display c ; c over K = c + (1-aC ) K = c One formula for compositing color and alpha
c = cF + (1-aF) cB
Less arithmetic Associated: OVER (1 sub, 4 muls, 4adds) Unassociated: OVER (1 sub, 7 muls, 4 adds)
Page 11
CS148 Lecture 14 Pat Hanrahan, Fall 2009
Interpolating RGBA
Two ways of interpolating an image 1. Interpolate a and C; then multiply c = a*C 2. Interpolate premultiplied colors c These lead to different answers
For example, suppose we interpolate (C, a) = (1,0,0,0) and (0,1,0,1) at ½.
1. (1/2(1+0),1/2(0+1),1/2(0+0),1/2(0+1)) = (1/2,1/2,0,1/2); then we multiply by a to get (1/4,1/4,0,1/2)
2. First we multiply to get (0,0,0,0) and (0,1,0,1); then we interpolate to get (0,1/2,0,1/2)
CS148 Lecture 14 Pat Hanrahan, Fall 2009
Which Way is Correct?
Interpolating composited images Composite foreground over the background and
then interpolate Interpolate foreground and background and then
composite These should yield the same result!
Only true if we interpolate c (method 2 on prev. slide)
Page 12
Matte Extraction
CS148 Lecture 14 Pat Hanrahan, Fall 2009
Blue Screen
Page 13
CS148 Lecture 14 Pat Hanrahan, Fall 2009
“Pulling a Matte” - Matte Creation
From digitized images Image processing
Set of colors marked transparent, region growing … Demonstration: Photoshop Magic Wand
Video or chroma-keying Range of luminances marked transparent
Blue-screen matting (Petro Vlahos) Separate blue background from foreground image
CS148 Lecture 14 Pat Hanrahan, Fall 2009
Blue/Green-Screen Matte Extraction
Given: C - Observed color (RGB) CB - Backing color = blue or green (RGB)
Compute foreground color and alpha (RGBA) cF = (aF RF, aF GF, aF BF, aF)
Page 14
CS148 Lecture 14 Pat Hanrahan, Fall 2009
Blue/Green-Screen Matte Extraction
Compositing equation: C = cF + (1-aF) CB
R = (aF RF) + (1-aF) RB
G = (aF GF) + (1-aF) GB
B = (aF BF) + (1-aF) BB
Knowns: (R,G,B) and (RB, GB, BB) Unknowns: (RF, GF, BF, aF ) or (rF, gF, bF, aF )
3 equations in 4 unknowns
CS148 Lecture 14 Pat Hanrahan, Fall 2009
Petros Vlahos Algorithm
Divide color space into two regions
The separating plane is given by the equation G = k2 B That is, colors with lots of green are outside
B
G
K
G = k2 B
CB
C Outside
C Inside
Page 15
CS148 Lecture 14 Pat Hanrahan, Fall 2009
Petros Vlahos Algorithm
Observed colors may be inside the region or outside Colors inside are opaque Colors outside are composited with Cb
B
G
G = k2 B
CB
C Outside
C Inside
K
CS148 Lecture 14 Pat Hanrahan, Fall 2009
Petros Vlahos Algorithm
Outside colors cF are on a line between CB and C C = aF CF + (1-aF) CB = cF + (1-aF) CB => cF = C - (1-aF) CB
But where? We don’t know aF
B
G CB
CF
1-aF C aF
K
Page 16
CS148 Lecture 14 Pat Hanrahan, Fall 2009
Petros Vlahos Algorithm
Outside colors are assumed to be on the plane
N cF = N C - (1-aF) A CB = 0 We now have a 4th equation and can solve for 4 unks.
B
G
K
G = k2 B
CB
CF
1-aF C N = (0,1,-k2) aF
CS148 Lecture 14 Pat Hanrahan, Fall 2009
Things to Remember
Classic techniques: masks, mattes, optical printing Definition of the alpha channel as opacity/coverage Premultiplied alpha Porter-Duff image compositing algebra Vlahos matte extraction algorithm