Date post: | 14-Jan-2016 |
Category: |
Documents |
Upload: | kelley-douglas |
View: | 220 times |
Download: | 5 times |
MTA SzTAKI & Veszprém University (Hungary)Guests at INRIA, Sophia Antipolis, 2000 and 2001
Paintbrush Rendering of Images
Tamás SzirányiTamás Szirányi
Stochastic Paintbrush RenderingStochastic Paintbrush Rendering
Stochastic relaxation method to generate images (based on a
reference image) by simulating a simple painting method,
Sharp contours, well-defined segmentation areas, elaborated fine details,
A priori models or interactions are not needed.
4
Stochastic painting method
Need for optimization
MCMC optimization
MRF and PB for segmentation
Objectives
Result of an impressionist-like process controlled by an edge mapResult of an impressionist-like process controlled by an edge map
(P. Litwinowicz, “Processing Images and Video for An Impressionist Effect”, (P. Litwinowicz, “Processing Images and Video for An Impressionist Effect”, Computer Computer GraphicsGraphics, Proc.SIGGRAPH’1997, , Proc.SIGGRAPH’1997, 1997.)1997.)
Paintings of A. Dürer and M. Munkácsy, representing careful life-style painting:Paintings of A. Dürer and M. Munkácsy, representing careful life-style painting: the painter tries to elaborate the painting without visible effects of the the painter tries to elaborate the painting without visible effects of the
brush-strokesbrush-strokes
All PB strokes are accepted, then redundants
are removed
Only good matching is accepted
A need for optimal estimation of stroke placement
Markov Chain Monte Carlo (MCMC) methods
Ergodic Markov-chain (X1,X2,…. Xn)Stationary f(X)
Sample series (X1,X2,…. Xn) generated with target density f(X)Paintbrush strokes:
X is the position of a stroke, f(X) is the probability density of distortion error between the stroke and the reference image at position X
Probability densities of distortion error of the proposed ( Probability densities of distortion error of the proposed ( YYtt) and accepted ) and accepted
((X X t+1t+1=Y=Ytt) strokes versus the original image when generating the strokes ) strokes versus the original image when generating the strokes
for ‘Barbara’ image. First, coarse (20x5), finally, fine (7x2) strokes are for ‘Barbara’ image. First, coarse (20x5), finally, fine (7x2) strokes are generatedgenerated
Distortion Error of PB Strokes vs. Original Image
0
0,05
0,1
0,15
0,2
0,25
1 4 7 10
13
16
19
22
25
% of Distortion
Pro
b. D
en
sit
y
Distortion Error ofProposed Strokes (Coarse PB)
Distortion Error ofProposed Strokes (FinePB)
Distortion Error ofAccepted Strokes (Coarse PB)
Distortion Error ofAccepted Strokes (Fine PB)
tY
Probability densities of Probability densities of difference btw distortion errorsdifference btw distortion errors of the of the proposed/proposed/acceptedaccepted strokes and the strokes and the distortion on previous area of the strokedistortion on previous area of the stroke ( ( DiffDiff(Y(Ytt) ) ), when generating the strokes for ‘Barbara’ image. First, coarse ), when generating the strokes for ‘Barbara’ image. First, coarse
(20x5), finally, fine (7x2) strokes are generated.(20x5), finally, fine (7x2) strokes are generated.
Difference of Distortion Errors at the Proposed/Accepted Positions
0
0,1
0,2
0,3
0,4
0,5
0,6
0,7
0,8
0,9
1 3 5 7 9 11
13
15
17
% of Distortion Difference
Pro
b. D
en
sit
y
Difference of Distortion Errors of AcceptedStrokes (Coarse PB)vs. previous values
Difference of Distortion Errors of AcceptedStrokes (Fine PB) vs.previous values
Difference of Distortion Errors of ProposedStrokes vs. previousvalues
The characteristic variable is the The characteristic variable is the Difference of distortion error instead of Difference of distortion error instead of
the errorthe error
)Diff(Yt )E(Y)E(x t)t(
)Diff(x t )( )E(x)E(x )t()t( 1
accept/reject ruleaccept/reject rule
ttt
tt
tt
Yxx
YxYX
,-1y probabilit with
,y probabilit with )()(
)()1(
1 ,)x|y(q
)y|x(q
)x(f
)y(fminy,x
else ,
0 if 0q
t
t
)Diff(Y
)Diff(Y,~yf
q)t( )Diff(x~xf
31
Narrowing effect of the distribution of the measured conditional probability
proposedtacceptedt )Diff(YP)Diff(YP /
32
else ,
if q
t
t
t
Var)Diff(Y
)Diff(Yyf
0,0~
q
t
t
Var)Diff(xxf
)(
~
ttt
tt
tt
Yxx
YxYX
,
,)()(
)()1(
-1
y probabilitwith
yprobabilitwith
1 ,)|(
)|(
)(
)(min,
xyq
yxq
xf
yfyx
Constraints for the flexible accept/reject rule
Random Stochastic search
35 61 2645
Metropolis Hastings process
26 31 1555
Method Number of non-
redundant PB strokes# thousand
Number of all drawn strokes
#
Number of all proposed strokes
# acceptedt )(Y
proposedtY
Limiting the number of colorsLimiting the number of colors
Coupling the neighboring strokes
MRF-like segmentation
Reference area is greater than the stroke’s area
Halftone effects by strokes (modulation)
Connection btw MRF segmentation and PB
En1:
Difference inside the area of a
brush-stroke
between
the original
and the proposed
stroke
En2: Difference between the color of the stroke and the present neighboring pixels at the boundary of the stroke
En=En1+(1- )En2
0 < < 1.0
Segmentation for 4 colors (Trinois 128*128) Running time <-> Misclassification
0
1000
2000
3000
4000
5000
6000
7000
0 5 10 15 20 25 30 35 40
Running time (relative CPU)
# of
mis
clas
sifie
d pi
xels
7x3 PB, Delta E1 < 0, ~MMD
10x4 PB, Delta E1 < 0, ~MMD
13x5 PB, Delta E1 < 0, ~MMD
20x6 PB, Delta E1 < 0, ~MMD
1x1, Delta E1 < 0, ~MMD MRF,Beta=0.1
1x1, Delta E1 < 0, ~MMD MRF,Beta=0.75
1x1, ~MMD MRF, Beta=0.5
45
Example Example 2/2a2/2a
frame1: keyframe1 frame2: painted motion area 1 frame3: painted motion area 2
frame4: painted motion area 3 frame5: keyframe2
•1. and 5. are keyframes
46
Example Example 2/2b2/2b
frame1: keyframe1 frame2: painted motion area 1 frame3: painted motion area 2
frame4: painted motion area 3 frame5: keyframe2
51
Segmentation for 2 colors Running time <-> Misclassification
0
20000
40000
60000
80000
100000
120000
0 20 40 60 80 100 120 140 160
Running time (relative CPU)
# o
f m
iscl
ass
ifie
d p
ixe
ls
1x1 PB, ~MMD MRF
5x5 PB, ~MMD
10x10 PB, ~MMD
12x12 PB, ~MMD
14x14 PB, ~MMD
15x15 PB, ~MMD
Segmentation by painting, when distortion error may increase
52
EnergEnergy calculus in optimization of y calculus in optimization of MRF MRF segmentationsegmentation
rsrss
ss V
,E
C,
2
s
2
2
2
Distortion between the original input color and the proposed random
value
rs
rsrsV
if ,
if ,,
Distortion among the neighboring pixels
IndependentIndependent MH MH
Since the probability of acceptance of Yt depends on X(t), the resulting sample is not independent and identically distributed (iid). X(t) is irreducible and aperiodic (thus ergodic) iff g is almost everywhere positive on the support of f. The above algorithm produces a uniformly ergodic chain if there exists a constant M such that