Background vs foreground segmentation of video sequences

= +

The Problem

bull Separate video into two layersndash stationary backgroundndash moving foreground

bull Sequence is very noisy reference image (background) is not given

Simple approach (1)

temporal mean

background

temporal median

Simple approach (2)

threshold

Simple approach noise can spoil everything

The Problem

bull Separate video into two layersndash stationary backgroundndash moving foreground

bull Sequence is very noisy reference image (background) is not given

Simple approach (1)

temporal mean

background

temporal median

Simple approach (2)

threshold

Simple approach noise can spoil everything

Simple approach (1)

temporal mean

background

temporal median

Simple approach (2)

threshold

Simple approach noise can spoil everything

Simple approach (2)

threshold

Simple approach noise can spoil everything

Simple approach noise can spoil everything

Variational approach

Find the background and foregroundsimultaneously by minimizing energy functional

Bonus remove noise

Notations

[0t

max ]

N(xt) original noisy sequence

B(x) background image

C(xt) background mask(1 on background 0 on foreground)

give

n

need

to fi

nd

Energy functional data term

B N

B - N C

Energy functional data term

Degeneracy can be trivially minimized bybull C 0 (everything is foreground)

bull B N (take original image as background)

Energy functional data term

C 1

Energy functional data term

there should be enough of background

original images should be close to the restored background image

in the background areas

Energy functional smoothness

For background image B

For background mask C

Energy functional

Edge-preserving smoothnessRegularization term

Quadratic regularization [Tikhonov Arsenin 1977]

ELE

Known to produce very strong isotropic smoothing

Edge-preserving smoothnessRegularization term

Change regularization

ELE

Edge-preserving smoothnessRegularization term

ELE

Edge-preserving smoothnessRegularization term

ELE

n

Change the coordinate system

across the edgealong the edge

Compare

Edge-preserving smoothnessRegularization term

Weak edge (s 0)

Conditions on

Isotropic smoothings) is quadratic at zero

(s)

s

Edge-preserving smoothnessRegularization term

Strong edge (s )

Conditions on

bull no smoothing across the edge

bull more smoothing along the edge

Anisotropic smoothings) does not grow too fast at infinity

(s)

s

Edge-preserving smoothnessRegularization term

ConclusionUsing regularization term of the form

we can achieve both

isotropic smoothness in uniform regions

and anisotropic smoothness on edges

with one function

0 1 2 3 4 50

05

1

15

2

25

3

35

4

Edge-preserving smoothnessRegularization term

Example of an edge-preserving function

0 005 01 015 020

0005

001

0015

002

Edge-preserving smoothnessSpace of Bounded Variations

Even if we have an edge-preserving functional

if the space of solutions u contains only smooth functions we may not achieve the desired minimum

-1 -05 0 05 1-1

-05

0

05

1

Edge-preserving smoothnessSpace of Bounded Variations

which one is ldquobetterrdquo

Bounded Variation ndash ND caseBounded Variation ndash ND case

sup |)(| dxdivffD

)( x

N

1i i

i xdiv

1 || )( ) ( )(

101

L

NN C

bounded open subset function NR )( 1 Lf

Variation of over f

where

φ

Edge-preserving smoothnessSpace of Bounded Variations

integrable (absolute value) and with bounded variation

Functions are not required to have an integrable derivative hellip

What is the meaning of u in the regularization term

Intuitively norm of gradient |u| is

replaced with variation |Du|

Total variation

Theorem (informally) if u BV() then

Hausdorff measure

area gt 0area = 0

How can we measure zero-measure sets

Hausdorff measure

1) cover with balls of diameter

2) sum up diameters for optimal cover (do not waste balls)

3) refine 0

Hausdorff measureFormally

For A RN k-dimensional Hausdorff measure of A

up to normalization factor covers are countable

bull HN is just the Lebesgue measure

bull curve in image

its length = H1 in R2

Total variation

Theorem (more formally) if u BV() then

u+

u-

u(x)

xx0

u+ u- - approximate upper and lower limits

Su = x u+gtu-the jump set

Energy functional

data term

regularization for background image

regularization for background masks

Total variation example

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

= perimeter = 4

Divide each side into n parts

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation(= sum of perimeters)

Large total variation(= sum of perimeters)

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation Large total variation

Edge-preserving smoothnessSpace of Bounded Variations

BV informally functions with discontinuities on curves

Edges are preserved texture is not preserved

original sequencetemporal median energy minimization

in BV

Energy functional

Time-discretized problem

Find minimum of E subject to

Existence of solution

Under usual assumptions

12 R+ R+ strictly convex nondecreasing

with linear growth at infinity

minimum of E exists in BV(BC1hellipCT)

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Notations

[0t

max ]

N(xt) original noisy sequence

B(x) background image

C(xt) background mask(1 on background 0 on foreground)

give

n

need

to fi

nd

Energy functional data term

B N

B - N C

Energy functional data term

Degeneracy can be trivially minimized bybull C 0 (everything is foreground)

bull B N (take original image as background)

Energy functional data term

C 1

Energy functional data term

there should be enough of background

original images should be close to the restored background image

in the background areas

Energy functional smoothness

For background image B

For background mask C

Energy functional

Edge-preserving smoothnessRegularization term

Quadratic regularization [Tikhonov Arsenin 1977]

ELE

Known to produce very strong isotropic smoothing

Edge-preserving smoothnessRegularization term

Change regularization

ELE

Edge-preserving smoothnessRegularization term

ELE

Edge-preserving smoothnessRegularization term

ELE

n

Change the coordinate system

across the edgealong the edge

Compare

Edge-preserving smoothnessRegularization term

Weak edge (s 0)

Conditions on

Isotropic smoothings) is quadratic at zero

(s)

s

Edge-preserving smoothnessRegularization term

Strong edge (s )

Conditions on

bull no smoothing across the edge

bull more smoothing along the edge

Anisotropic smoothings) does not grow too fast at infinity

(s)

s

Edge-preserving smoothnessRegularization term

ConclusionUsing regularization term of the form

we can achieve both

isotropic smoothness in uniform regions

and anisotropic smoothness on edges

with one function

0 1 2 3 4 50

05

1

15

2

25

3

35

4

Edge-preserving smoothnessRegularization term

Example of an edge-preserving function

0 005 01 015 020

0005

001

0015

002

Edge-preserving smoothnessSpace of Bounded Variations

Even if we have an edge-preserving functional

if the space of solutions u contains only smooth functions we may not achieve the desired minimum

-1 -05 0 05 1-1

-05

0

05

1

Edge-preserving smoothnessSpace of Bounded Variations

which one is ldquobetterrdquo

Bounded Variation ndash ND caseBounded Variation ndash ND case

sup |)(| dxdivffD

)( x

N

1i i

i xdiv

1 || )( ) ( )(

101

L

NN C

bounded open subset function NR )( 1 Lf

Variation of over f

where

φ

Edge-preserving smoothnessSpace of Bounded Variations

integrable (absolute value) and with bounded variation

Functions are not required to have an integrable derivative hellip

What is the meaning of u in the regularization term

Intuitively norm of gradient |u| is

replaced with variation |Du|

Total variation

Theorem (informally) if u BV() then

Hausdorff measure

area gt 0area = 0

How can we measure zero-measure sets

Hausdorff measure

1) cover with balls of diameter

2) sum up diameters for optimal cover (do not waste balls)

3) refine 0

Hausdorff measureFormally

For A RN k-dimensional Hausdorff measure of A

up to normalization factor covers are countable

bull HN is just the Lebesgue measure

bull curve in image

its length = H1 in R2

Total variation

Theorem (more formally) if u BV() then

u+

u-

u(x)

xx0

u+ u- - approximate upper and lower limits

Su = x u+gtu-the jump set

Energy functional

data term

regularization for background image

regularization for background masks

Total variation example

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

= perimeter = 4

Divide each side into n parts

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation(= sum of perimeters)

Large total variation(= sum of perimeters)

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation Large total variation

Edge-preserving smoothnessSpace of Bounded Variations

BV informally functions with discontinuities on curves

Edges are preserved texture is not preserved

original sequencetemporal median energy minimization

in BV

Energy functional

Time-discretized problem

Find minimum of E subject to

Existence of solution

Under usual assumptions

12 R+ R+ strictly convex nondecreasing

with linear growth at infinity

minimum of E exists in BV(BC1hellipCT)

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Energy functional data term

B N

B - N C

Energy functional data term

Degeneracy can be trivially minimized bybull C 0 (everything is foreground)

bull B N (take original image as background)

Energy functional data term

C 1

Energy functional data term

there should be enough of background

original images should be close to the restored background image

in the background areas

Energy functional smoothness

For background image B

For background mask C

Energy functional

Edge-preserving smoothnessRegularization term

Quadratic regularization [Tikhonov Arsenin 1977]

ELE

Known to produce very strong isotropic smoothing

Edge-preserving smoothnessRegularization term

Change regularization

ELE

Edge-preserving smoothnessRegularization term

ELE

Edge-preserving smoothnessRegularization term

ELE

n

Change the coordinate system

across the edgealong the edge

Compare

Edge-preserving smoothnessRegularization term

Weak edge (s 0)

Conditions on

Isotropic smoothings) is quadratic at zero

(s)

s

Edge-preserving smoothnessRegularization term

Strong edge (s )

Conditions on

bull no smoothing across the edge

bull more smoothing along the edge

Anisotropic smoothings) does not grow too fast at infinity

(s)

s

Edge-preserving smoothnessRegularization term

ConclusionUsing regularization term of the form

we can achieve both

isotropic smoothness in uniform regions

and anisotropic smoothness on edges

with one function

0 1 2 3 4 50

05

1

15

2

25

3

35

4

Edge-preserving smoothnessRegularization term

Example of an edge-preserving function

0 005 01 015 020

0005

001

0015

002

Edge-preserving smoothnessSpace of Bounded Variations

Even if we have an edge-preserving functional

if the space of solutions u contains only smooth functions we may not achieve the desired minimum

-1 -05 0 05 1-1

-05

0

05

1

Edge-preserving smoothnessSpace of Bounded Variations

which one is ldquobetterrdquo

Bounded Variation ndash ND caseBounded Variation ndash ND case

sup |)(| dxdivffD

)( x

N

1i i

i xdiv

1 || )( ) ( )(

101

L

NN C

bounded open subset function NR )( 1 Lf

Variation of over f

where

φ

Edge-preserving smoothnessSpace of Bounded Variations

integrable (absolute value) and with bounded variation

Functions are not required to have an integrable derivative hellip

What is the meaning of u in the regularization term

Intuitively norm of gradient |u| is

replaced with variation |Du|

Total variation

Theorem (informally) if u BV() then

Hausdorff measure

area gt 0area = 0

How can we measure zero-measure sets

Hausdorff measure

1) cover with balls of diameter

2) sum up diameters for optimal cover (do not waste balls)

3) refine 0

Hausdorff measureFormally

For A RN k-dimensional Hausdorff measure of A

up to normalization factor covers are countable

bull HN is just the Lebesgue measure

bull curve in image

its length = H1 in R2

Total variation

Theorem (more formally) if u BV() then

u+

u-

u(x)

xx0

u+ u- - approximate upper and lower limits

Su = x u+gtu-the jump set

Energy functional

data term

regularization for background image

regularization for background masks

Total variation example

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

= perimeter = 4

Divide each side into n parts

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation(= sum of perimeters)

Large total variation(= sum of perimeters)

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation Large total variation

Edge-preserving smoothnessSpace of Bounded Variations

BV informally functions with discontinuities on curves

Edges are preserved texture is not preserved

original sequencetemporal median energy minimization

in BV

Energy functional

Time-discretized problem

Find minimum of E subject to

Existence of solution

Under usual assumptions

12 R+ R+ strictly convex nondecreasing

with linear growth at infinity

minimum of E exists in BV(BC1hellipCT)

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Energy functional data term

Degeneracy can be trivially minimized bybull C 0 (everything is foreground)

bull B N (take original image as background)

Energy functional data term

C 1

Energy functional data term

there should be enough of background

original images should be close to the restored background image

in the background areas

Energy functional smoothness

For background image B

For background mask C

Energy functional

Edge-preserving smoothnessRegularization term

Quadratic regularization [Tikhonov Arsenin 1977]

ELE

Known to produce very strong isotropic smoothing

Edge-preserving smoothnessRegularization term

Change regularization

ELE

Edge-preserving smoothnessRegularization term

ELE

Edge-preserving smoothnessRegularization term

ELE

n

Change the coordinate system

across the edgealong the edge

Compare

Edge-preserving smoothnessRegularization term

Weak edge (s 0)

Conditions on

Isotropic smoothings) is quadratic at zero

(s)

s

Edge-preserving smoothnessRegularization term

Strong edge (s )

Conditions on

bull no smoothing across the edge

bull more smoothing along the edge

Anisotropic smoothings) does not grow too fast at infinity

(s)

s

Edge-preserving smoothnessRegularization term

ConclusionUsing regularization term of the form

we can achieve both

isotropic smoothness in uniform regions

and anisotropic smoothness on edges

with one function

0 1 2 3 4 50

05

1

15

2

25

3

35

4

Edge-preserving smoothnessRegularization term

Example of an edge-preserving function

0 005 01 015 020

0005

001

0015

002

Edge-preserving smoothnessSpace of Bounded Variations

Even if we have an edge-preserving functional

if the space of solutions u contains only smooth functions we may not achieve the desired minimum

-1 -05 0 05 1-1

-05

0

05

1

Edge-preserving smoothnessSpace of Bounded Variations

which one is ldquobetterrdquo

Bounded Variation ndash ND caseBounded Variation ndash ND case

sup |)(| dxdivffD

)( x

N

1i i

i xdiv

1 || )( ) ( )(

101

L

NN C

bounded open subset function NR )( 1 Lf

Variation of over f

where

φ

Edge-preserving smoothnessSpace of Bounded Variations

integrable (absolute value) and with bounded variation

Functions are not required to have an integrable derivative hellip

What is the meaning of u in the regularization term

Intuitively norm of gradient |u| is

replaced with variation |Du|

Total variation

Theorem (informally) if u BV() then

Hausdorff measure

area gt 0area = 0

How can we measure zero-measure sets

Hausdorff measure

1) cover with balls of diameter

2) sum up diameters for optimal cover (do not waste balls)

3) refine 0

Hausdorff measureFormally

For A RN k-dimensional Hausdorff measure of A

up to normalization factor covers are countable

bull HN is just the Lebesgue measure

bull curve in image

its length = H1 in R2

Total variation

Theorem (more formally) if u BV() then

u+

u-

u(x)

xx0

u+ u- - approximate upper and lower limits

Su = x u+gtu-the jump set

Energy functional

data term

regularization for background image

regularization for background masks

Total variation example

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

= perimeter = 4

Divide each side into n parts

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation(= sum of perimeters)

Large total variation(= sum of perimeters)

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation Large total variation

Edge-preserving smoothnessSpace of Bounded Variations

BV informally functions with discontinuities on curves

Edges are preserved texture is not preserved

original sequencetemporal median energy minimization

in BV

Energy functional

Time-discretized problem

Find minimum of E subject to

Existence of solution

Under usual assumptions

12 R+ R+ strictly convex nondecreasing

with linear growth at infinity

minimum of E exists in BV(BC1hellipCT)

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Energy functional data term

C 1

Energy functional data term

there should be enough of background

original images should be close to the restored background image

in the background areas

Energy functional smoothness

For background image B

For background mask C

Energy functional

Edge-preserving smoothnessRegularization term

Quadratic regularization [Tikhonov Arsenin 1977]

ELE

Known to produce very strong isotropic smoothing

Edge-preserving smoothnessRegularization term

Change regularization

ELE

Edge-preserving smoothnessRegularization term

ELE

Edge-preserving smoothnessRegularization term

ELE

n

Change the coordinate system

across the edgealong the edge

Compare

Edge-preserving smoothnessRegularization term

Weak edge (s 0)

Conditions on

Isotropic smoothings) is quadratic at zero

(s)

s

Edge-preserving smoothnessRegularization term

Strong edge (s )

Conditions on

bull no smoothing across the edge

bull more smoothing along the edge

Anisotropic smoothings) does not grow too fast at infinity

(s)

s

Edge-preserving smoothnessRegularization term

ConclusionUsing regularization term of the form

we can achieve both

isotropic smoothness in uniform regions

and anisotropic smoothness on edges

with one function

0 1 2 3 4 50

05

1

15

2

25

3

35

4

Edge-preserving smoothnessRegularization term

Example of an edge-preserving function

0 005 01 015 020

0005

001

0015

002

Edge-preserving smoothnessSpace of Bounded Variations

Even if we have an edge-preserving functional

if the space of solutions u contains only smooth functions we may not achieve the desired minimum

-1 -05 0 05 1-1

-05

0

05

1

Edge-preserving smoothnessSpace of Bounded Variations

which one is ldquobetterrdquo

Bounded Variation ndash ND caseBounded Variation ndash ND case

sup |)(| dxdivffD

)( x

N

1i i

i xdiv

1 || )( ) ( )(

101

L

NN C

bounded open subset function NR )( 1 Lf

Variation of over f

where

φ

Edge-preserving smoothnessSpace of Bounded Variations

integrable (absolute value) and with bounded variation

Functions are not required to have an integrable derivative hellip

What is the meaning of u in the regularization term

Intuitively norm of gradient |u| is

replaced with variation |Du|

Total variation

Theorem (informally) if u BV() then

Hausdorff measure

area gt 0area = 0

How can we measure zero-measure sets

Hausdorff measure

1) cover with balls of diameter

2) sum up diameters for optimal cover (do not waste balls)

3) refine 0

Hausdorff measureFormally

For A RN k-dimensional Hausdorff measure of A

up to normalization factor covers are countable

bull HN is just the Lebesgue measure

bull curve in image

its length = H1 in R2

Total variation

Theorem (more formally) if u BV() then

u+

u-

u(x)

xx0

u+ u- - approximate upper and lower limits

Su = x u+gtu-the jump set

Energy functional

data term

regularization for background image

regularization for background masks

Total variation example

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

= perimeter = 4

Divide each side into n parts

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation(= sum of perimeters)

Large total variation(= sum of perimeters)

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation Large total variation

Edge-preserving smoothnessSpace of Bounded Variations

BV informally functions with discontinuities on curves

Edges are preserved texture is not preserved

original sequencetemporal median energy minimization

in BV

Energy functional

Time-discretized problem

Find minimum of E subject to

Existence of solution

Under usual assumptions

12 R+ R+ strictly convex nondecreasing

with linear growth at infinity

minimum of E exists in BV(BC1hellipCT)

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Energy functional data term

there should be enough of background

original images should be close to the restored background image

in the background areas

Energy functional smoothness

For background image B

For background mask C

Energy functional

Edge-preserving smoothnessRegularization term

Quadratic regularization [Tikhonov Arsenin 1977]

ELE

Known to produce very strong isotropic smoothing

Edge-preserving smoothnessRegularization term

Change regularization

ELE

Edge-preserving smoothnessRegularization term

ELE

Edge-preserving smoothnessRegularization term

ELE

n

Change the coordinate system

across the edgealong the edge

Compare

Edge-preserving smoothnessRegularization term

Weak edge (s 0)

Conditions on

Isotropic smoothings) is quadratic at zero

(s)

s

Edge-preserving smoothnessRegularization term

Strong edge (s )

Conditions on

bull no smoothing across the edge

bull more smoothing along the edge

Anisotropic smoothings) does not grow too fast at infinity

(s)

s

Edge-preserving smoothnessRegularization term

ConclusionUsing regularization term of the form

we can achieve both

isotropic smoothness in uniform regions

and anisotropic smoothness on edges

with one function

0 1 2 3 4 50

05

1

15

2

25

3

35

4

Edge-preserving smoothnessRegularization term

Example of an edge-preserving function

0 005 01 015 020

0005

001

0015

002

Edge-preserving smoothnessSpace of Bounded Variations

Even if we have an edge-preserving functional

if the space of solutions u contains only smooth functions we may not achieve the desired minimum

-1 -05 0 05 1-1

-05

0

05

1

Edge-preserving smoothnessSpace of Bounded Variations

which one is ldquobetterrdquo

Bounded Variation ndash ND caseBounded Variation ndash ND case

sup |)(| dxdivffD

)( x

N

1i i

i xdiv

1 || )( ) ( )(

101

L

NN C

bounded open subset function NR )( 1 Lf

Variation of over f

where

φ

Edge-preserving smoothnessSpace of Bounded Variations

integrable (absolute value) and with bounded variation

Functions are not required to have an integrable derivative hellip

What is the meaning of u in the regularization term

Intuitively norm of gradient |u| is

replaced with variation |Du|

Total variation

Theorem (informally) if u BV() then

Hausdorff measure

area gt 0area = 0

How can we measure zero-measure sets

Hausdorff measure

1) cover with balls of diameter

2) sum up diameters for optimal cover (do not waste balls)

3) refine 0

Hausdorff measureFormally

For A RN k-dimensional Hausdorff measure of A

up to normalization factor covers are countable

bull HN is just the Lebesgue measure

bull curve in image

its length = H1 in R2

Total variation

Theorem (more formally) if u BV() then

u+

u-

u(x)

xx0

u+ u- - approximate upper and lower limits

Su = x u+gtu-the jump set

Energy functional

data term

regularization for background image

regularization for background masks

Total variation example

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

= perimeter = 4

Divide each side into n parts

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation(= sum of perimeters)

Large total variation(= sum of perimeters)

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation Large total variation

Edge-preserving smoothnessSpace of Bounded Variations

BV informally functions with discontinuities on curves

Edges are preserved texture is not preserved

original sequencetemporal median energy minimization

in BV

Energy functional

Time-discretized problem

Find minimum of E subject to

Existence of solution

Under usual assumptions

12 R+ R+ strictly convex nondecreasing

with linear growth at infinity

minimum of E exists in BV(BC1hellipCT)

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Energy functional smoothness

For background image B

For background mask C

Energy functional

Edge-preserving smoothnessRegularization term

Quadratic regularization [Tikhonov Arsenin 1977]

ELE

Known to produce very strong isotropic smoothing

Edge-preserving smoothnessRegularization term

Change regularization

ELE

Edge-preserving smoothnessRegularization term

ELE

Edge-preserving smoothnessRegularization term

ELE

n

Change the coordinate system

across the edgealong the edge

Compare

Edge-preserving smoothnessRegularization term

Weak edge (s 0)

Conditions on

Isotropic smoothings) is quadratic at zero

(s)

s

Edge-preserving smoothnessRegularization term

Strong edge (s )

Conditions on

bull no smoothing across the edge

bull more smoothing along the edge

Anisotropic smoothings) does not grow too fast at infinity

(s)

s

Edge-preserving smoothnessRegularization term

ConclusionUsing regularization term of the form

we can achieve both

isotropic smoothness in uniform regions

and anisotropic smoothness on edges

with one function

0 1 2 3 4 50

05

1

15

2

25

3

35

4

Edge-preserving smoothnessRegularization term

Example of an edge-preserving function

0 005 01 015 020

0005

001

0015

002

Edge-preserving smoothnessSpace of Bounded Variations

Even if we have an edge-preserving functional

if the space of solutions u contains only smooth functions we may not achieve the desired minimum

-1 -05 0 05 1-1

-05

0

05

1

Edge-preserving smoothnessSpace of Bounded Variations

which one is ldquobetterrdquo

Bounded Variation ndash ND caseBounded Variation ndash ND case

sup |)(| dxdivffD

)( x

N

1i i

i xdiv

1 || )( ) ( )(

101

L

NN C

bounded open subset function NR )( 1 Lf

Variation of over f

where

φ

Edge-preserving smoothnessSpace of Bounded Variations

integrable (absolute value) and with bounded variation

Functions are not required to have an integrable derivative hellip

What is the meaning of u in the regularization term

Intuitively norm of gradient |u| is

replaced with variation |Du|

Total variation

Theorem (informally) if u BV() then

Hausdorff measure

area gt 0area = 0

How can we measure zero-measure sets

Hausdorff measure

1) cover with balls of diameter

2) sum up diameters for optimal cover (do not waste balls)

3) refine 0

Hausdorff measureFormally

For A RN k-dimensional Hausdorff measure of A

up to normalization factor covers are countable

bull HN is just the Lebesgue measure

bull curve in image

its length = H1 in R2

Total variation

Theorem (more formally) if u BV() then

u+

u-

u(x)

xx0

u+ u- - approximate upper and lower limits

Su = x u+gtu-the jump set

Energy functional

data term

regularization for background image

regularization for background masks

Total variation example

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

= perimeter = 4

Divide each side into n parts

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation(= sum of perimeters)

Large total variation(= sum of perimeters)

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation Large total variation

Edge-preserving smoothnessSpace of Bounded Variations

BV informally functions with discontinuities on curves

Edges are preserved texture is not preserved

original sequencetemporal median energy minimization

in BV

Energy functional

Time-discretized problem

Find minimum of E subject to

Existence of solution

Under usual assumptions

12 R+ R+ strictly convex nondecreasing

with linear growth at infinity

minimum of E exists in BV(BC1hellipCT)

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Energy functional

Edge-preserving smoothnessRegularization term

Quadratic regularization [Tikhonov Arsenin 1977]

ELE

Known to produce very strong isotropic smoothing

Edge-preserving smoothnessRegularization term

Change regularization

ELE

Edge-preserving smoothnessRegularization term

ELE

Edge-preserving smoothnessRegularization term

ELE

n

Change the coordinate system

across the edgealong the edge

Compare

Edge-preserving smoothnessRegularization term

Weak edge (s 0)

Conditions on

Isotropic smoothings) is quadratic at zero

(s)

s

Edge-preserving smoothnessRegularization term

Strong edge (s )

Conditions on

bull no smoothing across the edge

bull more smoothing along the edge

Anisotropic smoothings) does not grow too fast at infinity

(s)

s

Edge-preserving smoothnessRegularization term

ConclusionUsing regularization term of the form

we can achieve both

isotropic smoothness in uniform regions

and anisotropic smoothness on edges

with one function

0 1 2 3 4 50

05

1

15

2

25

3

35

4

Edge-preserving smoothnessRegularization term

Example of an edge-preserving function

0 005 01 015 020

0005

001

0015

002

Edge-preserving smoothnessSpace of Bounded Variations

Even if we have an edge-preserving functional

if the space of solutions u contains only smooth functions we may not achieve the desired minimum

-1 -05 0 05 1-1

-05

0

05

1

Edge-preserving smoothnessSpace of Bounded Variations

which one is ldquobetterrdquo

Bounded Variation ndash ND caseBounded Variation ndash ND case

sup |)(| dxdivffD

)( x

N

1i i

i xdiv

1 || )( ) ( )(

101

L

NN C

bounded open subset function NR )( 1 Lf

Variation of over f

where

φ

Edge-preserving smoothnessSpace of Bounded Variations

integrable (absolute value) and with bounded variation

Functions are not required to have an integrable derivative hellip

What is the meaning of u in the regularization term

Intuitively norm of gradient |u| is

replaced with variation |Du|

Total variation

Theorem (informally) if u BV() then

Hausdorff measure

area gt 0area = 0

How can we measure zero-measure sets

Hausdorff measure

1) cover with balls of diameter

2) sum up diameters for optimal cover (do not waste balls)

3) refine 0

Hausdorff measureFormally

For A RN k-dimensional Hausdorff measure of A

up to normalization factor covers are countable

bull HN is just the Lebesgue measure

bull curve in image

its length = H1 in R2

Total variation

Theorem (more formally) if u BV() then

u+

u-

u(x)

xx0

u+ u- - approximate upper and lower limits

Su = x u+gtu-the jump set

Energy functional

data term

regularization for background image

regularization for background masks

Total variation example

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

= perimeter = 4

Divide each side into n parts

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation(= sum of perimeters)

Large total variation(= sum of perimeters)

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation Large total variation

Edge-preserving smoothnessSpace of Bounded Variations

BV informally functions with discontinuities on curves

Edges are preserved texture is not preserved

original sequencetemporal median energy minimization

in BV

Energy functional

Time-discretized problem

Find minimum of E subject to

Existence of solution

Under usual assumptions

12 R+ R+ strictly convex nondecreasing

with linear growth at infinity

minimum of E exists in BV(BC1hellipCT)

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Edge-preserving smoothnessRegularization term

Quadratic regularization [Tikhonov Arsenin 1977]

ELE

Known to produce very strong isotropic smoothing

Edge-preserving smoothnessRegularization term

Change regularization

ELE

Edge-preserving smoothnessRegularization term

ELE

Edge-preserving smoothnessRegularization term

ELE

n

Change the coordinate system

across the edgealong the edge

Compare

Edge-preserving smoothnessRegularization term

Weak edge (s 0)

Conditions on

Isotropic smoothings) is quadratic at zero

(s)

s

Edge-preserving smoothnessRegularization term

Strong edge (s )

Conditions on

bull no smoothing across the edge

bull more smoothing along the edge

Anisotropic smoothings) does not grow too fast at infinity

(s)

s

Edge-preserving smoothnessRegularization term

ConclusionUsing regularization term of the form

we can achieve both

isotropic smoothness in uniform regions

and anisotropic smoothness on edges

with one function

0 1 2 3 4 50

05

1

15

2

25

3

35

4

Edge-preserving smoothnessRegularization term

Example of an edge-preserving function

0 005 01 015 020

0005

001

0015

002

Edge-preserving smoothnessSpace of Bounded Variations

Even if we have an edge-preserving functional

if the space of solutions u contains only smooth functions we may not achieve the desired minimum

-1 -05 0 05 1-1

-05

0

05

1

Edge-preserving smoothnessSpace of Bounded Variations

which one is ldquobetterrdquo

Bounded Variation ndash ND caseBounded Variation ndash ND case

sup |)(| dxdivffD

)( x

N

1i i

i xdiv

1 || )( ) ( )(

101

L

NN C

bounded open subset function NR )( 1 Lf

Variation of over f

where

φ

Edge-preserving smoothnessSpace of Bounded Variations

integrable (absolute value) and with bounded variation

Functions are not required to have an integrable derivative hellip

What is the meaning of u in the regularization term

Intuitively norm of gradient |u| is

replaced with variation |Du|

Total variation

Theorem (informally) if u BV() then

Hausdorff measure

area gt 0area = 0

How can we measure zero-measure sets

Hausdorff measure

1) cover with balls of diameter

2) sum up diameters for optimal cover (do not waste balls)

3) refine 0

Hausdorff measureFormally

For A RN k-dimensional Hausdorff measure of A

up to normalization factor covers are countable

bull HN is just the Lebesgue measure

bull curve in image

its length = H1 in R2

Total variation

Theorem (more formally) if u BV() then

u+

u-

u(x)

xx0

u+ u- - approximate upper and lower limits

Su = x u+gtu-the jump set

Energy functional

data term

regularization for background image

regularization for background masks

Total variation example

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

= perimeter = 4

Divide each side into n parts

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation(= sum of perimeters)

Large total variation(= sum of perimeters)

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation Large total variation

Edge-preserving smoothnessSpace of Bounded Variations

BV informally functions with discontinuities on curves

Edges are preserved texture is not preserved

original sequencetemporal median energy minimization

in BV

Energy functional

Time-discretized problem

Find minimum of E subject to

Existence of solution

Under usual assumptions

12 R+ R+ strictly convex nondecreasing

with linear growth at infinity

minimum of E exists in BV(BC1hellipCT)

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Edge-preserving smoothnessRegularization term

Change regularization

ELE

Edge-preserving smoothnessRegularization term

ELE

Edge-preserving smoothnessRegularization term

ELE

n

Change the coordinate system

across the edgealong the edge

Compare

Edge-preserving smoothnessRegularization term

Weak edge (s 0)

Conditions on

Isotropic smoothings) is quadratic at zero

(s)

s

Edge-preserving smoothnessRegularization term

Strong edge (s )

Conditions on

bull no smoothing across the edge

bull more smoothing along the edge

Anisotropic smoothings) does not grow too fast at infinity

(s)

s

Edge-preserving smoothnessRegularization term

ConclusionUsing regularization term of the form

we can achieve both

isotropic smoothness in uniform regions

and anisotropic smoothness on edges

with one function

0 1 2 3 4 50

05

1

15

2

25

3

35

4

Edge-preserving smoothnessRegularization term

Example of an edge-preserving function

0 005 01 015 020

0005

001

0015

002

Edge-preserving smoothnessSpace of Bounded Variations

Even if we have an edge-preserving functional

if the space of solutions u contains only smooth functions we may not achieve the desired minimum

-1 -05 0 05 1-1

-05

0

05

1

Edge-preserving smoothnessSpace of Bounded Variations

which one is ldquobetterrdquo

Bounded Variation ndash ND caseBounded Variation ndash ND case

sup |)(| dxdivffD

)( x

N

1i i

i xdiv

1 || )( ) ( )(

101

L

NN C

bounded open subset function NR )( 1 Lf

Variation of over f

where

φ

Edge-preserving smoothnessSpace of Bounded Variations

integrable (absolute value) and with bounded variation

Functions are not required to have an integrable derivative hellip

What is the meaning of u in the regularization term

Intuitively norm of gradient |u| is

replaced with variation |Du|

Total variation

Theorem (informally) if u BV() then

Hausdorff measure

area gt 0area = 0

How can we measure zero-measure sets

Hausdorff measure

1) cover with balls of diameter

2) sum up diameters for optimal cover (do not waste balls)

3) refine 0

Hausdorff measureFormally

For A RN k-dimensional Hausdorff measure of A

up to normalization factor covers are countable

bull HN is just the Lebesgue measure

bull curve in image

its length = H1 in R2

Total variation

Theorem (more formally) if u BV() then

u+

u-

u(x)

xx0

u+ u- - approximate upper and lower limits

Su = x u+gtu-the jump set

Energy functional

data term

regularization for background image

regularization for background masks

Total variation example

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

= perimeter = 4

Divide each side into n parts

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation(= sum of perimeters)

Large total variation(= sum of perimeters)

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation Large total variation

Edge-preserving smoothnessSpace of Bounded Variations

BV informally functions with discontinuities on curves

Edges are preserved texture is not preserved

original sequencetemporal median energy minimization

in BV

Energy functional

Time-discretized problem

Find minimum of E subject to

Existence of solution

Under usual assumptions

12 R+ R+ strictly convex nondecreasing

with linear growth at infinity

minimum of E exists in BV(BC1hellipCT)

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Edge-preserving smoothnessRegularization term

ELE

Edge-preserving smoothnessRegularization term

ELE

n

Change the coordinate system

across the edgealong the edge

Compare

Edge-preserving smoothnessRegularization term

Weak edge (s 0)

Conditions on

Isotropic smoothings) is quadratic at zero

(s)

s

Edge-preserving smoothnessRegularization term

Strong edge (s )

Conditions on

bull no smoothing across the edge

bull more smoothing along the edge

Anisotropic smoothings) does not grow too fast at infinity

(s)

s

Edge-preserving smoothnessRegularization term

ConclusionUsing regularization term of the form

we can achieve both

isotropic smoothness in uniform regions

and anisotropic smoothness on edges

with one function

0 1 2 3 4 50

05

1

15

2

25

3

35

4

Edge-preserving smoothnessRegularization term

Example of an edge-preserving function

0 005 01 015 020

0005

001

0015

002

Edge-preserving smoothnessSpace of Bounded Variations

Even if we have an edge-preserving functional

if the space of solutions u contains only smooth functions we may not achieve the desired minimum

-1 -05 0 05 1-1

-05

0

05

1

Edge-preserving smoothnessSpace of Bounded Variations

which one is ldquobetterrdquo

Bounded Variation ndash ND caseBounded Variation ndash ND case

sup |)(| dxdivffD

)( x

N

1i i

i xdiv

1 || )( ) ( )(

101

L

NN C

bounded open subset function NR )( 1 Lf

Variation of over f

where

φ

Edge-preserving smoothnessSpace of Bounded Variations

integrable (absolute value) and with bounded variation

Functions are not required to have an integrable derivative hellip

What is the meaning of u in the regularization term

Intuitively norm of gradient |u| is

replaced with variation |Du|

Total variation

Theorem (informally) if u BV() then

Hausdorff measure

area gt 0area = 0

How can we measure zero-measure sets

Hausdorff measure

1) cover with balls of diameter

2) sum up diameters for optimal cover (do not waste balls)

3) refine 0

Hausdorff measureFormally

For A RN k-dimensional Hausdorff measure of A

up to normalization factor covers are countable

bull HN is just the Lebesgue measure

bull curve in image

its length = H1 in R2

Total variation

Theorem (more formally) if u BV() then

u+

u-

u(x)

xx0

u+ u- - approximate upper and lower limits

Su = x u+gtu-the jump set

Energy functional

data term

regularization for background image

regularization for background masks

Total variation example

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

= perimeter = 4

Divide each side into n parts

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation(= sum of perimeters)

Large total variation(= sum of perimeters)

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation Large total variation

Edge-preserving smoothnessSpace of Bounded Variations

BV informally functions with discontinuities on curves

Edges are preserved texture is not preserved

original sequencetemporal median energy minimization

in BV

Energy functional

Time-discretized problem

Find minimum of E subject to

Existence of solution

Under usual assumptions

12 R+ R+ strictly convex nondecreasing

with linear growth at infinity

minimum of E exists in BV(BC1hellipCT)

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Edge-preserving smoothnessRegularization term

ELE

n

Change the coordinate system

across the edgealong the edge

Compare

Edge-preserving smoothnessRegularization term

Weak edge (s 0)

Conditions on

Isotropic smoothings) is quadratic at zero

(s)

s

Edge-preserving smoothnessRegularization term

Strong edge (s )

Conditions on

bull no smoothing across the edge

bull more smoothing along the edge

Anisotropic smoothings) does not grow too fast at infinity

(s)

s

Edge-preserving smoothnessRegularization term

ConclusionUsing regularization term of the form

we can achieve both

isotropic smoothness in uniform regions

and anisotropic smoothness on edges

with one function

0 1 2 3 4 50

05

1

15

2

25

3

35

4

Edge-preserving smoothnessRegularization term

Example of an edge-preserving function

0 005 01 015 020

0005

001

0015

002

Edge-preserving smoothnessSpace of Bounded Variations

Even if we have an edge-preserving functional

if the space of solutions u contains only smooth functions we may not achieve the desired minimum

-1 -05 0 05 1-1

-05

0

05

1

Edge-preserving smoothnessSpace of Bounded Variations

which one is ldquobetterrdquo

Bounded Variation ndash ND caseBounded Variation ndash ND case

sup |)(| dxdivffD

)( x

N

1i i

i xdiv

1 || )( ) ( )(

101

L

NN C

bounded open subset function NR )( 1 Lf

Variation of over f

where

φ

Edge-preserving smoothnessSpace of Bounded Variations

integrable (absolute value) and with bounded variation

Functions are not required to have an integrable derivative hellip

What is the meaning of u in the regularization term

Intuitively norm of gradient |u| is

replaced with variation |Du|

Total variation

Theorem (informally) if u BV() then

Hausdorff measure

area gt 0area = 0

How can we measure zero-measure sets

Hausdorff measure

1) cover with balls of diameter

2) sum up diameters for optimal cover (do not waste balls)

3) refine 0

Hausdorff measureFormally

For A RN k-dimensional Hausdorff measure of A

up to normalization factor covers are countable

bull HN is just the Lebesgue measure

bull curve in image

its length = H1 in R2

Total variation

Theorem (more formally) if u BV() then

u+

u-

u(x)

xx0

u+ u- - approximate upper and lower limits

Su = x u+gtu-the jump set

Energy functional

data term

regularization for background image

regularization for background masks

Total variation example

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

= perimeter = 4

Divide each side into n parts

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation(= sum of perimeters)

Large total variation(= sum of perimeters)

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation Large total variation

Edge-preserving smoothnessSpace of Bounded Variations

BV informally functions with discontinuities on curves

Edges are preserved texture is not preserved

original sequencetemporal median energy minimization

in BV

Energy functional

Time-discretized problem

Find minimum of E subject to

Existence of solution

Under usual assumptions

12 R+ R+ strictly convex nondecreasing

with linear growth at infinity

minimum of E exists in BV(BC1hellipCT)

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Edge-preserving smoothnessRegularization term

Weak edge (s 0)

Conditions on

Isotropic smoothings) is quadratic at zero

(s)

s

Edge-preserving smoothnessRegularization term

Strong edge (s )

Conditions on

bull no smoothing across the edge

bull more smoothing along the edge

Anisotropic smoothings) does not grow too fast at infinity

(s)

s

Edge-preserving smoothnessRegularization term

ConclusionUsing regularization term of the form

we can achieve both

isotropic smoothness in uniform regions

and anisotropic smoothness on edges

with one function

0 1 2 3 4 50

05

1

15

2

25

3

35

4

Edge-preserving smoothnessRegularization term

Example of an edge-preserving function

0 005 01 015 020

0005

001

0015

002

Edge-preserving smoothnessSpace of Bounded Variations

Even if we have an edge-preserving functional

if the space of solutions u contains only smooth functions we may not achieve the desired minimum

-1 -05 0 05 1-1

-05

0

05

1

Edge-preserving smoothnessSpace of Bounded Variations

which one is ldquobetterrdquo

Bounded Variation ndash ND caseBounded Variation ndash ND case

sup |)(| dxdivffD

)( x

N

1i i

i xdiv

1 || )( ) ( )(

101

L

NN C

bounded open subset function NR )( 1 Lf

Variation of over f

where

φ

Edge-preserving smoothnessSpace of Bounded Variations

integrable (absolute value) and with bounded variation

Functions are not required to have an integrable derivative hellip

What is the meaning of u in the regularization term

Intuitively norm of gradient |u| is

replaced with variation |Du|

Total variation

Theorem (informally) if u BV() then

Hausdorff measure

area gt 0area = 0

How can we measure zero-measure sets

Hausdorff measure

1) cover with balls of diameter

2) sum up diameters for optimal cover (do not waste balls)

3) refine 0

Hausdorff measureFormally

For A RN k-dimensional Hausdorff measure of A

up to normalization factor covers are countable

bull HN is just the Lebesgue measure

bull curve in image

its length = H1 in R2

Total variation

Theorem (more formally) if u BV() then

u+

u-

u(x)

xx0

u+ u- - approximate upper and lower limits

Su = x u+gtu-the jump set

Energy functional

data term

regularization for background image

regularization for background masks

Total variation example

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

= perimeter = 4

Divide each side into n parts

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation(= sum of perimeters)

Large total variation(= sum of perimeters)

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation Large total variation

Edge-preserving smoothnessSpace of Bounded Variations

BV informally functions with discontinuities on curves

Edges are preserved texture is not preserved

original sequencetemporal median energy minimization

in BV

Energy functional

Time-discretized problem

Find minimum of E subject to

Existence of solution

Under usual assumptions

12 R+ R+ strictly convex nondecreasing

with linear growth at infinity

minimum of E exists in BV(BC1hellipCT)

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Edge-preserving smoothnessRegularization term

Strong edge (s )

Conditions on

bull no smoothing across the edge

bull more smoothing along the edge

Anisotropic smoothings) does not grow too fast at infinity

(s)

s

Edge-preserving smoothnessRegularization term

ConclusionUsing regularization term of the form

we can achieve both

isotropic smoothness in uniform regions

and anisotropic smoothness on edges

with one function

0 1 2 3 4 50

05

1

15

2

25

3

35

4

Edge-preserving smoothnessRegularization term

Example of an edge-preserving function

0 005 01 015 020

0005

001

0015

002

Edge-preserving smoothnessSpace of Bounded Variations

Even if we have an edge-preserving functional

if the space of solutions u contains only smooth functions we may not achieve the desired minimum

-1 -05 0 05 1-1

-05

0

05

1

Edge-preserving smoothnessSpace of Bounded Variations

which one is ldquobetterrdquo

Bounded Variation ndash ND caseBounded Variation ndash ND case

sup |)(| dxdivffD

)( x

N

1i i

i xdiv

1 || )( ) ( )(

101

L

NN C

bounded open subset function NR )( 1 Lf

Variation of over f

where

φ

Edge-preserving smoothnessSpace of Bounded Variations

integrable (absolute value) and with bounded variation

Functions are not required to have an integrable derivative hellip

What is the meaning of u in the regularization term

Intuitively norm of gradient |u| is

replaced with variation |Du|

Total variation

Theorem (informally) if u BV() then

Hausdorff measure

area gt 0area = 0

How can we measure zero-measure sets

Hausdorff measure

1) cover with balls of diameter

2) sum up diameters for optimal cover (do not waste balls)

3) refine 0

Hausdorff measureFormally

For A RN k-dimensional Hausdorff measure of A

up to normalization factor covers are countable

bull HN is just the Lebesgue measure

bull curve in image

its length = H1 in R2

Total variation

Theorem (more formally) if u BV() then

u+

u-

u(x)

xx0

u+ u- - approximate upper and lower limits

Su = x u+gtu-the jump set

Energy functional

data term

regularization for background image

regularization for background masks

Total variation example

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

= perimeter = 4

Divide each side into n parts

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation(= sum of perimeters)

Large total variation(= sum of perimeters)

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation Large total variation

Edge-preserving smoothnessSpace of Bounded Variations

BV informally functions with discontinuities on curves

Edges are preserved texture is not preserved

original sequencetemporal median energy minimization

in BV

Energy functional

Time-discretized problem

Find minimum of E subject to

Existence of solution

Under usual assumptions

12 R+ R+ strictly convex nondecreasing

with linear growth at infinity

minimum of E exists in BV(BC1hellipCT)

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Edge-preserving smoothnessRegularization term

ConclusionUsing regularization term of the form

we can achieve both

isotropic smoothness in uniform regions

and anisotropic smoothness on edges

with one function

0 1 2 3 4 50

05

1

15

2

25

3

35

4

Edge-preserving smoothnessRegularization term

Example of an edge-preserving function

0 005 01 015 020

0005

001

0015

002

Edge-preserving smoothnessSpace of Bounded Variations

Even if we have an edge-preserving functional

if the space of solutions u contains only smooth functions we may not achieve the desired minimum

-1 -05 0 05 1-1

-05

0

05

1

Edge-preserving smoothnessSpace of Bounded Variations

which one is ldquobetterrdquo

Bounded Variation ndash ND caseBounded Variation ndash ND case

sup |)(| dxdivffD

)( x

N

1i i

i xdiv

1 || )( ) ( )(

101

L

NN C

bounded open subset function NR )( 1 Lf

Variation of over f

where

φ

Edge-preserving smoothnessSpace of Bounded Variations

integrable (absolute value) and with bounded variation

Functions are not required to have an integrable derivative hellip

What is the meaning of u in the regularization term

Intuitively norm of gradient |u| is

replaced with variation |Du|

Total variation

Theorem (informally) if u BV() then

Hausdorff measure

area gt 0area = 0

How can we measure zero-measure sets

Hausdorff measure

1) cover with balls of diameter

2) sum up diameters for optimal cover (do not waste balls)

3) refine 0

Hausdorff measureFormally

For A RN k-dimensional Hausdorff measure of A

up to normalization factor covers are countable

bull HN is just the Lebesgue measure

bull curve in image

its length = H1 in R2

Total variation

Theorem (more formally) if u BV() then

u+

u-

u(x)

xx0

u+ u- - approximate upper and lower limits

Su = x u+gtu-the jump set

Energy functional

data term

regularization for background image

regularization for background masks

Total variation example

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

= perimeter = 4

Divide each side into n parts

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation(= sum of perimeters)

Large total variation(= sum of perimeters)

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation Large total variation

Edge-preserving smoothnessSpace of Bounded Variations

BV informally functions with discontinuities on curves

Edges are preserved texture is not preserved

original sequencetemporal median energy minimization

in BV

Energy functional

Time-discretized problem

Find minimum of E subject to

Existence of solution

Under usual assumptions

12 R+ R+ strictly convex nondecreasing

with linear growth at infinity

minimum of E exists in BV(BC1hellipCT)

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

0 1 2 3 4 50

05

1

15

2

25

3

35

4

Edge-preserving smoothnessRegularization term

Example of an edge-preserving function

0 005 01 015 020

0005

001

0015

002

Edge-preserving smoothnessSpace of Bounded Variations

Even if we have an edge-preserving functional

if the space of solutions u contains only smooth functions we may not achieve the desired minimum

-1 -05 0 05 1-1

-05

0

05

1

Edge-preserving smoothnessSpace of Bounded Variations

which one is ldquobetterrdquo

Bounded Variation ndash ND caseBounded Variation ndash ND case

sup |)(| dxdivffD

)( x

N

1i i

i xdiv

1 || )( ) ( )(

101

L

NN C

bounded open subset function NR )( 1 Lf

Variation of over f

where

φ

Edge-preserving smoothnessSpace of Bounded Variations

integrable (absolute value) and with bounded variation

Functions are not required to have an integrable derivative hellip

What is the meaning of u in the regularization term

Intuitively norm of gradient |u| is

replaced with variation |Du|

Total variation

Theorem (informally) if u BV() then

Hausdorff measure

area gt 0area = 0

How can we measure zero-measure sets

Hausdorff measure

1) cover with balls of diameter

2) sum up diameters for optimal cover (do not waste balls)

3) refine 0

Hausdorff measureFormally

For A RN k-dimensional Hausdorff measure of A

up to normalization factor covers are countable

bull HN is just the Lebesgue measure

bull curve in image

its length = H1 in R2

Total variation

Theorem (more formally) if u BV() then

u+

u-

u(x)

xx0

u+ u- - approximate upper and lower limits

Su = x u+gtu-the jump set

Energy functional

data term

regularization for background image

regularization for background masks

Total variation example

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

= perimeter = 4

Divide each side into n parts

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation(= sum of perimeters)

Large total variation(= sum of perimeters)

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation Large total variation

Edge-preserving smoothnessSpace of Bounded Variations

BV informally functions with discontinuities on curves

Edges are preserved texture is not preserved

original sequencetemporal median energy minimization

in BV

Energy functional

Time-discretized problem

Find minimum of E subject to

Existence of solution

Under usual assumptions

12 R+ R+ strictly convex nondecreasing

with linear growth at infinity

minimum of E exists in BV(BC1hellipCT)

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Edge-preserving smoothnessSpace of Bounded Variations

Even if we have an edge-preserving functional

if the space of solutions u contains only smooth functions we may not achieve the desired minimum

-1 -05 0 05 1-1

-05

0

05

1

Edge-preserving smoothnessSpace of Bounded Variations

which one is ldquobetterrdquo

Bounded Variation ndash ND caseBounded Variation ndash ND case

sup |)(| dxdivffD

)( x

N

1i i

i xdiv

1 || )( ) ( )(

101

L

NN C

bounded open subset function NR )( 1 Lf

Variation of over f

where

φ

Edge-preserving smoothnessSpace of Bounded Variations

integrable (absolute value) and with bounded variation

Functions are not required to have an integrable derivative hellip

What is the meaning of u in the regularization term

Intuitively norm of gradient |u| is

replaced with variation |Du|

Total variation

Theorem (informally) if u BV() then

Hausdorff measure

area gt 0area = 0

How can we measure zero-measure sets

Hausdorff measure

1) cover with balls of diameter

2) sum up diameters for optimal cover (do not waste balls)

3) refine 0

Hausdorff measureFormally

For A RN k-dimensional Hausdorff measure of A

up to normalization factor covers are countable

bull HN is just the Lebesgue measure

bull curve in image

its length = H1 in R2

Total variation

Theorem (more formally) if u BV() then

u+

u-

u(x)

xx0

u+ u- - approximate upper and lower limits

Su = x u+gtu-the jump set

Energy functional

data term

regularization for background image

regularization for background masks

Total variation example

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

= perimeter = 4

Divide each side into n parts

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation(= sum of perimeters)

Large total variation(= sum of perimeters)

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation Large total variation

Edge-preserving smoothnessSpace of Bounded Variations

BV informally functions with discontinuities on curves

Edges are preserved texture is not preserved

original sequencetemporal median energy minimization

in BV

Energy functional

Time-discretized problem

Find minimum of E subject to

Existence of solution

Under usual assumptions

12 R+ R+ strictly convex nondecreasing

with linear growth at infinity

minimum of E exists in BV(BC1hellipCT)

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Edge-preserving smoothnessSpace of Bounded Variations

which one is ldquobetterrdquo

Bounded Variation ndash ND caseBounded Variation ndash ND case

sup |)(| dxdivffD

)( x

N

1i i

i xdiv

1 || )( ) ( )(

101

L

NN C

bounded open subset function NR )( 1 Lf

Variation of over f

where

φ

Edge-preserving smoothnessSpace of Bounded Variations

integrable (absolute value) and with bounded variation

Functions are not required to have an integrable derivative hellip

What is the meaning of u in the regularization term

Intuitively norm of gradient |u| is

replaced with variation |Du|

Total variation

Theorem (informally) if u BV() then

Hausdorff measure

area gt 0area = 0

How can we measure zero-measure sets

Hausdorff measure

1) cover with balls of diameter

2) sum up diameters for optimal cover (do not waste balls)

3) refine 0

Hausdorff measureFormally

For A RN k-dimensional Hausdorff measure of A

up to normalization factor covers are countable

bull HN is just the Lebesgue measure

bull curve in image

its length = H1 in R2

Total variation

Theorem (more formally) if u BV() then

u+

u-

u(x)

xx0

u+ u- - approximate upper and lower limits

Su = x u+gtu-the jump set

Energy functional

data term

regularization for background image

regularization for background masks

Total variation example

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

= perimeter = 4

Divide each side into n parts

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation(= sum of perimeters)

Large total variation(= sum of perimeters)

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation Large total variation

Edge-preserving smoothnessSpace of Bounded Variations

BV informally functions with discontinuities on curves

Edges are preserved texture is not preserved

original sequencetemporal median energy minimization

in BV

Energy functional

Time-discretized problem

Find minimum of E subject to

Existence of solution

Under usual assumptions

12 R+ R+ strictly convex nondecreasing

with linear growth at infinity

minimum of E exists in BV(BC1hellipCT)

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Bounded Variation ndash ND caseBounded Variation ndash ND case

sup |)(| dxdivffD

)( x

N

1i i

i xdiv

1 || )( ) ( )(

101

L

NN C

bounded open subset function NR )( 1 Lf

Variation of over f

where

φ

Edge-preserving smoothnessSpace of Bounded Variations

integrable (absolute value) and with bounded variation

Functions are not required to have an integrable derivative hellip

What is the meaning of u in the regularization term

Intuitively norm of gradient |u| is

replaced with variation |Du|

Total variation

Theorem (informally) if u BV() then

Hausdorff measure

area gt 0area = 0

How can we measure zero-measure sets

Hausdorff measure

1) cover with balls of diameter

2) sum up diameters for optimal cover (do not waste balls)

3) refine 0

Hausdorff measureFormally

For A RN k-dimensional Hausdorff measure of A

up to normalization factor covers are countable

bull HN is just the Lebesgue measure

bull curve in image

its length = H1 in R2

Total variation

Theorem (more formally) if u BV() then

u+

u-

u(x)

xx0

u+ u- - approximate upper and lower limits

Su = x u+gtu-the jump set

Energy functional

data term

regularization for background image

regularization for background masks

Total variation example

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

= perimeter = 4

Divide each side into n parts

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation(= sum of perimeters)

Large total variation(= sum of perimeters)

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation Large total variation

Edge-preserving smoothnessSpace of Bounded Variations

BV informally functions with discontinuities on curves

Edges are preserved texture is not preserved

original sequencetemporal median energy minimization

in BV

Energy functional

Time-discretized problem

Find minimum of E subject to

Existence of solution

Under usual assumptions

12 R+ R+ strictly convex nondecreasing

with linear growth at infinity

minimum of E exists in BV(BC1hellipCT)

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Edge-preserving smoothnessSpace of Bounded Variations

integrable (absolute value) and with bounded variation

Functions are not required to have an integrable derivative hellip

What is the meaning of u in the regularization term

Intuitively norm of gradient |u| is

replaced with variation |Du|

Total variation

Theorem (informally) if u BV() then

Hausdorff measure

area gt 0area = 0

How can we measure zero-measure sets

Hausdorff measure

1) cover with balls of diameter

2) sum up diameters for optimal cover (do not waste balls)

3) refine 0

Hausdorff measureFormally

For A RN k-dimensional Hausdorff measure of A

up to normalization factor covers are countable

bull HN is just the Lebesgue measure

bull curve in image

its length = H1 in R2

Total variation

Theorem (more formally) if u BV() then

u+

u-

u(x)

xx0

u+ u- - approximate upper and lower limits

Su = x u+gtu-the jump set

Energy functional

data term

regularization for background image

regularization for background masks

Total variation example

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

= perimeter = 4

Divide each side into n parts

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation(= sum of perimeters)

Large total variation(= sum of perimeters)

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation Large total variation

Edge-preserving smoothnessSpace of Bounded Variations

BV informally functions with discontinuities on curves

Edges are preserved texture is not preserved

original sequencetemporal median energy minimization

in BV

Energy functional

Time-discretized problem

Find minimum of E subject to

Existence of solution

Under usual assumptions

12 R+ R+ strictly convex nondecreasing

with linear growth at infinity

minimum of E exists in BV(BC1hellipCT)

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Total variation

Theorem (informally) if u BV() then

Hausdorff measure

area gt 0area = 0

How can we measure zero-measure sets

Hausdorff measure

1) cover with balls of diameter

2) sum up diameters for optimal cover (do not waste balls)

3) refine 0

Hausdorff measureFormally

For A RN k-dimensional Hausdorff measure of A

up to normalization factor covers are countable

bull HN is just the Lebesgue measure

bull curve in image

its length = H1 in R2

Total variation

Theorem (more formally) if u BV() then

u+

u-

u(x)

xx0

u+ u- - approximate upper and lower limits

Su = x u+gtu-the jump set

Energy functional

data term

regularization for background image

regularization for background masks

Total variation example

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

= perimeter = 4

Divide each side into n parts

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation(= sum of perimeters)

Large total variation(= sum of perimeters)

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation Large total variation

Edge-preserving smoothnessSpace of Bounded Variations

BV informally functions with discontinuities on curves

Edges are preserved texture is not preserved

original sequencetemporal median energy minimization

in BV

Energy functional

Time-discretized problem

Find minimum of E subject to

Existence of solution

Under usual assumptions

12 R+ R+ strictly convex nondecreasing

with linear growth at infinity

minimum of E exists in BV(BC1hellipCT)

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Hausdorff measure

area gt 0area = 0

How can we measure zero-measure sets

Hausdorff measure

1) cover with balls of diameter

2) sum up diameters for optimal cover (do not waste balls)

3) refine 0

Hausdorff measureFormally

For A RN k-dimensional Hausdorff measure of A

up to normalization factor covers are countable

bull HN is just the Lebesgue measure

bull curve in image

its length = H1 in R2

Total variation

Theorem (more formally) if u BV() then

u+

u-

u(x)

xx0

u+ u- - approximate upper and lower limits

Su = x u+gtu-the jump set

Energy functional

data term

regularization for background image

regularization for background masks

Total variation example

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

= perimeter = 4

Divide each side into n parts

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation(= sum of perimeters)

Large total variation(= sum of perimeters)

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation Large total variation

Edge-preserving smoothnessSpace of Bounded Variations

BV informally functions with discontinuities on curves

Edges are preserved texture is not preserved

original sequencetemporal median energy minimization

in BV

Energy functional

Time-discretized problem

Find minimum of E subject to

Existence of solution

Under usual assumptions

12 R+ R+ strictly convex nondecreasing

with linear growth at infinity

minimum of E exists in BV(BC1hellipCT)

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Hausdorff measure

1) cover with balls of diameter

2) sum up diameters for optimal cover (do not waste balls)

3) refine 0

Hausdorff measureFormally

For A RN k-dimensional Hausdorff measure of A

up to normalization factor covers are countable

bull HN is just the Lebesgue measure

bull curve in image

its length = H1 in R2

Total variation

Theorem (more formally) if u BV() then

u+

u-

u(x)

xx0

u+ u- - approximate upper and lower limits

Su = x u+gtu-the jump set

Energy functional

data term

regularization for background image

regularization for background masks

Total variation example

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

= perimeter = 4

Divide each side into n parts

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation(= sum of perimeters)

Large total variation(= sum of perimeters)

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation Large total variation

Edge-preserving smoothnessSpace of Bounded Variations

BV informally functions with discontinuities on curves

Edges are preserved texture is not preserved

original sequencetemporal median energy minimization

in BV

Energy functional

Time-discretized problem

Find minimum of E subject to

Existence of solution

Under usual assumptions

12 R+ R+ strictly convex nondecreasing

with linear growth at infinity

minimum of E exists in BV(BC1hellipCT)

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Hausdorff measureFormally

For A RN k-dimensional Hausdorff measure of A

up to normalization factor covers are countable

bull HN is just the Lebesgue measure

bull curve in image

its length = H1 in R2

Total variation

Theorem (more formally) if u BV() then

u+

u-

u(x)

xx0

u+ u- - approximate upper and lower limits

Su = x u+gtu-the jump set

Energy functional

data term

regularization for background image

regularization for background masks

Total variation example

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

= perimeter = 4

Divide each side into n parts

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation(= sum of perimeters)

Large total variation(= sum of perimeters)

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation Large total variation

Edge-preserving smoothnessSpace of Bounded Variations

BV informally functions with discontinuities on curves

Edges are preserved texture is not preserved

original sequencetemporal median energy minimization

in BV

Energy functional

Time-discretized problem

Find minimum of E subject to

Existence of solution

Under usual assumptions

12 R+ R+ strictly convex nondecreasing

with linear growth at infinity

minimum of E exists in BV(BC1hellipCT)

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Total variation

Theorem (more formally) if u BV() then

u+

u-

u(x)

xx0

u+ u- - approximate upper and lower limits

Su = x u+gtu-the jump set

Energy functional

data term

regularization for background image

regularization for background masks

Total variation example

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

= perimeter = 4

Divide each side into n parts

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation(= sum of perimeters)

Large total variation(= sum of perimeters)

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation Large total variation

Edge-preserving smoothnessSpace of Bounded Variations

BV informally functions with discontinuities on curves

Edges are preserved texture is not preserved

original sequencetemporal median energy minimization

in BV

Energy functional

Time-discretized problem

Find minimum of E subject to

Existence of solution

Under usual assumptions

12 R+ R+ strictly convex nondecreasing

with linear growth at infinity

minimum of E exists in BV(BC1hellipCT)

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Energy functional

data term

regularization for background image

regularization for background masks

Total variation example

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

= perimeter = 4

Divide each side into n parts

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation(= sum of perimeters)

Large total variation(= sum of perimeters)

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation Large total variation

Edge-preserving smoothnessSpace of Bounded Variations

BV informally functions with discontinuities on curves

Edges are preserved texture is not preserved

original sequencetemporal median energy minimization

in BV

Energy functional

Time-discretized problem

Find minimum of E subject to

Existence of solution

Under usual assumptions

12 R+ R+ strictly convex nondecreasing

with linear growth at infinity

minimum of E exists in BV(BC1hellipCT)

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Total variation example

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

-1 -05 0 05 1 15 2

-1

0

1

2

0

05

1

= perimeter = 4

Divide each side into n parts

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation(= sum of perimeters)

Large total variation(= sum of perimeters)

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation Large total variation

Edge-preserving smoothnessSpace of Bounded Variations

BV informally functions with discontinuities on curves

Edges are preserved texture is not preserved

original sequencetemporal median energy minimization

in BV

Energy functional

Time-discretized problem

Find minimum of E subject to

Existence of solution

Under usual assumptions

12 R+ R+ strictly convex nondecreasing

with linear growth at infinity

minimum of E exists in BV(BC1hellipCT)

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation(= sum of perimeters)

Large total variation(= sum of perimeters)

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation Large total variation

Edge-preserving smoothnessSpace of Bounded Variations

BV informally functions with discontinuities on curves

Edges are preserved texture is not preserved

original sequencetemporal median energy minimization

in BV

Energy functional

Time-discretized problem

Find minimum of E subject to

Existence of solution

Under usual assumptions

12 R+ R+ strictly convex nondecreasing

with linear growth at infinity

minimum of E exists in BV(BC1hellipCT)

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Edge-preserving smoothnessSpace of Bounded Variations

Small total variation Large total variation

Edge-preserving smoothnessSpace of Bounded Variations

BV informally functions with discontinuities on curves

Edges are preserved texture is not preserved

original sequencetemporal median energy minimization

in BV

Energy functional

Time-discretized problem

Find minimum of E subject to

Existence of solution

Under usual assumptions

12 R+ R+ strictly convex nondecreasing

with linear growth at infinity

minimum of E exists in BV(BC1hellipCT)

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Edge-preserving smoothnessSpace of Bounded Variations

BV informally functions with discontinuities on curves

Edges are preserved texture is not preserved

original sequencetemporal median energy minimization

in BV

Energy functional

Time-discretized problem

Find minimum of E subject to

Existence of solution

Under usual assumptions

12 R+ R+ strictly convex nondecreasing

with linear growth at infinity

minimum of E exists in BV(BC1hellipCT)

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Energy functional

Time-discretized problem

Find minimum of E subject to

Existence of solution

Under usual assumptions

12 R+ R+ strictly convex nondecreasing

with linear growth at infinity

minimum of E exists in BV(BC1hellipCT)

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Existence of solution

Under usual assumptions

12 R+ R+ strictly convex nondecreasing

with linear growth at infinity

minimum of E exists in BV(BC1hellipCT)

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

(non-)Uniqueness

is not convex wrt (BC1hellipCT) Solution may not be unique

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Uniqueness

But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique

Interpretation if we are allowed to say that everything is foreground background image is not well-defined

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Finding solution

BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV

Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Approximating functionals

Recall 12(s) = s2 gives smooth solutions

Idea replace i with iwhich are quadratic

at s 0 and s

Approximating functionals

Approximating functionals

Approximating problems

has unique solution in the space

ndash convergence of functionals if E -converge to Ethen approximate solutions of min E

converge to min E

More results Sweden

More results Highway

More results INRIA_1

More results INRIA_1Sequence restoration

More results INRIA_2Sequence restoration

• Background vs foreground segmentation of video sequences
• The Problem
• Simple approach (1)
• Simple approach (2)
• Simple approach noise can spoil everything
• Variational approach
• Notations
• Energy functional data term
• Slide 9
• Slide 10
• Slide 11
• Energy functional smoothness
• Energy functional
• Edge-preserving smoothness Regularization term
• Slide 15
• Slide 16
• Slide 17
• Slide 18
• Slide 19
• Slide 20
• Slide 21
• Edge-preserving smoothness Space of Bounded Variations
• Slide 23
• Bounded Variation ndash ND case
• Slide 25
• Total variation
• Hausdorff measure
• Slide 28
• Slide 29
• Slide 30
• Slide 31
• Total variation example
• Slide 33
• Slide 34
• Slide 35
• Slide 36
• Existence of solution
• (non-)Uniqueness
• Uniqueness
• Finding solution
• Approximating functionals
• Slide 42
• Approximating problems
• More results Sweden
• More results Highway
• More results INRIA_1
• More results INRIA_1 Sequence restoration
• More results INRIA_2 Sequence restoration
• Slide 49

More results Sweden

More results Highway

More results INRIA_1

More results INRIA_1Sequence restoration

More results INRIA_2Sequence restoration

• Background vs foreground segmentation of video sequences
• The Problem
• Simple approach (1)
• Simple approach (2)
• Simple approach noise can spoil everything
• Variational approach
• Notations
• Energy functional data term
• Slide 9
• Slide 10
• Slide 11
• Energy functional smoothness
• Energy functional
• Edge-preserving smoothness Regularization term
• Slide 15
• Slide 16
• Slide 17
• Slide 18
• Slide 19
• Slide 20
• Slide 21
• Edge-preserving smoothness Space of Bounded Variations
• Slide 23
• Bounded Variation ndash ND case
• Slide 25
• Total variation
• Hausdorff measure
• Slide 28
• Slide 29
• Slide 30
• Slide 31
• Total variation example
• Slide 33
• Slide 34
• Slide 35
• Slide 36
• Existence of solution
• (non-)Uniqueness
• Uniqueness
• Finding solution
• Approximating functionals
• Slide 42
• Approximating problems
• More results Sweden
• More results Highway
• More results INRIA_1
• More results INRIA_1 Sequence restoration
• More results INRIA_2 Sequence restoration
• Slide 49

More results Highway

More results INRIA_1

More results INRIA_1Sequence restoration

More results INRIA_2Sequence restoration

• Background vs foreground segmentation of video sequences
• The Problem
• Simple approach (1)
• Simple approach (2)
• Simple approach noise can spoil everything
• Variational approach
• Notations
• Energy functional data term
• Slide 9
• Slide 10
• Slide 11
• Energy functional smoothness
• Energy functional
• Edge-preserving smoothness Regularization term
• Slide 15
• Slide 16
• Slide 17
• Slide 18
• Slide 19
• Slide 20
• Slide 21
• Edge-preserving smoothness Space of Bounded Variations
• Slide 23
• Bounded Variation ndash ND case
• Slide 25
• Total variation
• Hausdorff measure
• Slide 28
• Slide 29
• Slide 30
• Slide 31
• Total variation example
• Slide 33
• Slide 34
• Slide 35
• Slide 36
• Existence of solution
• (non-)Uniqueness
• Uniqueness
• Finding solution
• Approximating functionals
• Slide 42
• Approximating problems
• More results Sweden
• More results Highway
• More results INRIA_1
• More results INRIA_1 Sequence restoration
• More results INRIA_2 Sequence restoration
• Slide 49

More results INRIA_1

More results INRIA_1Sequence restoration

More results INRIA_2Sequence restoration

• Background vs foreground segmentation of video sequences
• The Problem
• Simple approach (1)
• Simple approach (2)
• Simple approach noise can spoil everything
• Variational approach
• Notations
• Energy functional data term
• Slide 9
• Slide 10
• Slide 11
• Energy functional smoothness
• Energy functional
• Edge-preserving smoothness Regularization term
• Slide 15
• Slide 16
• Slide 17
• Slide 18
• Slide 19
• Slide 20
• Slide 21
• Edge-preserving smoothness Space of Bounded Variations
• Slide 23
• Bounded Variation ndash ND case
• Slide 25
• Total variation
• Hausdorff measure
• Slide 28
• Slide 29
• Slide 30
• Slide 31
• Total variation example
• Slide 33
• Slide 34
• Slide 35
• Slide 36
• Existence of solution
• (non-)Uniqueness
• Uniqueness
• Finding solution
• Approximating functionals
• Slide 42
• Approximating problems
• More results Sweden
• More results Highway
• More results INRIA_1
• More results INRIA_1 Sequence restoration
• More results INRIA_2 Sequence restoration
• Slide 49

More results INRIA_1Sequence restoration

More results INRIA_2Sequence restoration

• Background vs foreground segmentation of video sequences
• The Problem
• Simple approach (1)
• Simple approach (2)
• Simple approach noise can spoil everything
• Variational approach
• Notations
• Energy functional data term
• Slide 9
• Slide 10
• Slide 11
• Energy functional smoothness
• Energy functional
• Edge-preserving smoothness Regularization term
• Slide 15
• Slide 16
• Slide 17
• Slide 18
• Slide 19
• Slide 20
• Slide 21
• Edge-preserving smoothness Space of Bounded Variations
• Slide 23
• Bounded Variation ndash ND case
• Slide 25
• Total variation
• Hausdorff measure
• Slide 28
• Slide 29
• Slide 30
• Slide 31
• Total variation example
• Slide 33
• Slide 34
• Slide 35
• Slide 36
• Existence of solution
• (non-)Uniqueness
• Uniqueness
• Finding solution
• Approximating functionals
• Slide 42
• Approximating problems
• More results Sweden
• More results Highway
• More results INRIA_1
• More results INRIA_1 Sequence restoration
• More results INRIA_2 Sequence restoration
• Slide 49

More results INRIA_2Sequence restoration

• Background vs foreground segmentation of video sequences
• The Problem
• Simple approach (1)
• Simple approach (2)
• Simple approach noise can spoil everything
• Variational approach
• Notations
• Energy functional data term
• Slide 9
• Slide 10
• Slide 11
• Energy functional smoothness
• Energy functional
• Edge-preserving smoothness Regularization term
• Slide 15
• Slide 16
• Slide 17
• Slide 18
• Slide 19
• Slide 20
• Slide 21
• Edge-preserving smoothness Space of Bounded Variations
• Slide 23
• Bounded Variation ndash ND case
• Slide 25
• Total variation
• Hausdorff measure
• Slide 28
• Slide 29
• Slide 30
• Slide 31
• Total variation example
• Slide 33
• Slide 34
• Slide 35
• Slide 36
• Existence of solution
• (non-)Uniqueness
• Uniqueness
• Finding solution
• Approximating functionals
• Slide 42
• Approximating problems
• More results Sweden
• More results Highway
• More results INRIA_1
• More results INRIA_1 Sequence restoration
• More results INRIA_2 Sequence restoration
• Slide 49

• Background vs foreground segmentation of video sequences
• The Problem
• Simple approach (1)
• Simple approach (2)
• Simple approach noise can spoil everything
• Variational approach
• Notations
• Energy functional data term
• Slide 9
• Slide 10
• Slide 11
• Energy functional smoothness
• Energy functional
• Edge-preserving smoothness Regularization term
• Slide 15
• Slide 16
• Slide 17
• Slide 18
• Slide 19
• Slide 20
• Slide 21
• Edge-preserving smoothness Space of Bounded Variations
• Slide 23
• Bounded Variation ndash ND case
• Slide 25
• Total variation
• Hausdorff measure
• Slide 28
• Slide 29
• Slide 30
• Slide 31
• Total variation example
• Slide 33
• Slide 34
• Slide 35
• Slide 36
• Existence of solution
• (non-)Uniqueness
• Uniqueness
• Finding solution
• Approximating functionals
• Slide 42
• Approximating problems
• More results Sweden
• More results Highway
• More results INRIA_1
• More results INRIA_1 Sequence restoration
• More results INRIA_2 Sequence restoration
• Slide 49