Pyramids

Gaussianpre-filtering• Solution:filter

theimage,thensubsample

blur

F0 H*

subsample blur subsample …F1

F1 H*

F2F0

blur

F0 H*

subsample blur subsample …F1

F1 H*

F2F0

{Gaussianpyramid

Gaussianpyramids[BurtandAdelson,1983]

• Incomputergraphics,amip map[Williams,1983]

GaussianPyramidshaveallsortsofapplicationsincomputervision

Source:S.Seitz

Gaussianpyramids- Searchingoverscales

Gaussianpyramids- Searchingoverscales

2)*( 23 ¯= gaussianGG

1G

The Gaussian Pyramid

High resolution

Low resolution

Image=0G

2)*( 01 ¯= gaussianGG

2)*( 12 ¯= gaussianGG

2)*( 34 ¯= gaussianGG

blur

blur

blur

blur

Gaussianpyramidandstack

Source: Forsyth

MemoryUsage• Whatisthesizeofthepyramid?

9

Laplacianpyramid

Re-duce

=--

-

=

=

=-

Laplacianpyramid

L3 =G3 - expand(G4)=L2 =G2 - expand(G3)=

L1 =G1 - expand(G2)=

L0 =G0 - expand(G1)=

L4 =G4 =

ReconstructingtheimagefromaLaplacianpyramid

=++

+

=

=

=+

Laplacian pyramid

Source: Forsyth

Edgedetection

Whyedges?

• Resiliencetolightingandcolor• usefulforrecognition,matchingpatchesacrossimages

Whyedges?

• Humansaresensitivetoedges• Converta2Dimageintoasetofcurves

– Extractssalientfeaturesofthescene, morecompact

Whyedges?

• Cuetoshapeandgeometry• usefulforrecognition,understanding3Dstructure

Credit:Jitendra Malik

Credit:Attneave

Whyedges?

• Groupingpixelsintoobjects(“perceptualorganization”)

Thislecture

• Edgedetectioningeneral• Edgedetectionforgrouping

Edges

• Edgesarecurvesintheimage,acrosswhichthebrightnesschanges“alot”

• Corners/Junctions

Aside

Closeupofedges

Source:D.Hoiem

Closeup ofedges

Source:D.Hoiem

Closeup ofedges

Source:D.Hoiem

Closeup ofedges

Source:D.Hoiem

Characterizingedges•Anedgeisaplaceofrapidchangeintheimageintensityfunction

imageintensityfunction

(alonghorizontalscanline) firstderivative

edgescorrespondtoextremaofderivativeSource:L.Lazebnik

Intensityprofile

Source:D.Hoiem

Derivativesandconvolution

• Differentiationislinear

• Differentiationisshift-invariant• Derivativeofshiftedsignalisshiftedderivative

• Hence,differentiationcanberepresentedasconvolution!

@(af(x) + bg(x))

@x

= a

@f(x)

@x

+ b

@g(x)

@x

• Howcanwedifferentiateadigital imageF[x,y]?– Option1:reconstructacontinuousimage,f, thencomputethederivative

– Option2:takediscretederivative(finitedifference)

1 -1

Howwouldyouimplementthisasalinearfilter?

Imagederivatives

-1

1: :

Source:S.Seitz

Thegradientpointsinthedirectionofmostrapidincreaseinintensity

Imagegradient• Thegradient ofanimage:

Theedgestrength isgivenbythegradientmagnitude:

Thegradientdirectionisgivenby:

• howdoesthisrelatetothedirectionoftheedge?Source:SteveSeitz

Imagegradient

Source:L.Lazebnik

WithalittleGaussiannoise

Gradient

Source:D.Hoiem

Effectsofnoise

Whereistheedge?Source:S.Seitz

Noisyinputimage

Effectsofnoise

• Noiseishighfrequency• Differentiationaccentuatesnoise

d sin!x

dx

= ! cos!x

Solution:smoothfirst

f

h

f * h

Source:S.SeitzTofindedges,lookforpeaksin

•Differentiationisaconvolution• Convolutionisassociative:• Thissavesusoneoperation:

Associativepropertyofconvolution

f

Source:S.Seitz

2Dedgedetectionfilters

GaussianderivativeofGaussian(x)

DerivativeofGaussianfilter

x-direction y-direction

TwoDimensionalGaussian

OrientedGaussianFirstandSecondDerivatives

Grouping

Whatisgrouping?

Whygrouping?

• Pixelspropertyofsensor,notworld• Reasoningatobjectlevel(might)makethingseasy:

• objectsatconsistentdepth• objectscanberecognized• objectsmoveasone

"I stand at the window and see a house, trees, sky. Theoretically I might say there were 327 brightnesses and nuances of colour. Do I have "327"? No. I have sky, house, and trees."Max Wertheimer

RegionsBoundaries

Isgroupingwell-defined?

• Dependsonpurpose• Objectparts• Backgroundsegmentation

A

B C

Isgroupingwell-defined?

A

B C

Image

BG L-bird R-bird

grass bush

headeye

beakfar body

headeye

beak body

Perceptualorganizationformsatree:

BA

C

D.Martin,C.Fowlkes,D.Tal,J.Malik."ADatabaseofHumanSegmentedNaturalImagesanditsApplicationtoEvaluatingSegmentationAlgorithmsandMeasuringEcologicalStatistics",ICCV,2001

Howdowegroupthings?

• Gestalt principles• Principleofproximity

https://courses.lumenlearning.com/wsu-sandbox/chapter/gestalt-principles-of-perception/

Howdowegroupthings?

• Gestaltprinciples• Principleofsimilarity

https://courses.lumenlearning.com/wsu-sandbox/chapter/gestalt-principles-of-perception/

Howdowegroupthings?

• Gestaltprinciples• Principleofcontinuityandclosure

https://courses.lumenlearning.com/wsu-sandbox/chapter/gestalt-principles-of-perception/

Howdowegroupthings?

• Gestaltprinciples• Principleofcommonfate

Gestaltprinciplesinthecontextofimages• Principleofproximity:nearbypixelsarepartofthesameobject

• Principleofsimilarity:similarpixelsarepartofthesameobject

• Lookfordifferencesincolor,intensity,ortextureacrosstheboundary

• Principleofclosureandcontinuity:contoursarelikelytocontinue

• High-levelknowledge?

RegionsBoundaries

Designingagoodboundarydetector• Differencesincolor,intensity,ortextureacrosstheboundary

• Continuityandclosure• High-levelknowledge

Criteriaforagoodboundarydetector

•Criteriaforagoodboundarydetector:• Gooddetection: Fireonlyonrealedges,notanywhereelse• Goodlocalization

• theedgesdetectedmustbeascloseaspossibletothetrueedges

• thedetectormustreturnonepointonlyforeachtrueedgepoint

Source: L. Fei-Fei

Cannyedgedetector•Theclassicedgedetector•Baselineforalllaterworkongrouping• Theoreticalmodel:step-edgescorruptedbyadditiveGaussiannoise

J.Canny,AComputationalApproachToEdgeDetection,IEEETrans.PatternAnalysisandMachineIntelligence,8:679-714,1986.

Source: L. Fei-Fei22,000 citations!

Example

originalimage

ComputeGradients(DoG)

X-Derivative of Gaussian Y-Derivative of Gaussian

Gradientmagnitudeandorientation• Orientationisundefinedatpixelswith0gradient

Orientationtheta = numpy.arctan2(gy, gx)

Magnitude

Non-maximumsuppressionforeachorientation

Atq,wehaveamaximumifthevalueislargerthanthoseatbothpandatr.Interpolatetogetthesevalues.

Source: D. Forsyth

BeforeNon-maxSuppression

AfterNon-maxSuppression

Hysteresisthresholding• Thresholdatlow/highlevelstogetweak/strongedgepixels• Doconnectedcomponents,startingfromstrongedgepixels

FinalCannyEdges

Cannyedgedetector

1. Filterimagewithx,yderivativesofGaussian2. Findmagnitudeandorientationofgradient3. Non-maximumsuppression:

• Thinmulti-pixelwide“ridges”downtosinglepixelwidth

4. Thresholding andlinking(hysteresis):• Definetwothresholds:lowandhigh• Usethehighthresholdtostartedgecurvesandthelow

thresholdtocontinuethem

Source: D. Lowe, L. Fei-Fei

DoesCannyalwayswork?

Thechallengesofedgedetection

• Texture• Low-contrastboundaries