Interactive Image Segmentation using Graph Cuts

Mayuresh Kulkarni and Fred NicollsDigital Image Processing Group

University of Cape Town

PRASA 2009

Outline

• Image Segmentation Problem• Our Approach• Graph cuts and Gaussian Mixture Models• Results and Discussion• Future Research

What is foreground?

Image Segmentation

Our Approach

Graph Cuts SegmentationCost Function : E(A) = λ R(A) + B(A)

Region information Boundary information Pixel connectivity

8 – pixel neighbourhoodDifference between adjacent pixels

Image propertieseg. colour, texture

Graph Cuts

Source (foreground)

Sink (background)

Cost Function : E(A) = λ R(A) + B(A)

Pixel connectivity (boundaries)Inter-pixel weights (boundaries)

Source and Sink weights (regions)

Gaussian Mixture Models

Background GMM Foreground GMM

Gaussian Mixture ModelsForeground

GMM

Background GMM

Log Likelihood Ratio = log(K *pf/pb)

pf

pb

GMM components

• Greyscale images– Intensity values– Intensity values and

MR8 filters

• Colour images– RGB values– G, (G-R), (G-B) values– Luv values– MR8 filters

Boundary information

• Inter-pixel weights– Edge detection– Difference between

adjacent pixels– Gradient

• Pixel connectivity

Results

Κ = 0.01 Κ = 0.1 Κ = 1

Results

Original Image

RGB, Luv and MR8 (Fscore = 0.916)

Luv and MR8 (Fscore = 0.921)

Luv (Fscore = 0.934)

Results

Original Image RGB, Luv and MR8 (Fscore = 0.906)

RGB (Fscore = 0.951)Luv (Fscore = 0.945)

Analysis of Results

• Accurate segmentation achieved• Components in the GMM depend on image• Segmentation can be controlled using K and λ

Future Research

• Different grid (non-pixel grid)• Ratio cuts• Exploring other statistical models• ObjCut – segmenting particular objects

References• Y. Boykov and M. P. Jolly. Interactive graph cuts for optimal boundary and region

segmentation of objects in N-D images. In ICCV, volume 1, pages 105–112, July 2001.• Yuri Boykov and Vladimir Kolmogorov. An experimental comparison of min-cut/max-flow

algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell., 26(9):1124–1137, 2004.

• Pushmeet Kohli, Jonathan Rihan, Matthieu Bray, and Philip H. S. Torr. Simultaneous segmentation and pose estimation of humans using dynamic graph cuts. International Journal of Computer Vision, 79(3):285–298, 2008.

• H. Permuter, J. Francos, and I. Jermyn. Gaussian mixture models of texture and colour for image database. In ICASSP, pages 25–88, 2003.

• D. Martin, C. Fowlkes, D. Tal, and J. Malik. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proc. 8th Int’l Conf. Computer Vision, volume 2, pages 416–423, July 2001.

• Carsten Rother, Vladimir Kolmogorov, and Andrew Blake. “GrabCut”: interactive foreground extraction using iterated graph cuts. ACM Trans. Graph., 23(3):309–314, August 2004.