8/2/2019 Radial Distortion Snakes

1/4

MVA2000 IAPR Workshop on Machine Vision Applications, Nov. 28-30.2000, The University of Tokyo, Japan

Radial Distortion SnakesSing Bing Kang *Microsoft Research, Microsoft Corp orationOne M icrosoft Way, Redmond WA 98052, USA

sbkang~microsoft.com

AbstractIn this paper, we address the problem of recover-ing the camera radial distortion coefficients from oneimage. The approach tha t we propose uses a specialkind of snakes called radial distortion snakes. Ra-dial distortion snakes behave like conventional de-formable contours, except that their behavior areglobally connected via a consistent model of imageradial distortion. Experiments show that radial dis-tortion snakes are more robust and accurate thanconventional snakes and manual point selection.

1 IntroductionMost cameras with wide fields of view suffer fromnon-linear distortion due to simplified lens construc-tion and lens imperfection. In general, there are twoforms of camera distortion, namely radial distortionand tangential distortion. In this paper, we addressthe problem of recovering the camera radial distor-tion coefficients from one image. We present a newtechnique that uses what we call radial distortion

snakes. Unlike conventional snakes, the behavior ofradial distortion snakes are globally connected viaa consistent model of image radial distortion. Ourradial distortion recovery is directly linked with thefeature (edge) detection process.1.1 Prior work

A lot of work on camera calibration require acalibration pattern with known exact dimensions.

Brown [3] uses a number of parallel plumb linesto compute the radial distortion parameters using aniterative gradient-descent technique. The extractionof points on the plumb lines is very manual intensive.Swaminathan and Nayar [9] use a user-guided self-calibration approach. The distortion parameters arecomputed from user-picked points along projectionsof straight lines in the image.Stein [8] uses point correspondences betweenmultiple views to extract radial distortion coeffi-cients. He uses epipolar and trilinear constraintsand searches for the amount of radial distortion thatminimizes the errors in these constraints.Photogrammetry methods usually rely on usingknown calibration points or st ructures [2 , 3, 101. For

example, Tsai [lo] uses corners of regularly spacedboxes of known dimensions for full camera calibra-tion.The idea of active deformable contours, or snakes,was first described in [ 6 ] .Since then, there has beennumerous papers on the applications and refinementof snakes. Virtually all the snakes, some of whichmay be parameterized, work independently of eachother. Our snakes are globally parameterized, andthey deform in a globally consistent manner.

2 Finding the radial distortion param-etersWe begin this section with a brief description ofthe lens distortion equation.

There are camera calibration techniques tha t use thescene image or images themselves, and possibly tak- 2.1 The radial distortion equationing advantage of special structures such as straightlines, parallel straight lines, and perpendicular lines. The modeling of lens distortion can be found inAn example is that of Becker and Bove [I]. They use [7]. In essence, there are two kinds of lens distor-the minimum vanishing point dispersion constraint tion, namely radial and tangential (or decentering)to estimate both radial and decentering (or tangen- distortion. Each kind of distortion is represented bytial) lens distortion. The user has to group parallel an infinite series, but generally, a small number islines together. adequate.

* Work done while the author was with Cambridge Re- We assume that the tangential clistortion can besearch Lab., Compaq Computer Corp. neglected and the principal point is the center of the

8/2/2019 Radial Distortion Snakes

2/4

image. The radial distortion equations are then

where IC'S are the radial distortion parameters,(xu,y,) is the theoretical undistorted image pointlocation, (xd, yd) is the measured distorted imagepoint location, and Rd = xz + yz.In our approach, the user draws lines on the imagethat correspond to projections of straight 3-D lines.These drawn lines need not be exact. We then usesnakes to search for the best-fit lines to extract radialdistortion parameters. A direct approach would beto use normal snakes.2.2 Using conventional snakes

In this method, the motion of the snakes is basedon two factors: motion smoothness (due to exter-nal forces) and spatial smoothness (due to internalforces). Given the original configuration of a snake,a point on the snake p j moves by the amount 6pjat each step given by

(3)where Nj and N, are th e neighborhood of pixel atpj, including pj. 6pedgej s the computed motionof the ith point towards the nearest detectable edge,with its magnitude being inversely proportional toits local intensity gradient. pjk and pi k are the re-spective neighborhood weights. In our implementa-tion, X = 0.5 and Nj =Nj, the radius of the neigh-borhood being 5 and the weights pjk = pik being{1,2,4,8,16,32,16,8,4,2,1).Once the snakes have settled, the camera radialdistortion parameters can then be recovered using aleast-squares formulation.2.3 Using radial distortion snakes

Using conventional snakes have the problem ofgetting stuck on wrong local minima. This problemcan be reduced by imposing more structure on thesnake-namely, the shape of the snake has to beconsistent with the expected distortion of straightlines due to global radial distortion. For this reason,we call .such snakes radial distortion snakes.The complexity of the original objective functioncan be reduced if we consider the fact that the effectof radial distortion is rotationally invariant about

the principal point, ignoring asymmetric distortionsdue to tangential distortion and non-unit aspect ra-tio.This method has the following steps:1. For each snake, find the best fit line,2. Rotate each snake about the principal point soth at the rotated best fit line is horizontal. Letthe angle of this rotation be a, for the i th snake.3. Estimate best fit set of radial distortion param-eters I C ~ ,.., ICL from the rotated set of lines (de-scribed shortly).4. Find the expected rotated distorted point pi =(xj, yj), whose undistorted version lies on a hor-

izontal line, i.e.,

(0) (0 )Given the initial points (xj ,yj ), we take X j =x y) and iteratively compute yj from

until the difference between successive values ofyik)s is negligible.5. Update points using current best estimate of n'sand edge normal. In other words, the point p jis updated based on

with 0 5 q 5 1. pYma1s the expected newposition of the snake point using the conven-tional snake approach (see ( 3 ) ) . For the ithsnake,pyPas obtained by rotating p i (calcu-lated from the previous step) about the princi-pal point by (-ai).

6. Iterate all the above steps until overall meanchange is negligible, or for a fixed number ofiterations. The latter condition is adopted inour work.In our case, we set the time-varying function of qto be linear from 0 to 1 with respect to the presetmaximum number of iterations.To find the radial distortion parameters ~ 1 ' siven

rotated coordinates, we minimize the objective func-tion

8/2/2019 Radial Distortion Snakes

3/4

where N , is the number of snakes, Rij = x:j + Y,2jand wij is the confidence weight associated with thej t h point in the it h snake. In our work, wij is theedge strength at ( x i j , i j ) We use the rotated ver-sions in order t o simplify the analysis of the objectivefunction.Taking the part ial derivatives with respect to theunknown parameters and equating them to zero, weget a linear system of equations that can be easilysolved for K,'s. For L > 1, we estimate the K'S insuccession. In other words, we first estimate K I , fol-lowed by 61 an d K Z , and so on, up ti1 the last stagewhere we estimate all the radial distortion parame-ters KI, ...,KL .3 Results

In this section, we present results from both syn-thetic and real images. For all the experiments de-scribed in this section, we recover K I and KZ only,i.e., we set L = 2. This is generally sufficient for lowto moderately distorted images in practice.3.1 Experiments using synthetic images

In our first set of experiments, we use syntheticimages containing straight lines and distort themwith known radial distortion parameters. In addi-tion, we vary the image noise to see how it affectsboth the conventional and radial distortion snakealgorithms. In particular, the actual radial distor-tion parameters corresponding to = and~2 = 10-lo are applied to images with a resolutionof 480 x 512. The gaussian image noise (specified bythe standard deviation in intensity level a ) is variedfrom 0 to 100 intensity levels. Figure 1 shows re-sults for a test image with a = 100. It is clear fromFigure 1 that the radial distortion snakes yielded abetter result than that of conventional snakes.The results of the series of experiments are shownin Figure 2. As can also be seen, for low image noiselevels, both snake algorithms exhibit reasonable ro-bustness to image noise. However, the radial distor-tion snake algorithm is even more stable despite thepresence of significant image noise, in comparison tothe conventional snake algorithm.3.2 Experiments using real images

Figure 3 illustrates a situation where the radialdistortion snakes appear to have converged to amore optimal local minima than that of conventionalsnakes for the same snake initialization. This exam-ple shows that the radial distortion snakes are moretolerant to errors in snake initialization by the user.Our algorithm works for highly distorted images aswell, as Figure 4 shows.

Figure 1: Synthetic image with a = 100: (a) Origi-nal image, (b) Manually drawn lines, (c) With con-ventional snakes, (d) With radial distortion snakes.

Radial distortion snakes appear to have the ef-fect of widening the range of convergence comparedto conventional snakes (as exemplified by Figure 3) .Despite this, wrong convergence do occur with radialdistortion snakes, especially in cases of bad initialline placements.4 Discussion

I t is clear from experiments that using the radialdistortion snakes is better than using conventionalsnakes. We have demonstrated that the radial dis-tortion snakes find best adaptation according to bestglobal fit to radial distortion parameters. They ap-pear to be less prone to being trapped in bad localminima in comparison to conventional snakes. Atevery step, the radial distortion snakes act togetherto give an optimal estimate of the global radial dis-tortion parameters and deform in a consistent man-ner in approaching edges in the image.In comparison to the radial distortion snake, eachconventional snake is locally adaptive and works in-dependently of all the other snakes in the same im-age. They are not specialized, nor are they designedto be optimal to the task (in our case, the recoveryof radial distortion parameters). This is clearly an-other demonstration of the benefit of incorporatingglobal task knowledge directly in the early stagesof the problem-solving algorithm. The concept ofthe radial distortion snake is very much in the samespirit as that of task-oriented vision [5].

8/2/2019 Radial Distortion Snakes

4/4

5 Summary and future work--. Conventional snakes- adial dirtonio n snakes. Manually placed painls-. - Manually placed ines

0.0-.0 20.0 40.0 64.0 80.0 ICQ.0Image n o i x

Figure 2: Graph showing the effect of gaussian imagenoise (standard deviation in intensity level) on RMSundistortion error ERMS.

Figure 3: Office scene: (a) Initial snake configura-tion, and final snake configuration for (b) Conven-tional snakes, (c) Radial distortion snakes, (d) Cor-rected image from (c).

Figure 4: An example with very significant distor-tion: (a) Initial snake configuration, (b) Final snakeconfiguration, and (c) Corrected image. Note th atthe snakes are shown in black here.

We have described radial distortion snakes as amechanism to recover radial distortion parametersfrom a single image. Radial distortion snakes de-form in concert based on a common radial distortionmodel.One direction for future work is to extend thiswork to estimate the principal point and tangen-tial (or decentering) distortion parameters as well.Another area is to fully automate the process of de-termining radial distortion by edge detection andlinking, followed by hypothesis and testing. A ro-bust estimator may be used to reject outliers (e.g.,RANSAC-like algorithm [4]).References

[I ] S. Becker and V. B. Bove. Semiautomatic 3-Dmodel extraction from uncalibrated 2-D cameraviews. In SPIE Visual Data Exploration andAnalysis 11,volume 2410, pages 447-461, Feb.1995.[2] D. C. Brown. Decentering distortion of lenses.Photogrammetric Engrg., 32(3):444-462, May1966.[3] D. C. Brown. Close-range camera calibration.Photogrammetric Engrg., 37(8):855-866, Aug.1971.[4] M.A. Fischler and R.C. Bolles. Random sampleconsensus: A paradigm for model fitting withapplications to image analysis and automatedcartography. Comm. of the ACM, 24(6):381-395, June 1981.[5] K. Ikeuchi and M. Hebert. Task oriented vision.In Image Understanding Workshop, pages 497-507, Pittsburgh, PA, Sept. 1990.[6] M. Kass, A. Witkin, and D. Terzopoulos. Snakes:Active contour models. In ICCV, pages 259-268, London, England, June 1987.[7] C. C. Slama, editor. Manual of Photogramme-try. American Soc. of Photogrammetry, FallsChurch, VA, 1980.[8] G. P. Stein. Lens distortion calibration usingpoint correspondences. A. I. Memo 1595, MIT,Nov. 1996.[9] R. Swaminathan and S. Nayar. Non-metric cal-ibration of wide-angle lenses and polycameras.In CVPR, volume 2, pages 413-419, Fort Collins,CO, June 1999.

[ lo] R. Y. Tsai . A versatile camera calibration tech-nique for high-accuracy 3D machine vision metrol-ogy using off-the-shelf TV cameras and lenses.IEEE J. Robotics and Automation, RA-3(4):323-344, Aug. 1987.