Three-dimensional image orientation through only onerotation applied to image processing in engineering
Jaime Rodríguez,1,* María T. Martín,1 José Herráez,2 and Pedro Arias3
1Politecnic High School, University of Santiago de Compostela, 27002 Lugo, Spain2Cartographic Engineering School, Polytechnic University of Valencia, 46022 Valencia, Spain
3Mining School, University of Vigo, Rua Maxwell s/n, 36310 Vigo, Spain
An advantage of photogrammetry over othermeasur-ing techniques is its capacity to take measurementswithout any contact with the object, for which it isnecessary to determine the relationship between im-age and object spaces in terms of their exterior orien-tation and scale.Close range photogrammetry applications have
been under study for the last few years by variouspieces of commercial software such as Elcovision [1],Iwitnessphoto [2], and Photomodeler [3] amongothers, as well as by some researchers that havein mind shortening the gap between photogramme-try and nonspecialized users in engineering applica-tions. These are mainly based on the use of low costdigital cameras and on the suppression of methodsand topographic equipment for ground control pointsmeasurement. Ethrog presented a photogrammetric
method for determining the interior orientationparameters and the orientation angles using objectswith parallel and perpendicular lines instead ofcontrol points [4]. The concept of linear featurewas introduced to represent the formulation forphotogrammetric observations and linear featuresto be combined in an adjustment procedure [5].The use of a single image with geometric restrictionsto rebuild objects was studied by Van Den Heuvel [6].The restrictions are based on geometric relationshipamong straight lines like coplanarity, parallelism,perpendicularity, symmetry, and distance. A similartreatment was applied in order to obtain the camerainterior orientation parameters through a photo-graphic shot of an unknown scale introducing a dis-tance in the space object [7]. Arias et al. used a simplemethod of close range photogrammetry to documentagroindustrial constructions in Galicia (Spain) basedon the use of a conventional digital camera calibratedand on vertical plumb lines that allow the orientationof the models generated [8].
The applications of this research in engineeringare very diverse, including fields such as construc-tion and cultural heritage where it is usual to mea-sure objects through direct methods, using tapes andplumb lines. The substitution of these methods by in-direct ones based on close range photogrammetrycan contribute with advantages such as improvingmeasurement precision, time reduction, coarse errorelimination, or the creation of a digital record of thephotographed object, and also from a personal secur-ity support point, since it would limit the operator’smovement around buildings under construction. Therelationship between the scale of image and objectspaces should therefore be set externally to it. To thatpurpose, Arias et al. [8] introduced marks in the or-ientation plumb lines setting at the same time thescale between the spaces for measuring the agroin-dustrial constructions, Tommaselli and Lópes Reisspresented a photogrammetric method based on theuse of a camera and a handheld lasermeter withwhich the dimensions of flat surfaces through a sin-gle picture and the measurement of the distance tothe building surfaces are scaled [9]. More recently,a device for measuring topographic surfaces basedon photos and laser measurements based on thesame idea but including systems of mirrors to alignthe devices axis has been developed [10].This research focuses on the spatial orientation of
the image without having any contact with the object(no ground control points). The methodology, basedon a single photographic image, conducts the exteriororientation between image and object spaces throughthe vanishing line of the image space, that is, the in-tersection of the image plane with a parallel plane tothe object plane passing through the optical center ofthe camera, by means of a single rotation through achange in the coordinate system focused on the mainpoint of the photogram. This enables multiple appli-cations in construction as well as in the field of ro-botics and artificial vision, by only subsequentlyintroducing a single piece of dimensional data toscale a multiple composition of isolated images.
2. Image Orientation
Usually, the 3D coordinate system defined by threeaxes focused on the main point of the image is notparallel to the corresponding coordinate system ofthe object space. Determining the relationship be-tween both systems proves to be simple throughthe calculation of the three existing rotations be-tween them (Euler angles), according to Eq. (1):
0@
XYZ
1A ¼
0@
cosϕ cos χ cosω sin χ þ sin ω sinϕ cos χ sinω sin χ − cosω sinϕ cos χ− cosϕ sin χ cosω cos χ − sinω sinϕ sin χ sinω cos χ þ cosω sinϕ sin χ
sinϕ − sinω cosϕ cosω cosϕ
1A0@
xyc
1A; ð1Þ
where ðX ;Y ;ZÞ are the coordinates of a point in theobject space, ðx; y; cÞ are their corresponding coordi-nates in the image space, and ðω;Φ; χÞ are the treerotations to be determined between both systems.The solution to the system is based on the knowledgeof known points of coordinates in the object spaceðX;Y ;ZÞ, which is not always possible.
To solve this problem we appeal to geometric con-straints such as the use of parallel and perpendicularstraight lines in the object space [12], by which it ispossible to calculate the vanishing line they generatein the image space, through which the orientationcould be made. So that the relationship betweenthe two spaces only requires a photo and subse-quently the knowledge of a scaling factor (λ) if it isintended to measure magnitudes on the object.
In our case, since the objective of the research isthe exterior orientation between the image and theobject spaces to be applied to the measurement ofthe object element’s magnitudes, independently fromeach absolute spatial position, we use a variant of theVan Den Heuvel method restricting the rotation cal-culations, through determining the vanishing line inthe image space, to only one with a variation of theimage coordinate system. Thereby we obtain the or-ientation in an alone rotation, whereas the algorithmof Van Den Heuvel requires Euler’s three rotationsfor the orientation because it keeps the system of co-ordinates fixed in the image using the geometric re-strictions on the element.
A. Calculation of the Vanishing Lines
The determination of the vanishing line requires cal-culation of the vanishing points resulting from theintersections generated by the pairs of parallelstraight lines in the picture, whose image in thephotogram is convergent because of the conical pro-jection.
The vanishing line can be calculated either byusing two pairs of parallel lines to obtain a uniquesolution or by using the adjustment by least-squaresfinding a solution formed by numerous intersections,by means of different pairs of the same plane or evenfrom parallel planes [13].
Given two pairs of straight lines defined by points1, 2, 3, and 4 in Fig. 1, the equation of the vanishingline is
ðy − yaÞ ¼ðx − xaÞðyb − yaÞ
ðxb − xaÞ; ð2Þ
where ðxa; yaÞ and ðxb; ybÞ are the resulting points ofthe intersection of straight lines 1–4 and 2–3 for
point a and straight lines 1–2 and 3–4 for point b inthe image space.The calculation of a and b is done either directly by
marking the points or through automatic edge ex-traction and recognition algorithms [14]. The firstof the procedures suffers from errors caused by theaccuracy in the marking of the points, while the sec-ond is affected by the lighting conditions and theradiometry of the object space (especially outdoors),so that on many occasions the recognition of edgesturns out to be invalid. Proceeding through manualsignaling methods, in order to automate the calcula-tion, it is necessary to add a final premise that en-sures the consistency of the calculation of thevanishing line in accordance with the order of entryof the coordinates of the vertices. An error in captureorder (cross signaling) is easily detectable by
δ ¼ ðx − x1Þðx2 − x1Þ
¼ ðy − y1Þðy2 − y1Þ
; ð3Þ
where, if the result is 0 ≤ δ ≤ 1, the calculated inter-section point is estimated between points 1 and 2 (inthe same way as it would be calculated with points 3and 4), in which case a mistake has been made andthe process must be repeated.
B. Image Orientation
Since the vanishing line is the intersection of theimage plane with a parallel plane to the object planepassing through the optical center of the camera(Fig. 2), the vanishing line is parallel to the objectplane, and at the same time is also parallel with re-spect to the image plane (as it is contained in it). So,using the vanishing line as the axis of rotation of theimage plane, with a single rotation both spaces couldbe oriented.
To do so, the only thing needed will consist of theapplication of a unique rotation ψ to the coordinatesystem focused on the main point of the image, toplace one of its axes parallel to the vanishing lineby creating a new system of coordinates in the image[Fig. 3(a)]. This rotation of the coordinate systemdoes not affect the spatial position of the image. Itis applied to the coordinate system, so that the coor-dinates of each point in the image are recalculatedwith respect to a new system, but its spatial positionremains invariant. If A and B are the two vanishingpoints, with image coordinates ðxa; yaÞ and ðxb; ybÞ,respectively, obtained by the intersection in the im-age of the parallel straight lines in the object space,the angle is given by the equation
ψ ¼ arctanðxa − xbÞðya − ybÞ
: ð4Þ
Once the coordinate image system is positioned, theorientation of the image consists of a rotation ξ on thex0 axis parallel to the vanishing line:
ξ ¼ arctancd; ð5Þ
Fig. 1. Calculation of the vanishing line.
Fig. 2. Image and object coordinate systems. Geometric determi-nation of the vanishing line.
Fig. 3. Image orientation: (a) change of coordinate system inthe image space, rotation ψ and (b) image spatial orientation,rotation ξ.
where c is the focal distance of the camera and d isthe distance between the main point and the vanish-ing line in the image space [Fig. 3(b)] that is given, inthe function of the coordinates of a and b, by the fol-lowing expression:
The orientation of the image calculated by this meth-od becomes relative, taking into account that one ro-tation, with which the absolute orientation would beobtained, has been omitted. However, that rotation iscompletely irrelevant in processes based on isolatedphotographs (without stereoscopy), being able tomeasure any magnitude, as they are given by differ-ences of coordinates, regardless of the orientation ofthe object space.For the measurement of magnitudes it is only ne-
cessary to calculate a scaling factor λ between bothspaces. The scaling can be obtained through any ofthe conventional methods based on a single knownmagnitude, by means of two or more control pointsof known coordinates in the object, or by usingnew hybrid systems based on laser [9] or [10] bythe collinearity condition [15] reduced:
0@
XYZ
1A ¼ λðRξÞ
0@
x0
y0
c
1A ¼ λðRξÞðRψÞ
0@
xyc
1A; ð7Þ
where Rξ is the ξ angle rotation matrix upon the x0axis (spatial rotation of the image and Rψ is the ψangle rotation matrix applied to the coordinate sys-tem of the image (image coordinate system rotationwithout spatial variation).
3. Results
To test the methodology it is necessary to employ anobject space over which magnitudes and/or points ofsupport are known, to provide the scale factor asdata, in order to calculate spacial magnitudes usingboth the implemented and the conventional metho-dology, and contrast results. Numerous shots havebeen taken from different angles of inclination anddistances on a calibrated panel. Any error or differ-ence obtained will be due to the orientation of the im-
age since a common scale will be the starting point inboth cases.
To prove the suitability of the system a series oftests have been done using the following in-struments:
2. Calibrated panelDimensions: 0:798 × 0:399m (forming a mesh of
9 × 5 marks)
In order to determine the precision of the imple-mented algorithm, several tests were made measur-ing over a rectangular panel of known dimensionsplaced on a wall.
In the first place, a test conducted at a distance ofapproximately 3 m with a slope of approximately 40g
is shown (Fig. 4). Once the image is set, the four cor-ners of the panel are marked over the image andtheir coordinates calculated, and then the two di-mensions of the panel (height and width) are calcu-lated. Table 1 shows the real length of the two sidesof the panel and the onesmeasured with the proposalmethod, taking for calculation of the value the scale
Fig. 4. Photographic shot at a distance of approximately 3 mwitha slope of approximately 40g over a rectangular panel of knowndimensions.
Table 1. Comparison of the Dimensions of a Calibrated Panel with the Proposal Method for Different Definitions of the Vanishing Line in theSame Image (in Meters)
Horizontal Dimension Vertical Dimension
Vanishing Line Real Photograph Difference Real Photograph Difference
factor of the known distance between two marks ofthe panel, as well as the differences between them(marks have been numbered correlatively beginningfrom the lower left corner). The maximum absoluteerror is generated in the horizontal dimension(0:6 cm), whereas the maximum relative error is gen-erated in the vertical one (1%), which seems to be co-herent with the definition of the vanishing pointsdepending on the geometry of the element. Each va-lue corresponds to a different definition of the van-ishing line. In order to study the cumulativefrequency distribution of the absolute errors, thatare obtained comparing real dimensions with theones obtained with the proposed method, a squarepanel of 0:8 × 0:8m has been used (Fig. 5). A totalof 50 measurements from the panel were calculated(hence, 50 values of the error were obtained), eachone corresponding to a vanishing line obtained start-ing from two different pairs of parallel lines. As canbe seen, 90% of the errors are less than 5mm, whichforms a relative error of 0.6%.Tests carried out at a distance of about 7 m from
different positions are shown in Table 2 (Fig. 6).The results obtained show a lower accuracy causedby the distance to the item, whichmeans lower imageresolution and therefore the marking of the pointsthat lead to the calculation of the vanishing line is
less precise. Still the maximum error obtained at thisdistance is 3:2 cm (a relative error of 4% in width)and 2:2 cm in height (relative error of 5.5%).
In both cases we can state that best results wereobtained when the camera axis is not exactly perpen-dicular to the wall plane, in a way that it is more ac-curate to define the vanishing lines. The studiescarried out from different places can be closer tothree intervals for short distances to the element (be-tween 1 and 5 m): the interval between −70g < α <70g samples very precise values while in −100g < α <−70g and 70g < α < 100g errors caused by the per-spective suppose a loss of precision that increasesvery rapidly, reaching inconsistent results in the de-termination of the vanishing line as we approach100g (Fig. 7). However studies conducted at distancesgreater than 5 m show a drastic loss of precision outof range −50g < α < 50g due to the fact that the per-spective is considerably affected by any minimum er-ror in the marking of the points that define thevanishing line. In order to maintain the accuracywhile increasing the distance it would be necessaryto have a focal increase in the same proportion. Like-wise there is a “black point” within the optimal inter-val in all cases which corresponds to the perfectlyperpendicular position to the object, whose vanishingline is formed at infinity.
4. Conclusions
The obtained precision in the measurement of ele-ments is directly proportional to the accuracy withwhich the vanishing line is calculated, and thereforethe angle of inclination with respect to the objectplane is decisive.
However, for analysis of angle of inclination, it isfirst necessary to separate, as two independent op-erations, the calculation of the vanishing line inthe image space and the measuring of elements inthe object space, since the formation of the vanishingline can be carried out through the element beingmeasured or can be totally independent of it. In caseswhere the element to be measured itself determinesthe vanishing line, the accuracy obtained will dependon the suitability of the mentioned object, both for itsgeometry and by its position and size within the im-age. In contrast, in those cases where the vanishing
Table 2. Comparison of the Dimensions of a Calibrated Panel with the Proposed Method for Different Definitions of the Vanishing Line fromDifferent Inclination Angles (α) (in Meters)
Horizontal Dimension Vertical Dimension
Camera Position α (approx.) Real Photograph Difference Real Photograph Difference
line is not determined by the object subject to mea-surement it will be crucial, besides its geometric suit-ability, whether the selected element includes theelement to be measured, resulting in the first caseof greater precision.Separating both concepts (calculation of the van-
ishing line on one side and measurement of the ele-
ment on the other), with respect to determination ofthe vanishing line, we can state that the more regu-lar the object is (obtaining very unlike errors whiledetermining one of the vanishing points when thegeometry is very different in one direction), the short-er the distance to the object is, and/or the greater theelement in the image is (the points defining the van-ishing line that result are much more accurate), andthe more focused it is, the better the vanishing linewill be (the conical projection and distortions willhave less effect). With regard to the inclination thedisplacement along a theoretical horizontal axis,from which α intervals are obtained, must be linkedto similar values of inclination in the vertical so asnot to get in any case parallel straight lines in thephotogram, which would cause one to obtain one ofthe points a or b that determine the vanishing lineat infinity.
Regarding the measurement of the elements, themethod results are independent of their dimensions(always when they are not too small within the imageand/or are very off-center) and perspective (withinthe specified ranges in which the definition of thepoints is univocal). The developed methodology en-ables the measurement of any kind of regular or ir-regular flat geometry, as well as of an infinitenumber of elements that can be found in the sameobject plane, as long as there are parallel lines inthe image that will allow us to determine the vanish-ing line of the plane with accuracy. Likewise, the cal-culation of the exterior orientation is possible by thedetermination of the vanishing line in an iterativefashion across all elements that meet the require-ments within the same image. This enables a moreaccurate estimation at the same time as it shows pos-sible mistakes in the guidance through the residuals
Fig. 6. Photographic shots at a distance of approximately 7 mfrom different slopes over a rectangular panel of known dimen-sions.
Fig. 7. Optimal interval of inclination of the shots in a theoreticalhorizontal plane with respect to the perpendicular from the object(α decreases with distance).
of the adjustment. Besides which it is essential inshots where the object represents a very small por-tion in the image (either by its small size or by thedistance from the camera), allowing a better fit. Inthe same way, it is convenient to indicate that the in-clusion of equations of rectangularity conditions ofthe objects (if any) means more consistency to the ad-justment including restrictions through unit vectors.However, despite the satisfactory results obtained,
the image resolution represents a key factor to obtainenough accuracy, since the marking of the vanishingpoints as well as the marking of the vertices of anelement is directly affected by the pixel size of theimage. Also regarding the data capture that deter-mines the calculation of the vanishing lines, onceit has been manually verified that the developedmethod is valid, its automation through automaticextraction of edges is of greater interest. The advan-tage of these algorithms is that they are very quick,no operator intervention is required, and also theycalculate all the possible vanishing lines on the im-age with the straight lines detected, which accordingto the above-mentioned statements represents a sig-nificant advantage. However, the problems arisingfrom the lighting conditions and radiometry while se-lecting invalid straight lines assume that the processshould always be reviewed by an operator, allowinghis or her interaction with the aim of eliminating in-valid data.
References
1. Elcovision, Product information on the internet at http://www.elcovison.com (accessed January 2008).
2. Iwitnessphoto, Product information on the internet at http://www.iwitnessphoto.com (accessed January 2008).
3. Photomedeler, Product information on the internet at http://(accessed January 2008).
4. U. Ethrog, “Non-metric camera calibration and photo orienta-tion using parallel and perpendicular lines of the photo-graphed objects,” Photogrammetria 39, 13–22 (1984).
5. D. C. Mulawa and E. M. Mikhail, “Photogrammetric treat-ment of linear features,” in International Archives of Photo-grammetry and Remote Sensing (International Society forPhotogrammetry and Remote Sensing, 1988), pp. 383–393.
6. F. A. Van Den Heuvel, “3D reconstruction from a single imageusing geometric constraints,” ISPRS J. Photogramm. RemoteSens. 53, 354–368 (1998).
7. R. M. Haralick, “Determining camera parameters from theperspectiva projection of a rectangle,” Pattern Recogn. 22,225–230 (1989).
8. P. Arias, C. Ordóñez, H. Lorenzo, and J. Herráez, “Documen-tation for the preservation of traditional agro-industrial build-ings in N.W. Spain using simple close range photogrammetry,”Surv. Rev. 38, 525–540 (2006).
9. A. M. G. Tommaselli and M. L. Lopes Reiss, “A photogram-metric method for single orientation and measurement,”Photogramm. Eng. Remote Sens. 71, 727–732 (2005).
10. T. Ohdake and H. Chikatsu, “Evaluation of image based inte-grated measurement system and its application to topo-graphic survey,” International Archives of Photogrammetryand Remote Sensing (International Society for Photogramme-try and Remote Sensing, 2006).
11. P. R. Wolf, ed., Elements of Photogrammetry, with AirPhoto Interpretation and Remote Sensing (McGraw-Hill,1983).
12. F. A. Van Den Heuvel, “Exterior orientation usingcoplanar parallel lines,” Proccedings of the 10th ScandinavianConference on Image Analysis (Lappeenranta, 1997), pp.71–78.
13. A. Criminisi, I. Reid, and A. Zisserman, “Single view metrol-ogy,” in Proccedings of the 11th International Conference onComputer Vision (Kerkyra, 1999), pp. 434–441.
14. F. Schaffalitzky and A. Zisserman, “Planar grouping forautomatic detection of vanishing lines and points,” Image Vi-sion Comput. 18, 647–658 (2000).
15. W. Zhizhuo, “Principles of Photogrammetry (with remotesensing),” in Press of Wuhan Tecnical University of Surveyingand Mapping (Publishing House of Surveying and Map-ping, 1991).
