University of ZurichZurich Open Repository and Archive

Winterthurerstr. 190

CH-8057 Zurich

http://www.zora.uzh.ch

Year: 2010

Precision and accuracy of the 3dMD photogrammetric system incraniomaxillofacial application

Lübbers, H T; Medinger, L; Kruse, A; Grätz, K W; Matthews, F

Lübbers, H T; Medinger, L; Kruse, A; Grätz, K W; Matthews, F (2010). Precision and accuracy of the 3dMDphotogrammetric system in craniomaxillofacial application. Journal of Craniofacial Surgery, 21(3):763-767.Postprint available at:http://www.zora.uzh.ch

Posted at the Zurich Open Repository and Archive, University of Zurich.http://www.zora.uzh.ch

Originally published at:Journal of Craniofacial Surgery 2010, 21(3):763-767.

Lübbers, H T; Medinger, L; Kruse, A; Grätz, K W; Matthews, F (2010). Precision and accuracy of the 3dMDphotogrammetric system in craniomaxillofacial application. Journal of Craniofacial Surgery, 21(3):763-767.Postprint available at:http://www.zora.uzh.ch

Posted at the Zurich Open Repository and Archive, University of Zurich.http://www.zora.uzh.ch

Originally published at:Journal of Craniofacial Surgery 2010, 21(3):763-767.

1

Precision and accuracy of the 3dMDfaceTM photogrammetric system in cranio-

maxillo-facial application

Lübbers, Heinz-Theo MD, DMD*†; Medinger, Laurent MD, DMD*; Kruse, Astrid MD, DMD*; Grätz, Klaus Wilhelm

MD, DMD*; Matthews, Felix MD, MBA†

Address correspondence and reprint requests to Heinz-Theo Lübbers, MD, DMD, Clinic for Cranio-Maxillofacial

Surgery, University Hospital of Zurich, Frauenklinikstrasse 24, CH-8091 Zurich, Switzerland; E-mail: heinz-

theo.luebbers@usz.ch

Abstract

Background

In modern anthropometry of such complex structures as the face, 3D scanning

techniques have become more and more common. Before establishing them as a

golden standard, however, meticulous evaluation of their precision and accuracy

under both ideal and clinical circumstances is essential. Potential sources of error

need to be identified and addressed.

Materials and methods

Under ideal circumstances, a phantom is used to examine the precision and

accuracy of the 3dMDfaceTM system. A clinical setting is simulated by varying

different parameters like angle, distance, and system re-registration, as well as data

evaluation under different levels of magnification.

Results

The handling of the system was unproblematic in matters of data acquisition

and data analysis. It was very reliable, with a mean global error of 0.2mm (range 0.1

– 0.5mm) for mannequin head measurements. Neither the position of the head nor of

the camera influenced these parameters. New referencing of the system did not

influence precision and accuracy.

2

Conclusions

The precision and accuracy of the tested system is more than sufficient for

clinical needs and greater than that of other methods, such as direct anthropometry

and 2d-photography. The evaluated system can be recommended for evaluation and

documentation of the facial surface and could offer new opportunities in

reconstructive, orthognatic, and craniofacial surgery.

Keywords

plastic surgery; maxillofacial surgery; craniofacial surgery; comparative study;

phantom; imaging; photogrammetric

3

INTRODUCTION

Anthropometry, which was developed in the late 19th century, is the biological

science of measuring the human body and its characteristics. (1) Though its

applications are usually medical, today it also plays an important role in commercial

settings, like clothing design, ergonomics, and architecture.

In cranio-maxillofacial and plastic surgery, anthropometry is especially

challenging due to the complex structures of the face, which do not allow an

assessment with simple measurements. For the underlying bony structures, the

development of computer tomography (CT) by Hounsfield (2) and Ambrose (3)

solved the difficulties. An objective, accurate, and reliable system for quantifying the

soft tissues of the face in dimension and color is still needed.

Today, direct measurements and 2d photography are still state of the art for

craniofacial anthropometry (4, 5), even though the pitfalls are well known and

discussed. (1, 6-11) However, interest in overcoming the limitations of these

techniques has led to the development of numerous 3D scanning devices which have

an obvious appeal over the “old-fashioned” techniques. (12-20) Kau et al. give an

overview of the 3d scanning device types available. (21)

Despite the huge amount of literature about the new 3D-systems, a clear and

objective evaluation of accuracy and reliability under different circumstances is

missing for many of them. Obviously, before any of these new techniques is applied

clinically, it is crucial to evaluate their reliability. However, 3D representation of skull

and soft tissue is a promising tool in orthognatic, craniofacial, and reconstructive

surgery. In complicated cases, 3D stereolithography models are nowadays often

necessary and their production is time- and cost-consuming.

Aim of the study

4

The aim of the study is to evaluate the precision (repeatability and

reproducibility) and accuracy of the 3dMDfaceTM system. Therefore, a phantom was

used, and various sources of error were examined. Furthermore, besides operator

and capture errors, accuracy and bias in relation to direct anthropometry are

evaluated.

Material and Methods

Model

For the experiments, a mannequin head was chosen as an ideal object because

it does not move or perform facial expression. To minimize errors resulting from

landmark identification, the mannequin head was prepared with 41 artificial

landmarks as shown in figure 1. The labels were positioned to cover all face regions,

with emphasis on the oral-nasal region.

Data acquisition

The direct line distances between the pre-labeled landmarks on the mannequin

head were measured with standard clinical sliding and spreading calipers and

measuring tape. Measurement was performed by three observers in one session at

the same time and place that the images were captured by the 3dMDfaceTM system

(Table 1, Study No. 1). Each observer measured the distances 5 times. The median

of all 15 measurements for each distance was accepted as the real distance between

the two labels.

The 3D data was acquired under clinical lighting using a 3dMDfaceTM System

(3dMD Inc., Atlanta, GA, USA). The system is based on the combination of

stereophotogrammetry and structured light, and is connected to a personal desktop

computer where the captured data set is saved and calculated into a 3D VRML file

(45,000 to 65,000 polygons) ready for evaluation. Data acquisition was performed in

5

natural head posture (NHP), with the Frankfurt horizontal line parallel to the floor and

with variations following the protocol given in table 1. If shown in the table, new

system calibration was performed before image capturing.

Data processing

Further data processing was performed on a standard desktop computer using

the 3dMD-Patient-Software (3dMD Inc., Atlanta, GA, USA) belonging to the capture

device. The labels were digitized on the surface of the 3D model and the x-, y- and z-

coordinates of this markings were exported to an Excel 2003 file (Microsoft

Corporation, Redmond, WA, USA) for further calculations. A zoom tool could be used

for magnification on the screen. Single coordinates were excluded when not being

captured because they were out of the field of vision due to rotation of the head.

Operational definitions

As the aim of this study was to validate how accurate the 3dMDfaceTM system is

compared to the “gold standard” of direct measurements, this standard is

operationally defined by accuracy, bias, and precision.

1. Accuracy is the agreement between a measurement and the “true” value of a

parameter (22, 23)—in our case, the 3D model and the results of direct

anthropometry.

2. Bias measures whether 3dMD tends to over- or underestimate direct values

systematically.

3. Precision is divided into the following sub elements:

a. Repeatability is the degree of similarity of multiple measurements of the

same part using the same technique. This aspect has 3 subdivisions:

i. “Operator error,” which results from inaccuracies during

repeated digital measurements of the same 3D model

derived once out of one dataset;

6

ii. “Capture error,” which results from a systems error when

capturing the same object multiple times.

iii. Registration error, which is that added by new calibration

of the system in between two captures.

b. Reproducibility is the magnitude of the differences between repeated

measurements by different operators who are using the same

technique. (22, 24)

The discrepancy, which is the distance between 2 landmark coordinates and is

calculated as the square root of the sum of squared deviation in all 3 spatial

directions is 222 zyx Δ+Δ+Δ , an analog to the target registration error (TRE)

described in different articles. (25-27)

Data analysis

To assess the above-mentioned parameters, two kinds of measurements were

performed:

1. Point error measurement

By means of a fusion analysis, the 3D coordinates of each landmark were

aligned via translation and rotation to match the coordinates of the

corresponding landmark on another model. The null hypothesis was that there

is one translation matrix for all corresponding landmarks, leading to a perfect

fit.

2. Distance error measurements

Though 3D coordinates weren’t available for direct anthropometry, the

distances directly measured on the mannequin head were compared to the

corresponding distances calculated from the 3D coordinates of the VRLM

models by the above-mentioned formula caliper distance, which equals

7

222 zyx Δ+Δ+Δ . The null hypothesis was that corresponding distances are

identical.

Statistic tools

The acquired data was analyzed using descriptive statistics as well as

parametric student t-tests. The tests were performed with SPSS 11.5 (SPSS Inc,

Chicago, IL, USA) and were considered significant if p<0.05.

Results

Operator error

Operator error as an error resulting from inaccuracies in placing the landmarks

was assessed by multiple redigitizing of one dataset. The dataset was analyzed 20

times without the zoom feature of the Software and 20 times with zoom (factor 10).

The TRE between corresponding landmarks was calculated.

The measurements without zoom (figure 2) show an operator error of an

average TRE of 0.10mm with a minimum of 0.001 and a maximum of 0.419mm.

The measurements with zoom (figure 3) show a significantly (p<0.01) reduced

operator error with an average TRE of 0.04mm.

Capture error, recalibration

Comparing landmark configurations from different image datasets of the same

object quantifies the instability of the system. This was examined by studies No. 2

and 3 as outlined in table 1.

Figure 4 shows the results for a situation with re-calibration of the capture

system before the acquisition of each dataset. The average TRE was 0.11mm with a

range of 0.01 to 0.57mm. Taking into account the operator error from figure 2, the

result is an instability of about 0.01mm (p=0.15, difference not significant) when

images are taken with a re-calibration of the system in between.

8

Capture error, object positions

The influence of different object positions was evaluated through studies no. 4,

5 and 6 according to table 1. Representatively, the results of study row 5 are shown

in figure 5. For evaluation the datasets were fused onto each other by rotation and

translation until maximum superposition was achieved. Zero degree rotation was

defined as the reference dataset. The mean TRE was 0.195mm with a range of 0.01

to 0.59mm. Taking into account the operator error from figure 2, the results are an

instability of about 0.095mm when images are taken of the same object in different

positions. These differences were not statistically significant for all measurements,

even if figure 5 shows a little increase of the mean TRE with a greater deviation from

the neutral position.

Accuracy and bias to direct anthropometry

To evaluate any differences between the 3D photo and reality, 201 distances

between landmarks were measured by caliper and compared to the corresponding

distances derived out of the 3D data.

A Pearson’s product-moment correlation coefficient (r) was calculated to

compare direct and digital values for the measurements. Correlations were all

statistically significant, with a grand mean of the r values calculated across all valid

distances as 1.00. Paired t-tests comparing means between digital and direct

measurements for all distances demonstrated a statistically insignificant difference.

Additionally, linear regression analysis was performed. The allocation of the

differences between direct measurements and distances derived out of the 3D data is

shown in figure 6.

A comparative analysis of all error classes is given in figure 7.

Observation during application

9

The observation during testing showed some downsides of the technology.

First, there are difficulties in capturing data if hair compromises any of the camera’s

view of the area. Prominent areas can compromise the view of less prominent areas,

resulting in poor 3D representation.

DISCUSSION

Conventional methods for studying facial symmetry have limitations. Radiography

measures skeletal landmarks, but ignores the aesthetic aspects of soft tissue. The

3dMD face system allows the collection of images stored in digital format. In the

present study, data acquisition was performed in NHP, because Kau et al. were able

to show that this position is clinically reproducible. (28)

The evaluated parameters of accuracy, bias, and precision are outlined above

and basically represent the quality of the produced 3D model when it is matched to

reality. (29)

Concerning operator error, we recommend using the zoom tool whenever the

operator is in any doubt about a landmark, but not as a routine. Even if the operator

error can be reduced by a factor of 2 (from 0.1mm to 0.04mm), through using the

zoom facility of the software, this strategy seems unnecessary due to the negligible

error even without zoom.

The error of recalibration of the system (about 0.01mm with consideration of

operator error), as outlined is figure 4, is negligible for itself and especially in

comparison to the operator error, which is about 10 times higher.

The error resulting from the object’s position, as shown in figure 5 (about

0.095mm with consideration of operator error), is about the size of the operator error

and in itself is negligible for clinical application.

10

Another point of discussion is the accuracy and bias compared to direct

anthropometry. The distances directly measured and the distances calculated out of

the datasets revealed no relevant difference and therefore no imminent error that

would lead to systematically wrong results.

Overall, the system error of the 3dMDfaceTM system is comparable to that of

other 3D imaging systems.(12, 30-34)

The problems with areas covered by hair (obvious in figure 1) are not very

significant for the facial application. However, there is a cranial extension for the

system which is meant to capture a 3D dataset of the whole head. In this, the hair is

expected to be a major problem for correct detection of the skull.

Prominent and less prominent areas like the nose and the edges of a not-yet-

treated cleft sometimes make it impossible to get a good 3D representation of these

important regions.

CONCLUSIONS

The 3dMDfaceTM provides a good digital representation of reality under clinical

circumstances. All occurring errors are negligible in themselves as well as in

aggregation. Further development is necessary to reduce the influence of impaired

camera vision in the cleft and nose areas. This can probably be addressed by

additional cameras with different view angles.

Of course the user interface of the camera system and the software platform for

further investigation can always be improved in matters of user friendliness and

performance.

Further investigation has to be done in matters of the influence of facial

expression on the results. A main goal might be the identification of reliable

landmarks that are not affected by facial expression, but are still clinically relevant.

11

The possibility of reproducible identification of these landmarks has to be taken into

account. As a last step the whole concept should be transferred into 4D, which

means the 3D capturing not only of a still image, but also of a moving object.

ACKNOWLEDGEMENTS

The authors would like to thank the Surgical Planning Lab, Harvard Medical

School, Boston, for hosting the planning phase of the study. We also thank Hildegard

Eschle, senior librarian of the Dental School at the University in Zurich for helping

with the literature research.

Conflict of interests

The authors declare that they have no conflict of interest.

Figure 1. Mannequin head as captured by the system

Figure 2. Operator error without zoom tool

Figure 3. Operator error with zoom tool

Figure 4. Error with re-calibration (including operator error)

Figure 5. Error through rotation of the object (including operator error)

Figure 6. Allocation of the differences between direct measurements and

distances derived out of the 3D dataset

Figure 7. Investigated error classes and results

Study No. A-P-position Vertical inclination Horizontal rotation No. of acquired datasets 1 Direct caliper measurements 15

2 15cm posterior neutral none 18 (new calibration for each)

3 neutral 10 degrees down none 17 (new calibration for each)

4 neutral neutral -5 to 5 degrees 11 (in 1 degree steps)

5 neutral neutral -30 to 30 degrees 21 (in 3 degree steps)

6 5cm posterior 5cm posterior -30 to 30 degrees 21 (in 3 degree steps)

Table 1. Protocol of data acquisition

References

1. Farkas, L. G., Bryson, W., Klotz, J. Is photogrammetry of the face

reliable? Plast Reconstr Surg 66: 346-355, 1980.

2. Hounsfield, G. N. Computerized transverse axial scanning

(tomography). 1. Description of system. Br J Radiol 46: 1016-1022, 1973.

3. Ambrose, J. Computerized transverse axial scanning (tomography). 2.

Clinical application. Br J Radiol 46: 1023-1047, 1973.

4. Ettorre, G., Weber, M., Schaaf, H., et al. Standards for digital

photography in cranio-maxillo-facial surgery - Part I: Basic views and

guidelines. J Craniomaxillofac Surg 34: 65-73, 2006.

5. Schaaf, H., Streckbein, P., Ettorre, G., et al. Standards for digital

photography in cranio-maxillo-facial surgery--Part II: Additional picture sets

and avoiding common mistakes. J Craniomaxillofac Surg 34: 444-455, 2006.

6. Niamtu, J. Image is everything: pearls and pitfalls of digital

photography and PowerPoint presentations for the cosmetic surgeon.

Dermatol Surg 30: 81-91, 2004.

7. Tanner, J. M., Weiner, J. S. The reliability of the photogrammetric

method of anthropometry, with a description of a miniature camera technique.

Am J Phys Anthropol 7: 145-186, 1949.

8. Grayson, B., Cutting, C., Bookstein, F. L., et al. The three-dimensional

cephalogram: theory, technique, and clinical application. Am J Orthod

Dentofacial Orthop 94: 327-337, 1988.

9. Al-Omari, I., Millett, D. T., Ayoub, A. F. Methods of assessment of cleft-

related facial deformity: a review. Cleft Palate Craniofac J 42: 145-156, 2005.

10. Moyers, R. E., Bookstein, F. L. The inappropriateness of conventional

cephalometrics. Am J Orthod 75: 599-617, 1979.

11. Altobelli, D. E., Kikinis, R., Mulliken, J. B., et al. Computer-assisted

three-dimensional planning in craniofacial surgery. Plast Reconstr Surg 92:

576-585; discussion 586-577, 1993.

12. Moss, J. P., Coombes, A. M., Linney, A. D., et al. Methods of three

dimensional analysis of patients with asymmetry of the face. Proc Finn Dent

Soc 87: 139-149, 1991.

13. Cutting, C. B., McCarthy, J. G., Karron, D. B. Three-dimensional input

of body surface data using a laser light scanner. Ann Plast Surg 21: 38-45,

1988.

14. McCance, A. M., Moss, J. P., Wright, W. R., et al. A three-dimensional

soft tissue analysis of 16 skeletal class III patients following bimaxillary

surgery. Br J Oral Maxillofac Surg 30: 221-232, 1992.

15. Deacon, A. T., Anthony, A. G., Bhatia, S. N., et al. Evaluation of a

CCD-based facial measurement system. Med Inform (Lond) 16: 213-228,

1991.

16. Bhatia, G., Vannier, M. W., Smith, K. E., et al. Quantification of facial

surface change using a structured light scanner. Plast Reconstr Surg 94: 768-

774, 1994.

17. Duffy, S., Noar, J. H., Evans, R. D., et al. Three-dimensional analysis

of the child cleft face. Cleft Palate Craniofac J 37: 137-144, 2000.

18. Kawano, Y. Three dimensional analysis of the face in respect of

zygomatic fractures and evaluation of the surgery with the aid of Moire

topography. J Craniomaxillofac Surg 15: 68-74, 1987.

19. Kawai, T., Natsume, N., Shibata, H., et al. Three-dimensional analysis

of facial morphology using moire stripes. Part II. Analysis of normal adults. Int

J Oral Maxillofac Surg 19: 359-362, 1990.

20. Kawai, T., Natsume, N., Shibata, H., et al. Three-dimensional analysis

of facial morphology using moire stripes. Part I. Method. Int J Oral Maxillofac

Surg 19: 356-358, 1990.

21. Kau, C. H., Richmond, S., Incrapera, A., et al. Three-dimensional

surface acquisition systems for the study of facial morphology and their

application to maxillofacial surgery. Int J Med Robot 3: 97-110, 2007.

22. Jamison, P. L., Zegura, S. L. A univariate and multivariate examination

of measurement error in anthropometry. Am J Phys Anthropol 40: 197-203,

1974.

23. Ulijaszek, S. J., Kerr, D. A. Anthropometric measurement error and the

assessment of nutritional status. Br J Nutr 82: 165-177, 1999.

24. Marks, G. C., Habicht, J. P., Mueller, W. H. Reliability, dependability,

and precision of anthropometric measurements. The Second National Health

and Nutrition Examination Survey 1976-1980. Am J Epidemiol 130: 578-587,

1989.

25. Luebbers, H. T., Messmer, P., Obwegeser, J. A., et al. Comparison of

different registration methods for surgical navigation in cranio-maxillofacial

surgery. J Craniomaxillofac Surg 36: 109-116, 2008.

26. Fitzpatrick, J. M., West, J. B. The distribution of target registration error

in rigid-body point-based registration. IEEE Trans Med Imaging 20: 917-927,

2001.

27. Marmulla, R., Muhling, J., Eggers, G., et al. [Markerless patient

registration. A new technique for image-guided surgery of the lateral base of

the skull]. HNO 53: 148-154, 2005.

28. Kau, C. H., Zhurov, A., Bibb, R., et al. The investigation of the

changing facial appearance of identical twins employing a three-dimensional

laser imaging system. Orthod Craniofac Res 8: 85-90, 2005.

29. Weinberg, S. M., Kolar, J. C. Three-dimensional surface imaging:

limitations and considerations from the anthropometric perspective. J

Craniofac Surg 16: 847-851, 2005.

30. Weinberg, S. M., Scott, N. M., Neiswanger, K., et al. Digital three-

dimensional photogrammetry: evaluation of anthropometric precision and

accuracy using a Genex 3D camera system. Cleft Palate Craniofac J 41: 507-

518, 2004.

31. Ayoub, A., Garrahy, A., Hood, C., et al. Validation of a vision-based,

three-dimensional facial imaging system. Cleft Palate Craniofac J 40: 523-

529, 2003.

32. Lowey, M. N. The development of a new method of cephalometric and

study cast mensuration with a computer controlled, video image capture

system. Part II: Study cast mensuration. Br J Orthod 20: 315-331, 1993.

33. Lowey, M. N. The development of a new method of cephalometric and

study cast mensuration with a computer controlled, video image capture

system: Part I. Br J Orthod 20: 203-214, 1993.

34. Trotman, C. A., Gross, M. M., Moffatt, K. Reliability of a three-

dimensional method for measuring facial animation: a case report. Angle

Orthod 66: 195-198, 1996.