Aligning digital holography images of tympanic membrane motion

Proceedings of Meetings on Acoustics

Volume 21, 2014 http://acousticalsociety.org/

167th Meeting of the Acoustical Society of America

Providence, Rhode Island

5 - 9 May 2014

Session 5aPP: Psychological and Physiological Acoustics

5aPP35. Aligning digital holography images of tympanic membrane motionJérémie Guignard*, Jeffrey T. Cheng, Michael E. Ravicz and John J. Rosowski

*Corresponding author's address: Eaton-Peabody Laboratory, Massachusetts Eye and Ear Infirmary, 243 Charles Street, Boston, MA02114-3096, [email protected] Modern studies of sound-induced tympanic membrane (TM) motion include stroboscopic Opto-Electronic Holography (OEH). OEH generates high-definition maps of the motion of human and animal TMs that define and quantify spatial patterns of motion. In order to compare such patternsquantitatively between repeated measurements, a common coordinate system is necessary. Differences in the relative position of the sensor and the measuredsample induce differences in magnification, shape, and location of the resulting TM image. We present a registration algorithm based on spatial features ofthe TM motion phase maps in two regimes of motion at distinct stimulus frequencies (200 Hz and 15 kHz). A log-polar representation of the motion phasemaps at 15 kHz stimulation enabled the correction of the in-plane rotation. The projective transformation was deduced and corrected on the basis of a set oflandmarks defined in the phase maps at 200 kHz stimulation. The technique presented here enables the comparison of repeated TM motion maps with aspatial resolution close to the pixel size and is thus an efficient strategy to align TM motion images. Some residual registration error still needs to beaccounted for.

Published by the Acoustical Society of America through the American Institute of Physics

Guignard et al.

© 2014 Acoustical Society of America [DOI: 10.1121/1.4886816]Received 9 Jun 2014; published 23 Jun 2014Proceedings of Meetings on Acoustics, Vol. 21, 050003 (2014) Page 1

Redistribution subject to ASA license or copyright; see http://acousticalsociety.org/content/terms. Download to IP: 188.61.185.241 On: Tue, 01 Jul 2014 15:55:41

Introduction In mammalian ears, the tympanic membrane (TM) moves in response to sound in the ear canal, but how TM motion is transmitted to the ossicles, especially at high frequencies, is not fully understood. This lack of understanding is due, in part, to a lack of detailed description of the motion of the entire TM surface. Previous work from our group used Opto-Electronic Holography (OEH), developed with the Center for Holographic Studies (CHSLT) of the Worcester Polytechnic Institute, to measure the motion of the mammalian TM in response to sound. In particular, the use of stroboscopic OEH enables measurements of steady state TM motion, with a high spatial resolution, that include the direction of motion (inward vs. outward)1. A schematic of a typical OEH setup is shown in Figure 1. The OEH technique has demonstrated the presence of modal and travelling-wave-like motions on the surface of the TM2. The influence of physiological and structural changes to the TM and middle ear on TM motion has also been investigated with OEH on human cadaver temporal bone specimens in studies of the consequences of middle-ear reconstructive surgery or prosthetics3,4. In the course of such studies, the middle ear structure is manipulated, which typically requires that the specimen be removed from the measurement setup and replaced afterward. By doing so, the relative position between the specimen and the OEH camera can be altered. In another study from our group, in which the influence of the orientation of the ear canal to the TM on TM motion is being investigated, the TM surface motion is measured with OEH in cadaveric temporal bones as an artificial ear canal is placed at varied orientations relative to the TM. In these and other manipulation studies, point-by-point comparisons of repeated measurements in a given specimen require registration (expression in a common coordinate system) of the digital holograms measured in the different conditions.

Figure 1: Simplified schema of the stroboscopic OEH experimental setup. The thick blue lines show the laser beam’s path. Thin black lines indicate connections between the components of the setup. The coordinate systems of the camera (xc, yc, zc) and of the specimen (x, y, z) are shown.

signalgenerator

computer

laser

camera

speaker

specimen

strobeswitch

optical phaseshifter

beamsplitter

interferometer

xc

zcycz

x

y

obje

ct b

eam

re!ected object beam

reference beam

Guignard et al.

Proceedings of Meetings on Acoustics, Vol. 21, 050003 (2014) Page 2 Redistribution subject to ASA license or copyright; see http://acousticalsociety.org/content/terms. Download to IP: 188.61.185.241 On: Tue, 01 Jul 2014 15:55:41

Study aims Differences in the relative position of the camera and the measured specimen induce differences between the resulting holograms that can be approximated by geometrical transformations. The aim of this study is to develop transforms to bring motion maps measured with different views of the same specimen into a common coordinate system, thereby enabling point-by-point comparisons of the motions measured in multiple conditions. Methods a) Image generation with OEH The stroboscopic OEH setup (Figure 1) has been extensively described in previous publications1,2. In summary: the laser beam is separated into object and reference beams by a beam splitter. The object beam illuminates the specimen and the reflected object beam is mixed with the reference beam in the interferometer. The resulting interference pattern is recorded with a digital camera. The reference beam is subject to a controlled optical phase shift. The specimen is excited with pure tones via the speaker, and the laser beam is pulsed with pulse durations of 2-5 % of the stimulus cycle. The TM motion in a cadaveric temporal bone preparation was measured through the transparent wall of a tube angled 45 degree to the TM annulus plane to mimic the human ear canal. The artificial ear canal was sealed to the bony part surrounding the TM with silicone ear canal impression material. The relative orientation of the TM and the artificial ear canal was varied in 6 different orientations; at each orientation OEH images were acquired at 19 stimulus frequencies ranging from 200 Hz to 15 kHz. b) Segmentation and pre-processing The motion patterns of the TM can be roughly divided into 3 regimes in different frequency ranges2: 1) Simple, at low frequencies (200-800 Hz), where most of the moving parts of the TM are in phase (Figure 2 A, C); 2) Complex, at mid-frequencies (1-10 kHz), where the motion magnitude and phase vary over the surface in more than a simple way; and 3) Ordered, at high frequencies (10-18 kHz), where a circular wave-like pattern forms around the umbo and manubrium (see Figure 2 B, D). Segmentation of the motion maps to identify the manubrium and estimate the TM boundary was done based on one motion phase image from both regimes 1 and 3. As the specimen is left in position while the stimulus frequency is changed, the specimen orientation is consistent across frequencies. A region describing the part of the TM that moves in regime 1 (simple) was segmented using the sharp edge visible in the phase plot (Figure 2C). The umbo was identified on the phase plot in regime 3 (ordered; Figure 2D), as it typically moves out of phase with the rest of the membrane. Segmentation was done with a manual seed shape that was deformed to match the edge in a process called Active Contour5.

Guignard et al.


The position of the umbo was normalized to the center of the image (uc, vc) for all motion maps by simple translation, according to the coordinates of the center of the segmented region representing the umbo.

Figure 2: Segmentation. A: magnitude of motion at 200 Hz stimulation, C: corresponding phase map. Regions outside of the segmented iso-phase region have been made transparent. B: magnitude of motion at 15 kHz stimulation, D: corresponding phase map. The umbo was segmented around most of its circumference, and arbitrarily closed on the side of the manubrium. Regions outside of the segmented umbo have been made transparent.

c) Projective geometry For the registration process, we consider the idealized situation of a planar object close to a pinhole camera. In these conditions, the geometrical transformation between the motion maps derived from OEH images from different orientations can be approximated by projective transformations. In reality, in the OEH setup, the lens system introduces aberration and blurring of the out-of focus parts of the object and the focal length is altered between repeated measurements. Such variations are assumed negligible in the present application, especially since the motion maps contain no information about these aberrations nor the 3D shape of the specimen.

A B

C D

200 Hz 15 kHzm

agni

tude

(!m

)ph

ase

(rad)

0

0.2

"

-"

Guignard et al.


The experimental setup resulted in minimal differences in global scaling between repeated measurements since the distance between the camera and the specimen and the focal length were relatively constant. For this reason, and because the in-plane translation was corrected in the pre-processing step (see section b above), we consider only the rotational components in the projective correction. Figure 3 shows the arrangement of the camera and specimen. If we consider a corresponding landmark point on the specimen in two measurements made with different orientations of the camera and specimen, P and P’ with respective coordinates (x, y, z) and (x’ ,y’, z’), their projections p(u, v) and p’(u’, v’) on the image plane I will have coordinates defined as:

€

u = f xz

,

€

v = f yz

€

u'= f x 'z'

,

€

v'= f y 'z'

where f is the focal length and the center of projection is at u, v =0. The focal length was assumed to be constant between repeated measurements. Any misalignment of the specimen will result in a transformation of the coordinate system W ->W’. The transformation between W and W’ can be defined by a rotation R and a translation T. The translation was addressed in the pre-processing step and is thus not considered here.

Figure 3: The camera and specimen arrangement. C is the camera reference frame, W is the world reference frame and I is the 2D image reference frame. 3D points in the W system have corresponding projected 2D points in the I system. The orientation of W relatively to C can be expressed as successive rotations around each of the x, y and z-axes represented here as ω, φ and κ respectively. P and P’ are the position of a given landmark point on the specimen in two measurements made with different orientations of the camera and specimen, and p and p’ their respective projection in the image reference frame.

C

IW

x

y

zu

v

xc

yc

zc

specimen

image

camera

!

"

#P P’

p p’

Guignard et al.


Thus: (x’, y’, z’)=R*(x, y, z) where R is defined as:

€

R(ω,φ,κ) = Rx (ω )Ry (φ)Rz (κ)

=

cos(φ)cos(κ) −cos(φ)sin(κ) sin(ω )sin(ω )sin(φ)cos(κ) + cos(ω)sin(κ) −sin(ω )sin(φ)sin(κ) + cos(ω )cos(κ) −sin(ω)cos(φ)−cos(ω )sin(φ)cos(κ) + sin(ω)sin(κ) cos(ω )sin(φ)sin(κ) + sin(ω )cos(κ) cos(ω)cos(φ)

&

'

( ( (

)

*

+ + +

where ω, φ and κ are rotations about the x, y and z-axes, respectively (see Figure 3). For simplicity we refer to the elements of R with their index; the relationship then becomes:

€

u'= R11u + R12v + R13 f(R31 / f )u + (R32 / f )v + R33

€

v'= R21u + R22v + R23 f(R31 / f )u + (R32 / f )v + R33

which is a projective transformation. Projective transformation estimation is typically posed as a least-square estimate of the transformation that best maps a set of corresponding landmarks between two images, and numerical methods to solve it are well documented6. d) Log-polar mapping and phase correlation The log-polar coordinate transformation of images is used for registration to correct in-plane rotation and uniform scaling between images, because they translate into simple translations in the log-polar domain7. Let (r, θ) be a polar coordinate system, where r is the radial distance from the center (uc, vc), and θ the angle relative to the horizontal (Figure 4A). Examples of phase maps expressed in (log(r), θ) are shown in Figure 4 D, E. In these cases, (uc, vc) was set at the center of the umbo. A shift along the θ axis in the polar-transformed images translates into a rotation around (uc, vc) in the Cartesian domain. Scaling is mapped into a translation by the logarithmic representation of r, as log(s ⋅ r) = log(s)+log(r), where s is the scaling factor. Consequently, a translation of one pixel is equivalent to a scaling factor s=e along a given radius. The logarithmic scaling of the r-axis has the benefit of minimizing the radial scaling differences resulting from varying orientations, as they are typically much smaller than e. Minimal differences along the r-axis allow efficient detection of the translation along the θ-axis.

Guignard et al.


Figure 4: In-plane rotation (κ) correction with the log-polar transform. A: Phase map of a given TM at 15 kHz stimulation in its reference orientation; the red cross (center of the umbo) and the dashed white line indicate the reference for the log-polar sampling pattern. B: The phase map of the same specimen with the same stimulus and a different orientation relative to the camera. C: image B after angle correction. D: image A with log-polar mapping, θ increases clockwise from the white dashed line in A. E: image B with log-polar mapping. F: image E after a circular-shift was applied along the θ-axis to correct translation with respect to image D.

The translation along θ was found with Phase Only Correlation. In short, let f1 and f2 be two images that differ only by a translation (xt, yt) such as f2(x, y)=f1(x-xt, y-yt). In the Fourier domain, the relation will be:

€

F2(w,n) = e− j 2π (wxt +nyt ) *F1(w,n) The cross-power spectrum is then defined as:

€

F1(w,n)F2*(w,n)

|F1(w,n)F2(w,n) |= e j 2π (wxt +nyt )

(where F* indicates complex conjugate of F). By the Fourier shift theorem, the inverse Fourier transform of the cross-power spectrum will be an impulse at the position equivalent to the shift between f1 and f2

7. The images were low-pass filtered before the calculation of their FFT to reduce the effect of high spatial frequency noise. e) Projective transform computation Landmarks for the perspective transformation computation were defined as follows: A line was defined in the approximate direction of the manubrium as well as its

! (d

egre

es)

X (pixels)

Y (p

ixel

s)

reference

200 400 600 800

200

400

600

800

X (pixels)

Y (p

ixel

s)

rotated

200 400 600 800

200

400

600

800

X (pixels)

Y (p

ixel

s)

corrected rota"o n

200 400 600 800

200

400

600

800

log(r)

! (d

egre

es)

200 400 600 800

4590

135180225270315360

log(r)200 400 600 800

4590

135180225270315360

log(r)200 400 600 800

4590

135180225270315360

#0.5

#0.4

#0.3

#0.2

#0.1

0

0.1

0.2

0.3

0.4

0.5

A B C

D E F

phase (cycles)

image a (reference) image b corrected image b

! (d

egre

es)

! (d

egre

es)

Guignard et al.


perpendicular passing through the center of the umbo. The correction of the in-plane rotation (κ) and the centering at the umbo ensure that the intersections of these lines with the mask and the circular pattern are collinear across the images to be registered. In cases where a portion of the membrane is masked, landmarks were selected as the intersections of the lines with local minima and maxima in the circular patterns. Results A typical set of registered images is shown in Figure 5. Upon visual inspection, the registered images have a similar orientation and shape as the reference image (Figure 4 A, G). One notable exception can be seen in Figure 5 J, where the correction of the effect of differences in out-of-plane rotation angles (φ and ω) is slightly exaggerated. The in-plane κ angles (counterclockwise) were found to be -40.5, 132.5, -160.0, 62.5, and -126.5 from images B, C, D, E, F to image A, respectively. The corresponding ratios of similar pixels in the masks after registration were 0.97, 0.96, 0.98, 0.97, and 0.97.

Images as measured Registered images

Figure 5: Typical registration result. Images A-F show the phase map at 15 kHz measured in a given specimen. The orientation of the TM relative to the camera was changed in between each measurement. Images G-L show the same 6 phase maps after registration to a common reference. The external boundary of the TM and the manubrium and umbo is outlined in white. The position of the outline is kept constant relative to the image coordinates. The phase value is coded with dark blue at -π to deep red at π.

200 400 600 800

200

400

600

800200 400 600 800

200

400

600

800

200 400 600 800

200

400

600

800200 400 600 800

200

400

600

800

200 400 600 800

200

400

600

800200 400 600 800

200

400

600

800

200 400 600 800

200

400

600

800200 400 600 800

200

400

600

800

200 400 600 800

200

400

600

800200 400 600 800

200

400

600

800

200 400 600 800

200

400

600

800200 400 600 800

200

400

600

800

A B

C D

E F

G H

I J

K L

Guignard et al.


Discussion We have presented a method to register OEH images for point-by-point comparison of repeated measurements in a given TM specimen. This strategy does not require the placement of markers on the specimen. Physical markers can give a fast and accurate set of landmarks for the registration and perspective correction but can take up a significant portion of the camera’s field of view, and it is not always practical to attach such markers on a cadaveric specimen. The angle κ (rotation about the z-axis) can be estimated from the angle found with the log-polar transform and gives a precise measurement of the difference in orientation of the specimen. The angles φ and ω can be deduced from the projective transformation matrix. Knowing the 3D angle difference between the measurements can be useful to estimate the error induced in the measurement of motion, as the OEH method is sensitive to motion along the direction of the camera axis. On a visual analysis of the registered motion maps, the number and location of the ring-like features at 15 kHz were similar between different orientations (Figure 5). As the differences in specimen orientation were related to differences in the direction of the artificial ear canal in which the sound stimulus travelled, it suggests that the effect of the ear-canal orientation on the spatial patterns of TM motion are small at this frequency. The technique presented here enables the comparison of repeated TM motion maps with a spatial resolution close to the pixel size. In the registered masks, about 95% of pixels matched the reference mask. In a case of landmark scarcity, the projective transform tends to exaggerate the correction of out-of-plane rotations. Some residual registration error thus still needs to be accounted for. Future work In its current state, the method necessitates manual inputs (e.g., seed contours for the segmentation, positioning of the reference cross for landmarks detection) and does not take lens aberration into account. A validation strategy still needs to be designed to evaluate the accuracy of the registration. Possible strategies include the comparison with a marker-based method, validation with the measure of the specimen’s shape or a comparison against a set of simulated transformed images. Acknowledgments The authors would like to thank Cosme Furlong, Ivo Dobrev and Morteza Khaleghi from the CHSLT, Worcester Polytechnic Institute, for their contribution. This work was supported by R03-DC011617 and R01-DC008642 from NIDCD, a donation from L. Mittal, and the Swiss National Science Foundation (SNSF).

Guignard et al.


References 1. Cheng JT, Aarnisalo AA, Harrington E, et al. (2010) 'Motion of the surface of the

human tympanic membrane measured with stroboscopic holography' Hear Res. 263(1-2), 66–77.

2. Cheng JT, Hamade M, Merchant SN, Rosowski JJ, Harrington E, Furlong C. (2013) 'Wave motion on the surface of the human tympanic membrane: holographic measurement and modeling analysis' J Acoust Soc Am. 133(2), 918–37.

3. Rosowski JJ, Cheng JT, Merchant SN, Harrington E, Furlong C. (2011) 'New data on the motion of the normal and reconstructed tympanic membrane' Otol Neurotol. 32(9), 1559–67.

4. Ulku CH, Cheng JT, Guignard J, Rosowski JJ. (2014) 'Comparisons of the mechanics of partial and total ossicular replacement prostheses with cartilage in a cadaveric temporal bone preparation' Acta Otolaryngol. [epub ahead of print]1–9.

5. Chan TF, Vese LA. (2001) 'Active contours without edges' IEEE Trans image Process. 10(2), 266–77.

6. Radke R, Ramadge P, Echigo T, Iisaku S. (2000) 'Efficiently estimating projective transformations' Proc. International Conference on Image Processing. Vol 1, 232–235.

7. Reddy BS, Chatterji BN. (1996) 'An FFT-based technique for translation, rotation, and scale-invariant image registration' IEEE Trans image Process. 5(8), 1266–71.

Guignard et al.


Date post:	15-Nov-2023
Category:	Documents
Upload:	independent
View:	0 times
Download:	0 times

Aligning digital holography images of tympanic membrane motion

Documents