Date post: | 07-Jan-2017 |
Category: |
Documents |
Upload: | trinhthuan |
View: | 214 times |
Download: | 0 times |
3D Reconstruction of Tomographic Images Applied to Largely Spaced Slices
Agma J. M. Traina, Afonso H. M. A. Prado, Josiane M. Bueno Computer Science Department - Mathematics Institute at São Carlos of
University of São PauloPo.Box 668
13560-970 - São Carlos, SP - BRAZILe-mail:{agma| josiane}@icmsc.sc.usp.br
[email protected]: +55-16-274-9136
Fax: +55-16-274-9150
Abstract This paper presents a full reconstruction process of magnetic resonance images. The first step
is to bring the acquired data from the frequency domain, using a Fast Fourier Transform
algorithm. A Tomographic Image Interpolation is then used to transform a sequence of
tomographic slices in an isotropic volume data set, a process also called 3D Reconstruction. This
work describes an automatic method whose interpolation stage is based on a previous matching
stage using Delaunay Triangulation. The reconstruction approach uses an extrapolation
procedure that permits appropriate treatment of the boundaries of the object under analysis.
Keywords: medical imaging, matching interpolation, tridimensional reconstruction, Delaunaytriangulation..
1. IntroductionThe capability to visualize the human internal organs in a noninvasive way, with regard to
anatomical and functional aspects, has intrigued humanity for centuries. This was achieved in
radiology by the invention of computerized tomography (CT) and more recently by magnetic
resonance imaging (MRI).
MRI and CT usually deliver cross-sectional images of the human body. The sampling is
done in a set of coplanar slices with adjustable distance and thickness. If sequences of adjacent
images are put together and piled up, the result will be a 3D volume representation of the human
body part under analysis. This 3D volume is a powerful tool to aid medical diagnosis and surgery,
because it can be visualized and manipulated as needed without any risk to the real body [1].
Additionlly, it must also guarantee the precision of the reconstructed volume and maintain the real
proportion for the three dimensions of the object. That is, the reconstructed volume must be an
isotropic 3D volume.
To reconstruct an isotropic 3D volume, the interslice distance should be close to the
interpixel distance. The distance between consecutive slices is usually larger than the distance
between consecutive pixels within a slice, so an interpolation must be done to obtain the pixels
between the slices, that allow the generation of an isotropic volume. However, two distinct steps
are necessary to reconstruct a 3D volume: the 2D reconstruction of each slice and the 3D volume
reconstruction.
2. Methodology of development
The first step in magnetic resonance imaging (MRI) reconstruction is the 2D reconstruction,
which is done through a Fast Fourier Transform (FFT) [2], producing slices in gray-level, typically
matrices, from the frequency domain data. We have been using data acquired from the MR
Tomograph of the Sao Carlos Institute of Physics, University of Sao Paulo, where a low cost and
low field MR Tomograph is being developed [3]. The second step is 3D Volume Reconstruction
using these reconstructed 2D slices [4].
The slices acquired from MRI are usually highly spaced due to time and equipment
restrictions. The sampling rate in 2D dimensions (actually at 256x256 pixels) generates an inter-
pixel distance lower than the distance between the slices, resulting in a nonisotropic volume. We
are seeking an isotropic volume, which means the distances in all dimensions must be equal,
because the volume really produces virtual data from the analysed data, and manipulation is more
precise. This illustrates the need to interpolate between 2D slices to obtain new slices that lead to
an isotropic 3D volume.
The raw data typically are sequences of 256x256x32 or 256x256x64 with 256 gray levels.
This work focuses on images with distance between image planes ranging from 6mm to 12mm.
3. Tridimensional reconstruction
Usually 3D reconstruction is made by trilinear interpolation, that is, a simple linear
interpolation in X, Y and Z directions. This method is fast, but if the distance between the images
is great, that method will produce discrepancies in the interpolated images. These discrepancies
occur because the tissues can shrink, expand, bend or even disappear between two consecutive
slices. When that happens, it is difficult to correct this problem and a staircase effect disturbs the
image. This paper describes a new method producing better results.
The image processing field defines as image registration or image matching the problem
to determine the correspondence between points in two images [5]. Such correspondences, usually
deal with the issue of how to determine the positions of the same object points in two images of
the same scene. Still, in tomographic images, having the same point in consecutive images is not
possible. Thus, it is necessary to perform these correspondences based on the shape or boundaries
of the tissues and structures present inside the images.
The method presented in this paper uses two phases to generate the images between each
pair of original slices: the first phase is the matching phase, based on the correspondence of image
points. In this phase, the relationship between tissues in consecutive image slices is maintained. The
matching method used here takes advantage of the fact that consecutive tomographic slices are
already approximately aligned, and establishes correspondence between points in the images
automatically [6].
The second phase of the method uses the correspondences obtained in the first phase and
performs an interpolation, limited by the disparities set from the matching phase. The matching
phase must define associations between points of same tissue, which have similar characteristics.
The border tissue points have high gradient values and can be used to do the matching. Therefore,
we divided the matching phase into two stages - matching based on high gradient points (points on
the image’s edges) and matching based on low gradient points (points in homogeneous regions of
the image).
Figure 1 - Defining the search window on the target image, where the methodsearches the high gradient point B, corresponding to the high gradient point A inreference image. Note that point A’ is not elected as the matching point, because it isnot a high gradient point.
3.1 Matching based on high gradient points
As two consecutive images have only few geometric differences between them, we can
imagine the matching process as a way to define the local deformation to be applied to the first
image of the sequence and to transform it into the second image of the slice sequence.
The points of the tissue contour are characteristic points in the images and they usually have
high gradient values. Using a search process, we can establish the correspondence between points
in two adjacent images, based on the differences of gradient, density level and geometric position.
This is the basis of the first stage of the matching phase. In this stage of the matching phase we are
using the proposal made by Goshtasby, Turner and Ackerman in [6].
The correspondence between two points in two adjacent images can be done using vertical
(by axis y) and horizontal (by axis x) disparity, that is, a high gradient point must match with
another high gradient point in the next image. We will call the first image of the sequence reference
image and the next image of the sequence target image. However, the second high gradient point
can be in a different geometric position from the first high gradient point, so searching for this point
in the region near that geometric position is necessary. We have called this search region, that
establishes where the higher point gradient in the target image will be searched, the search window.
The centre of the search window is the same geometric position of point A, point A’. The
dimensions of the search window suggest the horizontal and vertical disparity. Figure 1 illustrates
this idea.The correspondence line starting at point A and ending at point B, will guide the
generation of the interpolated points in the intermediate images, produced between the reference
image and the target image.
To determine the correspondences of points in the first stage of matching, we have used the
intensities and gradients of the points, as well as the geometric position (disparity) between them.
To accomplish this, we have used a vector cost function. The function is defined matching a point
(x,y) in the target image to a point (x’, y’) in the reference image by:
(1)
where I(x, y), M(x, y), �(x, y) are, respectively, the intensity, the gradient magnitude, and the
gradient direction of the point (x, y) in the target image; and I’(x’, y’), M’(x’, y’), �’(x’, y’) are,
respectively, the intensity, the gradient magnitude and the gradient direction of the point (x’, y’) in
the reference image. The distance corresponds to the Euclidian distance between (x, y) and (x’, y’).
Values w1, w2, w3 and w4 can adjust the contribution of each of these values. In this work we are
using the same contribution for each wi = 1.
Inside the search window, more than one high gradient point can be found. Now, assuming
the search window contains n points with gradient magnitudes above a given threshold value,
{(xi.,yi): i=1,...,n}. The vector cost function is used to find out the dissimilarity of each one of the
n points in the search window getting the least cost magnitude, which is elected as the
corresponding point to (x’, y’). Thus, the best matching point is (xm.,ym), where C(xm., ym, x’, y’) =
min {C(xi., yi, x, y) : I=1,...,n}. But, two adjacent tomographic images show different cross sections
of a 3D object, so the corresponding points may have different gradients and intensities, leading to
a possible mismatch. Therefore, those possible mismatches must be corrected. Due to two
consecutive image slices are nearly aligned, and the geometric differences of the structures are little
and few, we can use the continuity criterion to correct the mismatches. That is, close points in the
reference image should map to close points in the target image. If two neighboring points in the
reference image map to largely-separated points in the target image, the match performed on one
or on both point must have be wrong.
Based on the continuity criterion, we are using the median filter 3x3 for each high gradient
point (x,y) in the reference image, that have known correspondences. In other words, to correct
possible mistakes in the matching stage, we will use as disparities for these points the median of
the disparities of n � 9 high gradient points of the 3x3 neighbourhood, that have their disparities
set before. This approach only detects and corrects mismatches if they are not the majority in a 3x3
neighbourhood.
Nevertheless, if the mismatches occur collectively, the median filter cannot correct the
erroneous points. If there are mismatches, we can correct the mistakes by repeating the process
described above in a reverse direction, that is, switching reference and target images and carrying
out the matching again. Thus, only the points that match consistently in both ways are marked as
real high gradient level points and only these will be used in the second phase of the interpolation
method. In figure 1, point B found in the target image by this procedure is the best match for point
A in the reference image.
3.2 Matching based on low gradient points
The second stage of the matching phase aims to define the correspondences for the
remaining points of the image, that is, all the points that have low gradient magnitude and the
points with high gradient magnitudes, where it was not possible to establish a correspondence in
the first stage of the matching phase.
Points with low gradient magnitudes belong to homogeneous regions of the images. For low
gradient points we cannot use the same criterion used for points with high gradient magnitudes,
because such points have similar density levels and gradients. Using the correspondences
established in the first stage of the matching, an interpolation can be done based on those
relationships. Therefore, the adopted approach is:
- First, build a Delaunay triangulation [7], where the vertexes are the high gradient points
with correspondences established in the first phase of matching. The triangulation
describes the convex hull for that set of points, and will be used as a guide to perform the
interpolation (the second phase of the matching interpolation method).
- Second, define disparities for points in homogeneous regions, inside the convex hull (the
majority are points in the analysis structure), through a procedure of linear interpolation of
vertex triangle disparities where the point is in.
- Third, define disparities for points in homogeneous regions, outside the convex hull of
the Delaunay triangulation, where we use a procedure of extrapolation.
In human images obtained from tomography, the points outside the convex hull are usually
background points, so applying an expensive method for them is not necessary. The extrapolation
consists of defining a belt limiting the convex hull, where the disparities become progressively
linear. This special treatment for the boundary is needed, because usually the edges of the objects
under analysis are diffuse.
A triangulation is well suited for interpolation if its triangles are nearly equilateral. How
Delaunay triangulation produces this type of triangle [8], we chose it. After building the
triangulation an interpolation inside triangles is made by linear combination of disparities of the
triangles’ vertexes. The aim of the extrapolation procedure is to produce a smooth transition
between the disparities of the tissue edges in the images and the disparities of background images.
The extrapolation procedure can also smooth out the staircase effect around the edges of objects
in the reconstructed volume.
As Delaunay triangulation produces the convex hull of the main structure in the image, the
triangles will assist the interpolation process. Nevertheless, it is necessary to use an efficient
algorithm to implement this triangulation . The algorithm used here was proposed in [9] and
presents a good data structure to be used in the second stage of the matching phase.
Upon conclusion of the matching phase (the first phase of the matching method), a linear
interpolation can be done using all the correspondences established.
3.3 Interpolation procedure
The interpolation procedure produces the new interpolated slices between reference image
and target image, assisted by the correspondence line, as figure 1 shows. The task here is the
definition of the disparities of the points in homogeneous areas of the image. These points are inside
the convex hull, defined by the triangulation (the points with high-gradient magnitudes are the
vertexes of the triangles).
For each point in homogeneous regions of the image, the problem is to detect the triangle
that the point is in. After that, we can calculate the disparities of that point doing the linear
interpolation of the vertexes of the triangle detected. If a point is outside the triangulation, it will
have its disparities set by extrapolation procedure.
3.4 Extrapolation procedure
The extrapolation procedure consists of establishing correspondences (to define disparities)
for all points in the reference image that are outside the image’s convex hull. The points that
surround the convex hull, limit the object under analysis and the background of the image. Such
points should be processed differently from the background points of the image. That happens
because such points have different characteristics from the other points of homogeneous regions
inside the triangulation.
Special attention must be paid to these points outside of the triangulation. If these points
are far away from the convex hull, they are certainly background points, and we can attribute
disparities zero to them. For points in the image’s background, time and computational power
should not be wasted in the interpolation process, because these points are not relevant to the
analysis. A simple linear interpolation on them is sufficient to produce the intermediate image
slices.
However, if the points inside the extrapolation belt are near the convex hull, such points
can pertain to the object structure or its boundary. So, these points need have their disparities
conveniently set. For the points inside the extrapolation belt and outside the convex hull, the
disparities are set started with the latest value defined by the vertexes of the triangulation and
Figure 2- A Delaunay triangulation and the extrapolation belt. Points in region A havethe nearest points of the triangulation as the edges of triangles. Points in region Bhave the nearest points of the triangulation as the vertexes of triangles.
progress linearly towards zero as they approach the external limits of the extrapolation belt. In this
way, there are no abrupt disparity changes, and the continuity criterion is maintained. Figure 2
presents a triangulation and the extrapolation belt associated with it.
To carry out the extrapolation procedure, for each point inside the extrapolation belt we
need to detect the nearest point of the triangulation. Either the nearest point pertains to an edge of
the triangulation (points in region A of the extrapolation belt) or the nearest point pertains to a
vertex of the triangulation (points in region B of the extrapolation belt).
The width of the belt is a crucial question here. As we stated in the first stage of the
matching process (the matching based on high gradient points), we are using a search window to
establish the correspondences between two consecutive images. The ray of the search window is
a parameter of the matching process. Thus, the larger disparity allowed in the first stage of the
matching process is equal to the ray of the search window. Consequently, the largest disparity for
the points inside the triangulation is also equal to the ray of the search window. Therefore, we
defined the width of the extrapolation belt as at least the same width of the search window. For the
images reconstructed and shown in this paper we used the width equal to 2* r (where r is the ray
of the search window). The ray of the search window was set with the value as double the interslice
Figure 3 - The procedure of extrapolation is performed inside the extrapolation beltsurrounding the convex hull established by Delaunay triangulation. This approachsmooths the staircase effect of the structure edges. a) the structure present in thereference image expands to the target image. b) the structure present in the referenceimage diminishes to the target image
distance estimated in voxels.
Figure 3 shows the idea of the extrapolation procedure, performed inside the belt that
surrounds the convex hull. The slices are seen in an axial view. Clearly we can visualize that a
consequence of that approach is to smooth the staircase effect in the structure edges of the object
under analysis.
3.5 The 3D reconstruction algorithm
The algorithm presented here must be applied between each pair of consecutive image slice
sources generated by tomography, leading to the reconstruction of the desired isotropic volume. The
input data for the algorithm are: the source images (reference image and target image), the
parameters for the matching phase, the interslice distance and the spatial resolution of the images.
The complete algorithm is shown in the next two boxes.
Algorithm Match 1
1. Matching phase (establish correspondences):
1.1 - Matching based on high gradient points (see next algorithm - Match 2)
1.2 - Matching based on low gradient points (and high gradient points without
established correspondences):
1.2.1 - Build Delaunay triangulation, where the vertexes are the points
in the reference image with correspondences established.
1.2.2 - Define correspondences for points inside the triangulation of the
reference image (Interpolation procedure).
1.2.3 - Define correspondences for points outside the triangulation in
the reference image (Extrapolation procedure).
2. Interpolation phase (linear interpolation guided by correspondences established):
2.1 - Define number of intermediate slices and their position (to generate
isotropic volume) using the interslice distance and spatial resolution of
the original slices.
2.2 - Find the density values of the intermediate slices, doing a linear
interpolation from source images (reference and target), guided by the
correspondence line. If more than one correspondence line passes
through a pixel, its density level will be the average of the contributions.
2.3 - If there are pixels in an interpolated image slice (i.e. voxels in the isotropic
volume) without density level set:
2.3.1 - Set the density level of these voxels using trilinear interpolation.
Figure 4 - A block diagram of the matching interpolation method proposed.
Algorithm Match 2 - Matching based on high gradient points
1. For each high gradient point (x’, y’) in the reference image search the corresponding
(x,y) point in the target image:
1.1 - Define the search window in the target image centred at the same
geometric position of (x’, y’).
1.2 - For each high gradient point (x, y) in the target image inside the search
window, calculate the vector cost function C(x, y, x’, y’) that sets the
dissimilarity between the points (x,y) and (x’, y’).
1.3 - Find the point (x,y) in the target image with the least value of the vector
cost function. Let this point be the corresponding point (x’, y’) to the
point (x, y) in the reference image.
1.4 - Find the disparities (horizontal and vertical) for the correspondence
established.
2. Detect and correct any incorrect correspondences established, using the median
filter on the disparities (horizontal and vertical) just established.
3. Repeat steps 1 and 2 in the reverse direction (from the target image to the reference
image). Keep only the correspondences set in both directions.
A global view of the method proposed in this paper can be seen in figure 4
Figure 5 shows two interpolated slices reconstructed from a sequence of 22 axial images,
each 8 mm separate from the other. These images were obtained by MRI in the Sao Carlos Institute
Figure 5 (a) - Sagittal slice generatedby linear interpolation.
(b) - Sagittal slice generated bymatching interpolation.
(b) - Sagittal slice generated by matchinginterpolation.
Figure 6 (a) - Coronal slice generated bylinear interpolation.
of Physics, University of Sao Paulo - Brazil, and from they it was generatd two isotropic volumes,
the first volume generated by linear interpolation and the second volume generated by matching
interpolation . Figure 5(a) shows an estimated sagittal slice from the first volume (linear
interpolation). Figure 5(b) shows the same slice from the second volume (matching interpolation).
Figure 6 shows two coronal slices also from the same MR reconstructed volumes of the
7 (a) - Original axial CT image. Theinterslice distance of the slice sequence is 3mm.
Figure 7(c) - Image generated by matchinginterpolation, corresponding to originalimage (a).Figure 7(b) - Image generated by linear
interpolation, corresponding to originalimage (a).
figure 5.
4. Results
The matching interpolation method
proposed herein was evaluated using qualitative
and quantitative analysis. Direct comparisons
between interpolated images generated by
matching interpolation and trilinear interpolation,
yielding the difference image can be seen in figure
7. We have used RM brain images, CT images of
a dry skull and phantom images to perform these
analyses.
It is necessary to state that some analysis
were performed only comparing two
interpolated images: by matching interpolation
and trilinear interpolation. In those cases we were
not worried about producing an isotropic volume
Figure 7(d) - Image showing thedifferences between the CT original imagein part (a) and an interpolated imagegenerated by linear interpolation (b).
Figure 7(e) - Image showing the differencesbetween the CT original image in part (a)and the image generated by matchinginterpolated (c).
Figure 8(a) - CT sagittal slice in a volumereconstructed by linear interpolation.
Figure 8(b) - CT sagittal slice in a volumereconstructed by matching interpolation.
(see figure 8). Thus, we have compared the original image with the image generated by matching
interpolation and the image generated by trilinear interpolation.
Beyond a simple visual analysis, the qualitative analysis was also made interpolating
intermediary slices between serial slices. Using sequences of three images, the method works with
(2)
(3)
the first and the third image. The interpolated slice is compared with the second slice, which is in
fact an original collected slice. We carried out the same process selecting images skipping the slices
by a predefined number as one, two or three and comparing the interpolated images with the
original slices. This approach was also used to decide the least interslice distance where the
matching interpolation method proposed herein is better than the linear interpolation (regarding
accuracy and computational time).
In order to perform a quantitative analysis, we used two measurements: the root mean
square (RMS) error and the number of points where a wrong point is selected (number of points
that disagree - NPD), as equation 2 shows.
where O(i,j) is a point (i,j) in the original image and E(i,j) is a point in the interpolated image. The
RMS error determines the medium error on pixels between the collected and the interpolated
images, as equation 3 shows, where I and J are the number of rows and columns of the images, O
is the original image and E is the
interpolated image.
Figures 9 and 10 contain
images of phantoms used to illustrate
the method. Using the phantom of figure 9 with 30 original slices, we estimated slices 5, 15 and 25,
which were compared with the original slices. The diameter of the phantom is 50 pixels. Table 1
presents a quantitative analysis of this method. Table 1, figures 5 to 8 and 10, illustrate the
advantage of matching interpolation over trilinear interpolation.
Figure 9 - Phantom of a bending structure.
Figure 10 (a) - Phantom slice in yzplane, produced by linear interpolation.
Figure 10 (b) - Phantom slice in yz plane,produced by matching interpolation.
The results show
that in sets of slices with
distances between 2 or 3
mm linear interpolation is
advantageous, i.e., the
image depreciation is very
low and the computational
cost is not so high.
However, in sets of slices
with greater space between them, the results of matching interpolation are better.
5. Conclusions
This approach is a modification of that introduced in [6]. The matching phase on low
gradient points described herein uses the Delaunay triangulation, as well as interpolation and
extrapolation algorithms. This approach allows for reconstruction of a more accurate 3D volume,
Estimatedimage
referenceandtargetimage
Quantitative Analysis
RMS error NPD
linear match. % linear match. %
5 4 and 6 0.602 0.099 84 612 72 88.2
15 14 and 16 1.004 0.1365 86 1020 101 90.1
25 24 and 26 1.908 0.3118 84 1918 315 83.6
Average 1.171 0.1825 84 1183 163 86.2
Table 1 - Quantitative analysis performed on phantom data.
where the staircase effect is diminished.
The results obtained so far also show a better approximation of the volume in an isotropic
representation, and are therefore encouraging new refinements [10]. This method needs more power
from the computer equipment because matching interpolation may take ten times longer than linear
interpolation. A large part of the computational cost of this method occurs in the matching phase.
However, the 3D reconstruction is performed only once and the volume can be manipulated as
needed. Furthermore, this method produces more precise and accurate 3D volumes, thus making
the tomographic system increasingly useful to produce accurate diagnoses and a real aid in surgery.
6. References[1] J. K. Udupa and G.T. Herman, “3D Medical Imaging”, CRC Press, 1991.
[2] E.O. Brigham, "The Fast Fourier Transform", Prentice Hall, 1988.
[3] E.L. Vidoto, H. Panepucci, A. Tannús, M.J. Martins, “A New Type of Head Coil for MRI in
Ultra Low Magnetic Field”, in Socity of Magnetic Resonance, Second Meeting, Aug. 6-12
1994, S. Francisco, USA, vol III, pp. 1101-1109.
[4] J.M.Bueno, A.J.M. Traina, P.E.Cruvinel, “Reconstructing Tomographic Images Using 2D
Sections produced by Fast Fourier Transform” (in portuguese) Proceedings of XVI Iberian
Latin American Conference on Computacional Methods for Engineering, Nov. 1995, pp.
1605-1613.
[5] R.C. Gonzalez and P. Wintz, "Digital Image Processing” (Second Edition), Addison-Weslay,
1987.
[6] A. Goshtasby, D.A. Turner, “Matching of Tomographic Slices for Interpolation”, IEEE Trans.
on Medical Imaging, Vol. 11, No. 4, Dez. 1992, pp. 507-516.
[7] F. Aurenhammer, “Voronoi Diagrams - A Survey of a Fundamental Geometric Data Structure”,
ACM Comp. Surveys, vol. 23, set. 1991, pp. 345-405.
[8] R. Sibson, “Locally Equiangular Triangulations”, The Computer Journal, 21, 1977, pp. 243-
245.
[9] C.L. Lawson, “Software for C1 Surface Interpolation”, Mathematical Software III, J.R. Rice ed.,
Academic Press, New York, 1977, pp.161-194.
[10] A.H.A. Prado and A.J.M. Traina, “Interpolation of Tomographic Images using Delaunay
Triangulation” (in portuguese), Proceedings of the IX Brazilian Conference on Computer
Graphics and Image Processing, Oct. 1996.
AcknowledgmentsThe authors are grateful to CAPES and CNPq (Brazilian Funding Agencies ) for financial
support to develop this project.