IEEE TRANSACTIONS ON INFORMATION …visualized feedback (over 25 frames/s) during acquisition. II....

Copyright (c) 2010 IEEE. Personal use is permitted. For any other purposes, Permission must be obtained from the IEEE by emailing [email protected].

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication.

IEEE TRANSACTIONS ON INFORMATION TECHNOLOGY IN BIOMEDICINE 1

Real-time Visualized Freehand 3D UltrasoundReconstruction Based on GPU

Yakang Dai, Jie Tian*, IEEE Fellow, Di Dong, Guorui Yan andHairong Zheng, IEEE Member

Abstract—Visualized freehand 3D ultrasound reconstructionoffers to image incremental reconstruction during acquisition andguide users to scan interactively for high quality volumes. Weoriginally used the Graphics Processing Unit (GPU) to develop avisualized reconstruction algorithm that achieves real-time level.Each newly acquired image was transferred to the memory of theGPU and inserted into the reconstruction volume on the GPU.The partially reconstructed volume was then rendered using GPUbased incremental ray-casting. After visualized reconstruction,hole-filling was performed on the GPU to fill remaining emptyvoxels in the reconstruction volume. We examine the real-timenature of the algorithm using in vitro and in vivo data sets.The algorithm can image incremental reconstruction at speed of26∼58 frames/s and complete 3D imaging in the acquisition timefor the conventional freehand 3D ultrasound.

Index Terms—Freehand 3D ultrasound, Volume reconstruc-tion, Volume rendering, GPU, CUDA

I. INTRODUCTION

FREEHAND 3D ultrasound [1] uses a 2D ultrasoundmachine and a position sensor to reconstruct a volume.The position sensor (e.g. magnetic sensor [2]–[4], opticalsensor [5]) is attached to the probe of the 2D ultrasoundmachine to track the position and orientation of the B-scanimage, and a set of B-scan images with the relative positionsand orientations are acquired and reconstructed to build upthe volume. Compared to direct volumetric imaging with a 3Dprobe [6], [7], the freehand technique is cheaper and can obtainlarger volumes. Therefore the freehand technique is often usedin practical applications including fetal examination [8], [9],neurosurgery [10], [11] and so on.

Conventional freehand technique separates the acquisition,reconstruction and visualization steps [12], which leads to thefollowing problems: (a) the imaging result is unknown untilthe acquisition, reconstruction and visualization are completed(typically one minute [11], [13]), which disturbs the interactive

This work was supported in part by NBRPC (2006CB705700,2011CB707700), CAS HTP, CAS KIP (KSCX2-YW-R-262, KGCX2-YW-129), NSFC (81042002, 81071218, 30873462, 60910006) in China.

Y.-K. Dai, J. Tian, D. Dong and G.-R Yan are with the Medical ImageProcessing Group, Institute of Automation, Chinese Academy of Sciences,Beijing 100190, China.

J. Tian is also with the Life Science Center, Xidian University, Xian, Shanxi710071, China.

H.-R. Zheng is with Paul·C·Lauterbur Research Center for BiomedicalImaging, Institute of Biomedical and Health Engineering, Shenzhen Institutesof Advanced Technology, Chinese Academy of Sciences, Shenzhen 518067,China.

*Corresponding author: Jie Tian. Tel: +86-10-82628760. Fax: +86-10-62527995. E-mail: [email protected]. Website: http://www.mitk.net.

nature of the ultrasound examination [12]; (b) acquisitionfeedback is not provided during scanning, consequently the ac-quisition quality is highly dependent on scanning experience,which may result in low-quality imaging even re-scanning.

One method for addressing the problems is volume re-construction and visualization during acquisition (visualizedreconstruction):

1. Perform incremental volume reconstruction withthe newly acquired image (i.e., insert the image into thevolume).

2. Perform incremental volume visualization (i.e., vi-sualize the partially reconstructed volume), then repeatfrom step 1.

The method allows us to see which regions of the volumehave been reconstructed and where (e.g. have gaps) needfurther reconstruction while scanning, so that we can scaninteractively to obtain a high quality volume.

Many publications reported visualized reconstruction algo-rithms. Ohbuchi et al. [14], [15] used pixel 3D kernel inter-polation for incremental volume reconstruction and modifiedray-casting [16], [17] for incremental volume visualization.However, the algorithm [14], [15] speed was not more than1 frame/s on a workstation. Edwards et al. [18] used areplacement-value pixel distribution method for incrementalvolume reconstruction and a shear-warp MIP method for incre-mental volume rendering. Implemented with an imaging boardbased on the Texas Instruments TMS320C80 multimedia videoprocessor, the visualized reconstruction algorithm [18] couldoperate at 12.5 frames/s. Rather than performing incrementalreconstruction as each B-scan image arrived, Welch et al. [19]collected a fixed number of images and then inserted theimages into the volume. For incremental volume visualization,they [19] simultaneously displayed cross-sections throughthe volume and a volume-rendered perspective view. Theiralgorithm [19] could update the volume and render a newview at 15 frames/s on a Silicon Graphics 320 workstation.Gobbi and Peters [20] implemented a visualized reconstructionalgorithm with five parallel threads on a 2 CPU 933 MHzPentium III workstation. The algorithm [20] could performincremental reconstruction at maximum 30 frames/s (pixelnearest neighbor), and display three orthogonal slice viewsthrough the volume (without volume rendering) at 5 frames/s.Dai et al. [21] performed incremental rendering (ray-casting)according to the increment of the reconstruction ratio. Thevisualized reconstruction [21] speed was 12.5 frames/s on a3.0 GHz Pentium IV PC.

However, the visualized reconstruction is computationally

Copyright (c) 2010 IEEE. Personal use of this material is permitted.




expensive and current algorithms can not reach real-time level(i.e., the incremental reconstruction and rendering can notachieve the B-scan image acquisition rate that is typically25 or 30 frames/s [15], [20], [21]). Non-real-time visualizedreconstruction results in two problems: (a) we can not getreal-time visualized feedback to guide scanning; (b) the 3Dimaging will take long time, which may risk motion (e.g. res-piration) artifacts in the reconstructed volume. The problemscould highly reduce the speed and quality of the 3D imaging.

The contribution of this work is to achieve real-time vi-sualized reconstruction and address the above problems. Thepaper primarily describes a real-time visualized reconstruc-tion algorithm that implements all expensive computations,including the incremental volume reconstruction, incrementalvolume rendering and hole-filling on the off-the-shelf GraphicsProcessing Unit (GPU). The algorithm can provide real-timevisualized feedback (over 25 frames/s) during acquisition.

II. MATERIALS AND METHODSA. Hardware and Software

A Windows-PC, with an Intel Core2 1.86 GHz CPU and aNVIDIA GeForce 8800 GT GPU, was used for incrementalvolume reconstruction, incremental volume rendering and soon. The real-time visualized reconstruction algorithm wasimplemented in C++ and NVIDIA Compute Unified DeviceArchitecture (CUDA) [22]. The incremental volume renderingmodule was developed based on the volume rendering frame-work of a customized Medical Imaging Toolkit (MITK) [23].The core of each module in the algorithm was implementedin CUDA and run on the GPU.

For describing the algorithm, we present a brief overviewof the execution model of the CUDA. A GPU is regarded asa separate device that operates as a coprocessor to the hostcomputer. A C function defined in accordance with the CUDAis called a kernel , which, when invoked by a CUDA programon the host, is executed on the device by a 1D or 2D grid madeup of thread blocks. The blocks (1D, 2D or 3D) are distributedto multiprocessors in the device. The threads within a blockexecute the kernel in parallel on one multiprocessor. After allblocks accomplish the execution, the kernel is terminated.

B. General Flowchart

Fig. 1 shows the general flowchart of the real-time visual-ized reconstruction algorithm that consisted of three stages:preparation, real-time visualized reconstruction and post-processing. In the first stage, all necessary arrays and vari-ables were prepared for volume reconstruction and rendering.The key arrays were built in the device memory, including:• Ir: a 3D voxel value array for incremental reconstruction

and rendering.• Wt: a 3D voxel weight array for volume reconstruction.• Hf : a 3D voxel value array for hole-filling.• Bs: a 2D pixel value array for the B-scan image.• Cl: a 2D color (R, G, B) array for the projection image

of the volume.• So: a linear opacity array for the scalar-opacity transfer

function.

• Sc: a linear color (R, G, B) array for the scalar-colortransfer function.

The Hf, Bs, So and Sc were bound to textures, the Ir was boundto a texture when it was used for volume rendering, and theWt was bound to a texture when it was used for hole-filling.In the second stage, for each newly acquired image and itstransformation matrix from the position sensor, the incremen-tal volume reconstruction and rendering were performed onthe device. After the visualized reconstruction, there may beremaining unfilled voxels in the volume. Therefore in thethird stage, an additional hole-filling operation was performedto fill empty voxels. Finally the reconstructed volume couldbe transferred to the host computer and stored on the harddisk. In the following, we introduce the implementation detailsof the incremental volume reconstruction, incremental volumerendering and hole-filling.

C. Incremental Volume Reconstruction

Firstly, the transformation matrix from the B-scan coordi-nate system to the reconstruction volume coordinate systemCTP (used for transforming the B-scan image to the recon-struction volume) was figured out by [24]

CTP = CTT · T TR · RTP (1)on the host, where P is the coordinate system attached toeach B-scan image, R and T (subscript or superscript) arethe coordinate systems of the receiver and transmitter ofthe position sensor respectively, C is the coordinate systemof the reconstruction volume, RTP , T TR, CTT and CTPcan be written as a uniform format JTI which denotes thetransformation matrix from the coordinate system I to thecoordinate system J . The RTP and CTT can be predeterminedby spatial calibration [25] and semiautomatic definition [26]respectively. Secondly, the 2D image and CTP were trans-ferred to the device memory (the image was stored in the 2Dpixel value array Bs, see Section II-B). Finally, an incrementalreconstruction kernel was called to insert the image into thevolume.

The incremental reconstruction kernel was executed on thedevice by a 2D incremental reconstruction grid to insertthe image in parallel. Neighboring pixels in the image maycontribute to the same voxel in the reconstruction volume. Ifeach thread processes a pixel, multiple threads may accessthe same voxel in parallel, which may cause parallel insertionerrors. To avoid the errors, each thread in the block processed4 × 4 pixels (the size was selected by experiment) in order.Assume the size of each block in the grid was M × N (weused 16 × 16 to achieve fast speed), and the size of theimage was W × H , then the number of blocks in the gridwas (W/4M) × (H/4N), and the image could be insertedparallelly by (W/4) × (H/4) threads. For each pixel (i, j)of the B-scan image with the value Bs(i, j), firstly the pixellocation in the reconstruction volume was calculated by

CX = CTP · P X (2)where P X and CX are the coordinates of the pixel in P (theB-scan coordinate system) and C (the reconstruction volume




Fig. 1. The general flowchart of the real-time visualized reconstruction algorithm. Ir, Wt, Hf, Bs, Cl, So and Sc (see Section II-B) were the key arrays preparedin device memory for volume reconstruction and rendering. Each image was inserted into the reconstruction volume (incremental volume reconstruction) andthe partially reconstructed volume was then rendered (incremental volume rendering). Empty voxels in the reconstruction volume were filled (hole-filling)after visualized reconstruction. The incremental volume reconstruction, incremental volume rendering and hole-filling were all performed on the device.

coordinate system) respectively. Then the pixel was distributedinto the volume using pixel 3D kernel interpolation [1]. The3D kernel we used was a 2× 2× 2 cube [18], [20] (the sizewas used as a tradeoff between interpolation quality and speed)and the pixel contributed to eight neighboring voxels. For eachneighboring voxel (m,n, l), the value Ir(m,n, l) and weightWt(m,n, l) were updated by

sum = Ir(m,n, l) ·Wt(m,n, l) + Bs(i, j) · invDWt(m,n, l) = Wt(m,n, l) + invDIr(m,n, l) = sum/Wt(m,n, l)

(3)

where invD is the inverse distance between the voxel (m,n, l)and the pixel (i, j).

D. Incremental Volume Rendering

We used ray-casting to render the volume. The ray-castingreferred to four coordinate systems, including the recon-struction volume coordinate system C (see Fig. 2(a)), worldcoordinate system, view (also camera) coordinate system, anddisplay image coordinate system D (see Fig. 2(b)). The recon-struction volume and camera were set in the world coordinatesystem. The camera faced the reconstruction volume, and usedorthogonal (also parallel) projection to set a view volume (seeFig. 2(b)) to surround the reconstruction volume. The displayimage, whose size was the same as the display window size,corresponded to the near plane of the view volume.

After inserting the newly acquired B-scan image into thereconstruction volume, some voxels in the reconstructionvolume were updated. Projecting the sub-volume enclosingthe updated voxels onto the display image, we could geta sub-image in the projection image of the volume (seeFig. 2(b)). Actually only the sub-image need to be updated,therefore we merely performed ray-casting for the pixels inthe sub-image, and kept other pixels in the projection imageunchanged. The method can highly accelerate the incrementalvolume rendering if the sub-image is small. Following is the

implementation flow of the incremental volume rendering.Firstly, the position and size of the sub-image in the projectionimage were figured out on the host. Secondly, the positionand size were transferred to the device memory. Finally, aray-casting kernel was called to update the sub-image.

The position and size of the sub-image were figured outby two steps: sub-volume computation and sub-image compu-tation (see Fig. 2). In the sub-volume computation, firstlywe computed the coordinates of the four vertices of the newlyacquired image in the reconstruction volume coordinate systemby equation (2). Secondly, with the minimal and maximal x,y and z elements of the four coordinates, we determined acompact cube (eight vertices) enclosing the B-scan image.Finally, we performed clamping operations to ensure the sub-volume was in the reconstruction volume. In the sub-imagecomputation, firstly we computed the projection coordinates(x and y elements of the DX) of the eight vertices of thesub-volume in the display image by

DX = DTC · CX (4)where DTC is the transformation matrix from the reconstruc-tion volume coordinate system C to the display image coordi-nate system D, CX and DX are the coordinates of the vertexin the C and D respectively. Secondly, with the minimal andmaximal x and y elements of the eight coordinates in D, wedetermined a compact rectangle (four vertices) enclosing thepixels required to be updated. Finally, we performed clampingoperations to ensure the sub-image was in the projectionimage (the position and size of the projection image of thereconstruction volume in the display image were precomputedin the preparation stage).

The incremental ray-casting kernel was executed on thedevice by a 2D incremental ray-casting grid to update thesub-image in parallel. Each pixel in the sub-image was pro-cessed by a thread (the block sizes for the translation sequenceand fan sequence are 2 × 32 and 16 × 16 respectively, seesection III. The sizes were selected by experiment to achieve




x

y

z

Reconstruction

Volume

C

Sub-volume

B-scan

V1

V2 V3

V4

V5

V7

V8

(a) Sub-volume computation (b) Sub-image computation

Proje

ction

View

volume

x

y

z

Display image

V1

V2

V3

V4

V7

Sub-volume

Projection image

Sub-image

Fig. 2. Sub-volume and sub-image computations. (a) The coordinates of the four vertices of the B-scan image in the reconstruction volume coordinatesystem C were computed to determine a compact sub-volume. (b) The projection coordinates of the eight vertices of the sub-volume in the display imagewere computed to determine a compact sub-image.

fast speed). Each thread cast a ray from the associated pixel(i, j) along the z axis of the display image coordinate systemthrough the reconstruction volume and performed front-to-back sampling and recursive compositing [27] to compute thecolor of the pixel. Assume S and E are the start and endintersections between the ray and the reconstruction volume,CXS denotes the coordinates of S in the reconstructionvolume coordinate system C, r is the normal of the ray in Cand dSE is the distance between the S and E in C. Then thecomputation details of the pixel color Cl(i, j) are as follows:

d = 1;CXSample = CXS + r;while d < dSE and α < 0.98 do

Find the nearest voxel (m,n, l) around CXSample;v = Ir(m,n, l);if So(v) > 0 then

c = Sc(v) · So(v) · (1− α) + c;α = So(v) · (1− α) + α;

endd = d + 1;CXSample = CXSample + r;

endif α > 0 then

Cl(i, j) = c/α;end

where CXSample denotes the coordinates of the sample in thereconstruction volume coordinate system C, Ir(m,n, l) is thevoxel value, Sc(v) and So(v) are the color and opacity of thevoxel value v respectively (see Section II-B for the definitionsof Ir, Sc, So and Cl), c and α are the accumulated color andaccumulated opacity respectively.

E. Hole-filling

After visualized reconstruction, the values of the 3D voxelvalue array (Ir) for incremental reconstruction and ren-dering were copied into the 3D voxel value array (Hf )for hole-filling. In the hole-filling, all slices of the recon-structed volume were traversed in order. For each slice, a

hole-filling kernel was called to fill empty voxels in theslice. The hole-filling kernel was executed on the deviceby a 2D hole-filling grid. Each voxel in the slice wasprocessed by a thread (the block size we used is 16 ×16). Each thread detected the associated voxel (m,n, l),and filled the voxel with the average of the neighboringnonzero voxels if the voxel was empty (Wt(m,n, l) == 0):

if Wt(m,n, l) == 0 thensum = 0;number = 0;foreach voxel (i, j, k) in the neighborhood do

if Hf(i, j, k) > 0 thensum = sum + Hf(i, j, k);number = number + 1;

endendif number > 0 then

Ir(m,n, l) = sum/number;end

end

where Wt is the 3D voxel weight array for volume reconstruc-tion (see Section II-B and equation (3)) and the neighborhoodsize we used is 3 × 3 × 3 (B-scan images were denselyacquired and 3D kernel interpolation was used for incrementalreconstruction, therefore few empty voxels were remainingbetween B-scan images after visualized reconstruction and theneighborhood size was cost-effective for the hole-filling).

III. RESULTS

Two data sets are used to evaluate the incremental volumereconstruction, incremental volume rendering and hole-filling.One data set is 1000 B-scan images from an in vitro phantom(plastic elephant) [26] by translation scan, and the otheris 135 B-scan images from an in vivo human liver [28]by fan scan. The images are visualizedly reconstructed tovolumes with the described algorithm. As shown in Fig. 1,the time spent in the preparation (T0), incremental volumereconstruction (T2-T1), incremental volume rendering (T3-T2)and hole-filling (T5-T4) are recorded.




(a) Pose of C (b) Pose of V

C Cx

yy

z

B-scanProbe

x

y

Volume

Viewx

y

z

x

y

z

Volume

C

Centerxc

yc

zc

AxAy

Az

Scan trajectory

Fig. 3. Setting of reconstruction volume coordinate system C and view coordinate system V for translation sequence. xcyczc is the center coordinate systemof the reconstruction volume (see (b)), the origin of V is on the zc axis, the z axis of V is opposite to the zc axis, and the x axis of V is parallel to the xcaxis. The setting achieves an appropriate view direction for interactive scanning (see (a)) and small sub-image sizes for fast incremental rendering.

100 200 300 400 500

Fig. 4. Visualized reconstruction for translation sequence. The incremental reconstruction is imaged at 58 frames/s. A set of incremental rendering results(after 100, 200, 300, 400 and 500 images are inserted into the volume) are shown.

A. Evaluation with Translation Sequence

The in vitro data set is collected using image guidedacquisition, and the B-scans are approximately parallel to thezoy plane in the reconstruction volume coordinate system C(see Fig. 3(a)). The B-scan image size and volume size are552× 274 and 256× 256× 256 respectively, and the displayimage size is 512 × 512. As shown in Fig. 3(b), xcyczc isthe center coordinate system of the reconstruction volume,the origin of the view (also camera) coordinate system V (seeSection II-D) is on the zc axis, the z axis of V is oppositeto the zc axis, and the x axis of V is parallel to the xc axis.The above setting enables us to see gaps between translationB-scans, which can well guide translation scan, so we namethe setting as translation scan reference setting.

Table I shows the time spent in each main step of the algo-rithm under the reference setting. The speed of the preparation,incremental reconstruction (90 frames/s) and hole-filling areprimarily dependent on the B-scan image size and volumesize (in inverse proportion), while the speed of the incrementalrendering mainly lies on the volume size and sub-image size(in inverse proportion as well). Under the reference setting,the sub-image size (average 13 × 215) is small and a fewthreads are needed (the GPU we used can run 112 threadsconcurrently), which enables the incremental rendering toreach 166 frames/s. The incremental volume reconstructionand rendering speed (58 frames/s) is much faster than the

real-time level (25 frames/s). Fig. 4 shows the incrementalvolume rendering results after 100, 200, 300, 400 and 500images are inserted into the volume.

On the basis of the reference setting, we have the volumeonly rotate about the xc (yc or zc) axis by an angle Ax (Ayor Az) (see Fig. 3(b)), and perform visualized reconstructionunder the new setting. Fig. 5(a) illustrates speed of the incre-mental rendering at different Ax (Ay and Az are 0), Ay (Axand Az are 0), or Az (Ax and Ay are 0). As the incrementalvolume reconstruction takes 11 ms (see Table I), to achievereal-time visualized reconstruction, the incremental volumerendering must be completed in 29 ms. For different Ax orAz , the sub-image size is small, and the incremental volumerendering can be accomplished in 6∼11 ms (see Fig. 5(a)),making the incremental reconstruction and rendering achieve45∼58 frames/s. For different Ay , the incremental volumerendering can finish in 7∼23 ms, which makes the incrementalreconstruction and rendering reach 29∼55 frames/s.

B. Evaluation with Fan Sequence

The in vivo data set is from the Medical Imaging Group,University of Cambridge [28]. As shown in Fig. 6, the x axisof the reconstruction volume coordinate system C correspondsto the scan trajectory of the B-scans, the y axis of C faces theoverall y (longitudinal) axis of the B-scans, and the z axis ofC faces the overall x (lateral) axis of the B-scans. The B-scan




TABLE ITIME SPENT IN MAIN STEPS OF THE ALGORITHM

Time (ms) Preparation Incremental Reconstruction Incremental Rendering Hole-fillingTranslation sequence 21 11 6 15

Fan sequence 22 21 5 46

0

5

10

15

20

15˚ 30˚ 45˚ 60˚ 75˚ 90˚

Ax Ay Az

0

5

10

15

20

25

15˚ 30˚ 45˚ 60˚ 75˚ 90˚

Ax Ay Az

(a) Translation sequence (b) Fan sequence

T23 (ms) T23 (ms)

Fig. 5. Incremental volume rendering speed at different rotation angles for translation sequence (a) and fan sequence (b). The sub-image size for differentAx is small, therefore the incremental rendering speed is very fast and stable. The sub-image size grows when Ay increases, so the incremental renderingspeed rises until Ay reaches around 90◦. The sub-image size grows until Az reaches around 45◦ and decreases until Az reaches around 90◦, thus theincremental rendering speed firstly rises and then gradually tapers off. Speed differences between the translation sequence and fan sequence are caused byvolume contents, block size and thread scheduling managed by CUDA.

(a) Left oblique view

z

y

xC x

y

z

x

y

z

P

x

y

(b) Front view (c) Right oblique view

Volume

xy

zV

Fig. 6. Setting of the reconstruction volume coordinate system C and view coordinate system V for fan sequence. The origin of V is on the zc axis of thecenter coordinate system (xcyczc, see Fig. 3(b)) of the reconstruction volume, the z axis of V is opposite to the z axis of C, and the x axis of V is parallelto the x axis of C. The setting achieves an appropriate view direction for interactive scanning and small sub-image sizes for fast incremental rendering.

image size and volume size are 480×413 and 256×256×256respectively, and the display image size is 512×512. Similarly,we set the origin of the view coordinate system V on the zcaxis, the z axis of V opposite to the zc axis, and the x axisof V parallel to the xc axis (see Fig. 6(a) and Fig. 3(b)). Theabove setting enables us to see gaps between fan B-scans, sosimilarly we call the setting as fan scan reference setting.

The time spent in each main step of the algorithm underthe reference setting is shown in Table I. The sub-image sizeis average 34 × 238, enabling the incremental rendering toreach 200 frames/s. The incremental volume reconstructionand rendering can reach 38 frames/s, which is faster than the

real-time level (25 frames/s). Fig. 7 shows the incrementalvolume rendering results after 27, 54, 81, 108 and 135 imagesare inserted into the volume.

Similarly, based on the reference setting, we have thevolume only rotate about the xc (yc or zc) axis by an angle Ax(Ay or Az), and perform visualized reconstruction under thenew setting. Speed of the incremental rendering at differentAx, Ay , or Az is illustrated in Fig. 5(b). For different Ax orAz , similarly the sub-image size is small, and the incrementalrendering can be completed in 4∼9 ms (see Fig. 5(b)), makingthe incremental reconstruction (takes 21 ms, see Table I)and rendering achieve 33∼40 frames/s. For different Ay , the




27 54 81 108 135

Fig. 7. Visualized reconstruction for fan sequence. The incremental reconstruction is imaged at 38 frames/s. A set of incremental rendering results (after 27,54, 81, 108 and 135 images are inserted into the volume) are shown.

incremental rendering can finish in 7∼17 ms, which makesthe incremental reconstruction and rendering reach 26∼35frames/s.

IV. CONCLUSIONS AND FUTURE PERSPECTIVE

We have presented the implementation of a real-time visual-ized reconstruction algorithm and evaluated the algorithm withan in vitro data set based on translation scan and an in vivodata set based on fan scan. The algorithm uses the powerful butcheap GPU to implement the time-consuming reconstructionand rendering computations, which highly accelerates thevisualized reconstruction. The evaluations demonstrate theincremental volume reconstruction and rendering speed canexceed the B-scan image acquisition rate (25 frames/s). Usingthe algorithm, we can not only get real-time visualized feed-back on the acquisition and reconstruction during scanning,but also complete the 3D imaging within the data acquisitiontime for the conventional freehand 3D ultrasound.

For high quality imaging, we need see gaps between B-scansduring scanning, so as to well guide the scan and acquisition.Therefore, the x axis of the view coordinate system V shouldcorrespond to the scan trajectory, and the view direction (zaxis of V ) should correspond to the overall x or y axis ofthe B-scans. The setting makes the sub-images of the B-scans in the display image small, which can greatly speed upthe incremental volume rendering. Our evaluations show theincremental volume reconstruction and rendering can exceed30 frames/s by using the setting. In addition to translationscan and fan scan, the setting benefits rotation scan as well.Actually, the evaluations at different Az for the translation andfan sequences resemble the evaluation for a rotation sequenceunder the setting.

The technically developed but promising real-time visual-ized reconstruction algorithm is potentially useful for prac-tical applications, especially freehand 3D ultrasound guidedsurgery, where the speed and quality of the 3D imaging arecritical. We are developing a real-time freehand system basedon the algorithm and will try to apply the system to clinicalapplications in the future.

REFERENCES

[1] O. V. Solberg, F. Lindseth, H. Torp, R. E. Blake, and T. A. N.Hernes, “Freehand 3d ultrasound reconstruction algorithms–a review,”Ultrasound in Medicine & Biology, vol. 33, no. 7, 2007.

[2] D. F. Leotta, P. R. Detmer, and R. W. Martin, “Performance of aminiature magnetic position sensor for three-dimensional ultrasoundimaging,” Ultrasound in medicine & biology, vol. 23, no. 4, pp. 597–609, 1997.

[3] S. Berg, H. Torp, D. Martens, E. Steen, S. Samstad, I. Høivik, andB. Olstad, “Dynamic three-dimensional freehand echocardiography us-ing raw digital ultrasound data,” Ultrasound in medicine & biology,vol. 25, no. 5, pp. 745–753, 1999.

[4] S. Meairs, J. Beyer, and M. Hennerici, “Reconstruction and visualizationof irregularly sampled three- and four-dimensional ultrasound datafor cerebrovascular applications,” Ultrasound in medicine & biology,vol. 26, no. 2, pp. 263–272, 2000.

[5] J. W. Trobaugh, D. J. Trobaugh, and W. D. Richard, “Three-dimensionalimaging with stereotactic ultrasonography,” Computerized MedicalImaging and Graphics, vol. 18, pp. 315–323, 1994.

[6] O. T. von Ramm, S. W. Smith, and H. G. Pavy Jr, “High-speedultrasound volumetric imaging systemłpart ii: Parallel processing andimage display,” IEEE Transactions on Ultrasonics, Ferroelectrics, andFrequency Control, vol. 38, no. 2, pp. 109–115, 1991.

[7] E. Merz, F. Bahlmann, G. Weber, and D. Macchiella, “Three-dimensional ultrasonography in prenatal diagnosis,” Journal of PerinatalMedicine, vol. 23, no. 3, pp. 213–222, 1995.

[8] E. Esh-Broder, F. B. Ushakov, T. Imbar, and S. Yagel, “Application offree-hand three-dimensional echocardiography in the evaluation of fetalcardiac ejection fraction: a preliminary study,” Ultrasound in Obstetricsand Gynecology, vol. 23, no. 6, pp. 546–551, 2004.

[9] U. Herberg, H. Goldberg, and J. Breuer, “Three-and four-dimensionalfreehand fetal echocardiography: a feasibility study using a hand-held Doppler probe for cardiac gating,” Ultrasound in Obstetrics andGynecology, vol. 25, no. 4, pp. 362–371, 2005.

[10] G. Unsgaard, S. Ommedal, T. Muller, A. Gronningsaeter, and T. A. N.Hernes, “Neuronavigation by intraoperative three-dimensional ultra-sound: Initial experience during brain tumor resection,” Neurosurgery,vol. 50, pp. 804–812, 2002.

[11] G. Unsgaard, O. M. Rygh, T. Selbekk, T. B. Muller, F. Kolstad,F. Lindseth, and T. A. N. Hernes, “Intra-operative 3d ultrasound inneurosurgery,” Acta Neurochir, vol. 148, pp. 235–253, 2006.

[12] R. W. Prager, A. Gee, and L. Berman, “Stradx: real-time acquisitionand visualisation of freehand 3d ultrasound,” Medical Image Analysis,vol. 3, no. 2, pp. 129–140, 1998.

[13] O. M. Rygh, T. A. Nagelhus Hernes, F. Lindseth, T. Selbekk,T. Brostrup Müller, and G. Unsgaard, “Intraoperative navigated 3-dimensional ultrasound angiography in tumor surgery,” Surgical neu-rology, vol. 66, no. 6, pp. 581–592, 2006.

[14] R. Ohbuchi and H. Fuchs, “Incremental volume rendering algorithm forinteractive 3d ultrasound imaging,” in Proceedings of the 12 Interna-tional Conference on Information Processing in Medical Imaging, 1991,pp. 486–500.

[15] R. Ohbuchi, D. Chen, and H. Fuchs, “Incremental volume reconstructionand rendering for 3d ultrasound imaging,” in In SPIE Proceedings onVisualization in Biomedical Computing, vol. 1808, 1992, pp. 312–323.

[16] M. Levoy, “Display of surfaces from volume data,” IEEE Transactionon Computer Graphics and Applications, vol. 8, no. 3, pp. 29–37, 1988.

[17] R. Drebin, L. Carpenter, and P. Hanrahan, “Volume rendering,” Com-puter Graphics, vol. 22, no. 4, pp. 65–74, 1988.

[18] W. S. Edwards, C. Deforge, and Y. Kim, “Interactive three-dimensionalultrasound using a programmable multimedia processor,” InternationalJournal of Imaging Systems and Technology, vol. 9, no. 6, pp. 442–454,1998.




[19] J. N. Welch, J. A. Johnson, M. R. Bax, R. Badr, and R. Shahidi, “Areal-time freehand 3-d ultrasound system for image-guided surgery,” inProceedings of IEEE Ultrasound Symposium, San Juan, Puerto Rico:IEEE, vol. 2, 2000, pp. 1061–1064.

[20] D. G. Gobbi and T. M. Peters, “Interactive intra-operative 3d ultra-sound reconstruction and visualization,” in Proceedings of MedicalImage Computing and Computer Assisted Intervention (MICCAI), Tokyo,Japan: Springer, vol. 2489, 2002, pp. 156–163.

[21] Y. Dai, J. Tian, J. Xue, and J. Liu, “A qualitative and quantitative inter-action technique for freehand 3d ultrasound imaging,” in Proceedings ofIEEE Engineering in Medicine and Biology Society (EMBS), New York,USA: IEEE, 2006, pp. 2750–2753.

[22] C. NVIDIA, NVIDIA compute unified device architecture programmingguide version 2.0, NVIDIA Corporation, Santa Clara, CA, USA, 2008.[Online]. Available: http://www.nvidia.com/object/cuda develop.html

[23] J. Tian, J. Xue, Y. Dai, J. Chen, and J. Zheng, “A novel software platformfor medical image processing and analyzing,” IEEE Transactions onInformation Technology in Biomedicine, vol. 12, no. 6, 2008.

[24] R. Rohling, A. Gee, and L. Berman, “Three-dimensional spatial com-pounding of ultrasound images,” Medical Image Analysis, vol. 1, no. 3,pp. 177–193, 1997.

[25] L. Mercier, T. Langø F. Lindseth, and L. D. Collins, “A review ofcalibration techniques for freehand 3-d ultrasound systems,” Ultrasoundin Medicine & Biology, vol. 31, no. 4, 2005.

[26] Y. Dai, J. Tian, and J. Zheng, “Semiautomatic determination of thereconstruction volume for real-time freehand 3d ultrasound reconstruc-tion,” in SPIE Medical Imaging 2009, February 8, Lake Buena Vista,Florida, USA, vol. 7265, 2009, p. 72651E.

[27] A. Kaufman and K. Mueller, Overview of Volume Rendering. Chapterfor the Visualization Handbook, eds. C. Johnson and C. Hansen,Academic Press, 2005.

[28] Hepatic. Medical Imaging Group, University of Cambridge. [Online].Available: http://mi.eng.cam.ac.uk/∼rwp/stradwin/

Date post:	29-Jul-2020
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times

IEEE TRANSACTIONS ON INFORMATION …visualized feedback (over 25 frames/s) during acquisition. II....

Documents