Pergamon Comput. & Graphics, Vol. 20, No. 6, pp. 801-811, 1996

Copyright Q 1996 Elsevier Science. Ltd Printed in Great Britain. All rights reserved

0097-8493/96 $15.00+0.00

Pn: SOO97-8493(96)00053-2 Medical Visualization

TeleInViVoTM: TOWARDS COLLABORATIVE VOLUME VISUALIZATION ENVIRONMENTS

JOHN COLEMAN’+, AMMO GOETTSCH’, ANDRE1 SAVCHENKO’, HENDRIK KOLLMANN’, KU1 WANG’, EDWIN KLEMENT” and PETER BONO’

‘Fraunhofer Center for Research in Computer Graphics, Inc., 167 Angel1 Street, Providence, RI 02906, U.S.A.

‘Fraunhofer Institute for Computer Graphics, WilhelminenstraBe 7, D-64283, Darmstadt, Germany e-mail: jcoleman@crcg.edu

Abstract--Converging technologies in the areas of telecommunications, volume visualization, and computer hardware and peripherals have made possible in recent years the development of new tools for collaboration that extend the reach of health care professionals and other consumers of volumetric data around the world. We describe a recent development at the Center for Research in Computer Graphics in Providence, RI, that makes a significant contribution to this area. TeleInViVoTM is an application that supports collaborative volumetric data visualization and exploration. It is an extension and partial reworking of InViVoTM, a volume visualization application developed at the Fraunhofer IGD, in Darmstadt, Germany. InViVo, which is largely focused around the medical community and with an emphasis on diagnostic ultrasound, has been augmented with new modes of interaction, an intuitive collaboration mechanism, and an architectural modification to support future developments in this area. Copyright 0 1996 Elsevier Science Ltd

1. INTRODUCTION

InViVoTM (Interactive Visualizer for Volume data) is a volume visualization tool developed at the Fraun- hofer IGD in Darmstadt, Germany, in their Multi- media Systems and Image Processing department [l-3]. It is characterized by its high performance, versatility, and friendly, compact interface. The name, InViVo, has actually come to represent a modular family of volume visualization products each of which adds new functionality on top of a common core of capabilities. There are currently six major variations on the theme (Fig. 1):

l InViVo-Vis-the most basic version comes with a powerful set of tools to render surfaces (e.g. gradient shading), peer through volumes (e.g. maximum intensity projection), and slice through data [4].

l InViVo-MD--which includes DICOM3 compat- ibility and segmentation.

l InViVo-3DUS-which adds special ultrasound filters.

l InViVo-Plus-which adds advanced tools for segmentation, measurement, and diagnostic sup- port.

l InViVo-Scan-using 3DUS or Plus as the base, it enables conventional 2-D ultrasound systems to acquire volume data by tracking the sensor’s 3-D position and orientation.

+ Author for correspondence.

l InViVo-Tel-the topic of this paper, based on Plus or Scan, it provides telecommunications along with new interaction modes and collabora- tive features.

Throughout this paper the term, InViVo, will be used as a catch-all label for any of the first four members of the family; and the term, TeleInViVo, is interchangeable with InViVo-Tele. In the remainder of this article, we will motivate our recent develop- ments, frame some of the important issues, and elaborate on the form our solution has taken.

2. THE PROBLEM

2.1. Telemedicine There are a number of interrelated problems

facing the health care industry today. While cost containment is often cited as a major concern, clearly high quality of service, efficiency, customer satisfac- tion and even continuing medical education are worthy goals no less challenging to achieve. While a final judgement has not yet been made on the contribution of telemedicine to overall cost reduc- tion, it is increasingly clear that much can be gained in all other respects when telemedicine is properly integrated into a comprehensive health care environ- ment. In particular, the area of teleradiology has expanded considerably over the past decade and is generally considered one of telemedicine’s early success stories. There are several long-standing problems the technology addresses that makes it quite compelling.

801

802 J. Coleman et al.

InViVo-3DUS / I

l------ !

InViVo-RAD I 1

___-- y InViVo-VIS /

/

Fig. 1. The InViVoTM family architecture.

Expert opinion is available any time and at any place. It is said, “Somewhere in the world the sun is always shining.” Indeed, by leaping time zones, a Primary Care Physician has access at all times to whatever degree of support is necessary to provide high quality service. The corollary to the previous point is that critically-short personnel resources (i.e. experts) are now available both locally and globally. Even in those situations where special procedures require flying in an expert, the use of two-way communications implies that the expert’s skill set is continuously accessible to all. Workloads can be more effectively balanced by transferring cases from an overburdened facility to a less encumbered one. The corollary to the above point is that there is an improved turn-around time for service. This is especially important where casualty care is con- cerned. The ability to respond during the “first golden hour of trauma” can result in significant savings in human lives. There is less of a need for “circuit riders”. These are medical professionals who are called upon to make out-of-town excursions, often into rural areas, to provide much needed service. Telecon- sulting and telediagnosis add a whole new dimen- sion to the solution. The obverse of the preceding observation is that there is also less isoiation implied for those medical professionals who live in, and service, rural communities. The opportunities for con- tinuing medical education through access to on- line databases, expertise, telementoring and tutoring, etc., lessens the impact of working in remote regions, where one can easily lose track of the mainstream and new developments in one’s field.

2.2. From 2-D to 3-D Along with all the new developments also come

new challenges, new ideals. There is much more to teleradiology than simply adding telecommunica- tions to existing applications. Them i:; also ibc fortuitous convergence of a number of supportive technologies including new methods for data acquisi- tion, visualization, interaction, image processing and collaboration. Each development, while significant 1~ itself, when combined with complementary technol ogies, opens new vistas of possibilities not~previouslv possible or only minimally so. A good cxamplr ‘;;i’ this is in the area of volume visualization where :L number of efforts are currently underway thki combine, with different degrees of sophistication collaborative features [4, 61.

Running somewhat counter to a lay person‘? expectations, volume visualization of medical data has had a chequered history, viewed with some ambivalence by practitioners in the tieid. This seems strange at first since so many imaging modalities t’ti.7. MRI, MRA, CT, U/S, PET, SPECT) readily lend themselves to volumetric representations. Nonethe- less, to oversimplify, radiologists have all but ignored it, while surgeons and radiation oncologists love: it! On reflection, there is good reason behind the cold shoulder response of many radiologists, since most of them have spent years perfecting the ability tp construct in their heads 3-D relationships based <;n the sets of 2-D slices that have traditionally been available to them. It is likely this will change for-the next generation of MDs who will grow up in a ver.y different technological environment than heir

teachers. Yet the question arises: given what would seem like obvious benefits, how does one best convey these benefits to the current generation of radiolo- gists in a manner that clearly augments their current skill set rather than attempting to replace it?

The solution may lie, in part, in a popular notion circulating in various medical circles, especially where tetemedicine is concerned. This is the idea of “scanning the data and not the patient”. The idea here is to establish a kind of “information equiva- lence” between the patient and the representation of that patient in digital form, so that there is little to no loss in examining the patient through the data representation, compared with what would be lost during a direct in-person examination. Given that it is faster and cheaper to ship information around than to ship people (patients and doctors), voiu- metric representations become compelling as :I substitute for the live patient. The point for the radiologist is that the volume representation can be “resliced” from any direction one might desire. This avoids the need to acquire a whole new set of images (requiring the patient to make another trip to the hospital). if for some reason the original set leaves 3-D relationships ambiguous.

Additional motivation lies in mitigating the impact of the skill level of the radiology technologist. acquiring the data. For example, a good ultrasound

TeleInViVo? towards collaborative volume visualization environments 803

technologist can require as much as 2-3 years to perfect this skill set. Such qualified personnel are not always available, especially in emergency situations. The ability to rescan or reslice the data allows for the use of less skilled technicians who need only to be able to scan the appropriate region.

Recent developments in the area of 3-D Ultra- sound are also quite compelling. Interventional ultrasound, as in the case of biopsies, can benefit significantly from volume visualization. In addition, unlike its radiation-based counterparts such as MRI and CT, ultrasound devices can be reduced to back- pack size, making them portable. Augmented with telecommunications capabilities, this makes possible a degree of health care previously unachievable in regions that are difficult-to-serve, expensive-to-serve, or under-served. It is this area in particular, of portable ultrasonography, that has motivated recent developments at the Fraunhofer Center for Research in Computer Graphics in the area of volume visualization.

3. TeleInViVo

3 1. Technical approach The search for information equivalence in tele-

radiology requires the integration of many technol- ogies, each designed to allow for the free communication of some form of information nor- mally present to collaborators when engaged in their activity using traditional modes. There needs to be a kind of telepresence that either, in effect, transports the patient to the doctor/radiologist or vice versa.

Figure 2 shows the kinds of communication paths present in a typical scenario. To be effective a teleradiology application must capture and transmit the information content represented by the kinds of interaction shown here. There are issues concerning the data interaction/acquisition model, tools for measuring and processing the data, and meta- information contained in the collaborative process itself. All of these must be managed subject to the constraints imposed by computer hardware and transmission channels.

i i Radiology Primary Care

Technologist Physician

Protocol Consultation and Review

Subspecialist

Fig. 2. Communication pathways in radiology.

Our efforts in the development of TeleInViVo primarily address the channels of communication represented in the bottom half of Fig. 2. Radiologists must communicate their data acquisition needs to the radiology technologists, their perceptions and concerns to the subspecialists, and their conclusions to the primary care physicians. This can be formalized by specifying a number of requirements designed to achieve a level of information equiva- lence with conventional methods. The several kinds of requirements encountered can be broken into media, collaboration, and telecommunication con- straints. The media requirements for this interaction include text, voice, image, video, and volume data. Collaboration requirements include image annota- tion, voice annotation, shared whiteboard, telepoint- ing, store-and-forward, and shared views of the data. Telecommunication constraints involve whether the communication is synchronous, asynchronous, or isochronous, and specifying the desired level of security, bandwidth, latency, reliability, accessibility, network speed, and supported protocols. Table 1 shows some of the possible communication require- ments associated with the various tasks.

Any attempt to provide a general solution incorporating all of the preceding elements would require time and resources exceeding those currently available to us. In addition, there are many useful problems to be solved that span only a subset of these requirements. Thus we have embarked upon an evolutionary path that will allow us to capitalize on both the integration of existing or new technologies and our own advancements in selected areas of specialization. To support this effort an appropriate architecture must be defined that readily accommo- dates future growth. Since TeleInViVo is an exten- sion of Fraunhofer’s existing volume visualization application, InViVo, we have taken a close look at the restructuring implied, with a view to minimizing impact on performance-one of InViVo’s strengths. An object-oriented approach is being taken, with well-defined APIs between subsystems.

3.2. Implementation In our first iteration on a design solution we have

focused on enhancing the current state-of-the-art by bringing to bear our strengths in volume visualiza- tion, computer-supported cooperative work (CSCW), and telecommunications. That is, our focus has been on the data interaction model, telepointing, shared views, store-and-forward, and common pro- tocols. To round out our solution, we have incorporated off-the-shelf technologies such as SGI’s InPersonTM software that provides video, voice, images, and a shared whiteboard. We have also experimented with freeware tools such as vie, vat and nv. Fortunately the compactness of TeleInViVo’s user interface minimizes the sense of clutter normally associated with running several applications simulta- neously. Currently TeleInViVo is compatible with a

804 I. Coleman et al.

Table 1. Telaxmmumication requiranents

Diageostics consult. & Review Capture

security Bandwidtb Latency Reliability Aaxssgbility STAT support Network speed

PfOtOCdS

encryption, password guaranteed baudwidtb

low: prcfetcb, proxy server bigll: packet loss < 5% biglxdowntime ~2%

priority bandwidth Intranet/F+livate:

VPN. >Tl, FDDI, FwEtbemqSONET

DICOM. TCP, II’, ATM

enclyption, password guaranteedbandwidtb

medium medium medium

priority bandwidth Public: ISDN, Modem,

Internet

DICOM. MBONE, TCP, Ip

enmlption. password gDantntcedbaadwidtll

bigb (dcpell~ on liiodality)

g-i priority bfuldwidtb

point-tepoint

DICOM

wide variety of UNIX systems and a port to “Free” button. The connection can be reestablished Windows-NT is underway. by selecting “Take Control”.

Figure 3(a) gives a concise depiction of the major architectural components of TeleInViVo. The UI (user interface), State, and Kernel are the basic InViVo components with some TeleInViVo-specific erlhmaments. comln~catioll between subsytelns is mediated by two processes, labeled III-Manager and Kernel-Manager. This effectively decouples the subsystems so that each can be separately developed without having to rewrite the other. These managers, in turn, communicate with the TI-Session Manager, whose role is to mediate the interaction between session participants. Figure 3(b) illustrates the concept of a distributed, shared state. Whenever one participant makes inaememal changes at the local UI, the rcsuIts are analysed to determine the scope of the change. Depending on the nature of the change, incremental updates are sent to the remote process. On occasion, changes in state are sufIiciently large that an incremental update is invent. On such occasions the entire state is transferred. Computations necessary to synchronize states are performed locally.

3.2.1. Teleco mmunications. The current design supports point-to-point communication over TCP/ IF. To accommodate low-baudwidth channels (e.g. modems, ISDN), a “status-passing” mechanism has been implemented. This is sutIicient for most purposes. However, since TeleInViVo supports volume transfer, we have also added a variation of the LempehZiv compression and “packetiziq” of datafortmnsmi&on.Allofthisisaccompliahedina machill~indepelldent maimer. Data integrity is maintained through normal TCP mechanisms. Only hmitedco&derationhasbeengiventotheissueof latency, insofar that telepomting can accomodate dilkent remking speeds and will always show the most up-to-date state possible, even if the link is congested. Issues collcerning secllrity have not been addressed in the current implementation.

TeleInViVo aIs0 provides for the e&cient transfer of data. In one possible scenario, a technician carrying a portable 3-D ultrasound device in the field scans and transmits a “thumb” (i.e. a low resolution copy of a volrrme) of that data to a remotely-located radiologist pig. 4(h)]. Using the thumb as a guide, the radiologist specifies a subvolme, using TeleInViVo’s clipping tools, to be re-sent at high resolution [Fis. 4(c)]. Oxwe de&d, the volume is “pulled” across at the click of a button. Compression and decompression is handled automatically both at the transmitting and receiving ends. The final volume has a high resolution subvolume of interest and a low resolu- tion surrounding volume Fig. 4(d)]. The total time for tmnsmission is generally a small fraction of what it would have required to ship the entire vohnne at high resohition. This feature makes TeleInViVo particuhuly appealing in emergency situations where there is only modest bandwidth avaibble and time is crucial- In nonemergency situations, it is just as likely that the dam has been transmitted hours, perhaps days, in advance. In such a case, participants load the spec&d dataset prior to initiating the TeleInVrVo~ session using file transfer tools.

3.2.2. Graphics. While information equivalence is a worthy goal in itself, it does not capture the ergonomic dimensions of the problem~Two systems may be informationally erplivalen~ yet one might be user-friendly, while the other is not. This is one of the areas wherein IuVrVo excels. In develq& TeIe- InViVo, we have made additional modifications to the graphical user interf~ that further enhance its intuitive feel while supporting its new collaborative features.

A TeleInVtVo session begins when one party “calWanotherbyspacifyirrgtheremoteIPaddress pig. 4(a)]. The calling party imm&&ely est&Wes controlovertheother~taintheseasioncan return to stand-alone mode at any time by hitting the

Theuserinterfacehasbeenextendedtoincludea6 DOF Immer&n Prohe- arm pig. 5(a)]. The arm allowsadirect~ofthe“scanthedata and not the p@ent” concept in- earlier. The ende&&or(atylus)actsasavirtuaIuhmsou4 sensor.Acuttingphmeinthe3-Dviewispositioned and oriented according to the position and orienta- tion of the stylus @rig S(b)].

TeleInViVo? towards collaborative volume visualization environments 805

TeleInViVo State

Kernel

Go

Distributed Shared State

09 Fig. 3. TeleInViVo architecture and synchronization model.

A “freeze” button allows the user, having found while frozen, the cutting plane’s position can still he an interesting cross-section in the data, to return the modified (i.e. rotated and translated) using the stylus to its home position, while preserving the mouse. When unfrozen, the plane snaps to the current orientation of the cutting plane. However, current position and orientation of the stylus. To

806

Fig. 4. A Tel rem ote u ser.

ieIn

(4

J. Coleman ef 01.

ViVo session with volume transfer. (a) Initial hi-res dataset. (b) Remote user selects a subvolume for hi-res. (d) High-res sul

surrounding.

,ow-l

volun -es ne

C u

WY rith

sent low.

to .res

TeleInViVoTM: towards collaborative volume visualization environments 807

Fig. 5. (a) The Immersion ProbeTM. (b) Using the Immersion ProbeTM to “reslice” the data.

add visual clarity to the relationship between the cutting plane and the data, several graphical modifications were made to the interface.

Portions of the volume that lie between the viewer and the cutting plane are made semitransparent so that the plane shows through [Fig. 6(a)]. Where the plane slices through data, the data is highlighted in red. The polygon formed by the intersection of the plane with the bounding box is also outlined in red and the bounding box is outlined in green. Several aids to orientation and positioning have also been implemented.

A white-bordered rectangular plane is drawn surrounding the bounding box and is coplanar with the red polygon. The coplanarity of the two polygons acts as an aid in determinkg orientation and positioning. In addition, the polygons and the bounding box employ depth queuing. This combina- tion has been found to be very effective. Along with the 3-D view, there is a 2-D popup window that always displays the contents of the cutting plane

Fig. 6. (a) Highlighted plane cuts through data in 3-D view. (b) View of cutting plane in 2-D.

808 J. Coleman et al.

from a view direction parallel to the plane’s normal [Fig. 6(b)]. The polygon representing the intersection with the bounding box is again shown in red. As an aid in orienting the two views, a glyph representing a focal coordinate system in the cutting plane, labeled UFR, is displayed. In the 2-D view, these axes are always tixed. The 2-D view (without glyph and border) corresponds to what a radiologist would see on a standard ultrasound machine. In the 3-D view, the glyph moves with the plane.

Certain standard InViVo features have been enhanced. Rotation in the 3-D viewing window has traditionally been a two-step process in InViVo. One would first rotate in the wireframe view and then click an icon to update the 3-D view. TeleInViVo now supports direct, real-time rotation in the 3-D view by clicking and dragging (Fig. 7). A low resolution version of the volume is rendered in real- time. Upon release of the mouse button, the volume is re-rendered automatically at full resolution. InViVo’s measurement tools have also been im- proved in TeleInViVo with the addition of rubber- banding. The user can now see a line spanning the distance between two points as a measurement is taken (Fig. 8).

Fig. 8. Measurement in 2-D popup view shows “rubber- band.”

A simple, yet surprisingly useful, modification has been the addition of a help button that brings up a the Netscape Web browser with an HTML version of the TeleInViVo user manual (Fig. 9). With appro- priate hyper-links to other places on the Web, it opens up whole new possibilities.

3.2.3. CSCW. In casual conversation floor control is generally handled through implicit, nonverbal cues. Several kinds of information are constantly being communicated:

wanted to maintain this same sense of informality and intuitive control, while still communicating the three kinds of information. The hrst two are conveyed through the use of colored borders around each of TeleInViVo’s four main windows [cf. Fig. 4(d)]. These borders are present only when the local participant is in control of the session. In addition, they are green when the remote process is fully synchronized and red when it is not. The third item, taking control of the session, is as simple as clicking in any of TeleInVrVo’s four main windows. The local end gains colored borders, while the remote end loses color [c$ Fig. 4(b)]. In addition the remote end will have buttons that are grayed out (i.e. inactive). The reason that this approach has been successful, no

Who’s in control of the floor? Is the other party following the conversation (i.e. synchronized)? May I take control?

In designing a mechanism for TeleInViVo we

Fig. 7. Rotating the 3-D view in real-time using a low resolution stand-in.

Fig. 9. Clicking on the help icon, ‘?‘“, calls up NatscapeTM with an HTML version of the refkrence mairuat.

TelelnViVo? towards collaborative volume visualization environments 809

doubt, is due in part to the complimentary use of audio and video teleconferencing tools. There is the additional assmnption, apparently true, that both parties will conduct themselves in a non-aggressive, “Ci- manner.

Views in the four main windows and in the popups are shared. Synchronixation is primarily automatic, but some operations are under mamml control. These are usually compute-intensive operations such as rendering. There is a render button whose border goes red whenever remote rendering is needed [c-f- Fig. 4(c)]. Telepointing, a standard feature in many collaborative applications, has also been implemen- ted in all of the main windows and popups. Most of InVNo’s capabilities are under &control. This means that segmentation, the magnifying glass, trajectory planning, etc., can all be employed to trmsmit one user’s point of view to a remote observer.

With the addition of off-the-shelf technologies such as InPersonrr’r to provide audio, video, and white- one has a fairly complete telemedica& teleconferencing application that provides a high level of responsiveness (Fig. 10).

3.3. Results In April and May of 1996, an early prototype

version of TeleInVtVo was showcased at several comerences including the Association of the US Army (AUSA), Health Information Infmstructure ‘96, and Virtual Reality in Surgery. The demonstra- tion had TeleInViVo running on a SGI Indy with IRIX 5.3 and 32 Mb of memory, comvscted to a second copy of TeleInViVo running on a Pentium- based Laptop with LINUX and 64 Mb of memory. Since TeleInViVo is designed to be portable, it requires no special hardware for its performance. The PC system was signi6cantly faster than the Indy even for small datasets! In these demos, vie and vat

provided video and audio capability, mspectively. A QuickCamm camera was attached to the laptop. A local net was con@ured with just the two machines. In such an environment, latency was not an issue and performance was excellent.

Also in April, Fraunhofer CRCG and Fraunhofer IGD used TeleInViVo to demonstrate 3-D Teleme- dicine over global ATM links. This was part of the MAY (Multimedia Application on Intercontinental Highway) intercontinental ATM network sponsored by Global One, Teleglobe Canada, German Tele- kom, and its subsidiary, DeTeBerkom.

In collaboration with Richard Littlefield and researchers at Battelle Pacific Northwest Labora- tories in Richland, Washington, and Dr Chris Macedonia, currently at Georgetown University Medical Center, TeleInViVo has been incorporated into a portable 3-D Ultrasound system called MUSTPAC-1. This prototype system will be field tested in Bosnia in August 1996. Preliminary results have been promising based on sessions between CRCG in Providence and Battelle in Richhmd, Washington, and between CRCG and Dr Chris Macedonia in Washington, DC. Both stations were SGI Indy’s (one each of R4400 and R4600) with 32 Mb of memory. In one such test, a 9 Mb liver phantom was transmitted first as a thumb and then in high resolution for a subvolume. Transmission time was close to a minute for both transmimions. The most notable problem was the latency. View synchronization could at times exceed one or more seconds. Part of this latency was due to the use of congested links on commercial Internet connections. In all other respects, the system performed as expected.

4. CONCLUSIONS

Telemedical applications for volume visualization are a rather new phenomenon growing out of the

Fig. 10. TeleJnVIVom in combination with InPe~~on~.

810 J. Coleman et al.

happy contluence of several enabling technologies and from the need to provide high quality, round- the-clock service at any time, anywhere in the world. Expanding upon InViVo’s basic visualization tools and performance assets, TeleInViVo has demon- strated the feasibility of extending the teleradiolo- gist’s domain to incorporate resource-demanding, but informatically richer datasets, in a manner that is compatible with current bandwidth limitations. This is accomplished primarily through “status-passing” mechanisms and data compression.

Integration of the Immersion Probe input device has added a whole new dimension to the process. Manipulation of the probe allows the physician to exploit existing psychomotor skills, thus enhancing the naturalness of the interaction. This is consistent with the current trend towards the design of natural user interfaces that process voice, handwriting, and (in this case) gestures. This mitigates the need to develop whole new skill sets while making the tool immediately useful. The intuitive floor control method also adds to the sense of responsiveness and friendliness.

The search for user interaction models that are informatically equivalent to traditional or natural modes should not overlook the possibility of transcending the old paradigm. For new technology to replace old technology, it must do more than accomplish the same old thing. It must generate new possibilities that extend the metaphor in new ways. The typewriter had to do more than just mimic hand printing in order to replace it. The wordprocessor was far more than just a softcopy typewriter. Likewise, to be truly compelling, applications like TeleInViVo must redefine in the mind of the user what is possible. We believe we are in the first stages of doing that.

5. FUTURE DIRECTIONS

5.1. Architecture The current TeleInViVo architecture will be

extended to allow for multiuser sessions. It is likely that a combination of client-server and distributed computing approaches will be taken. This will allow for the incorporation of various additional services in a distributed environment such as supercomputing resources and various forms of input and output. InViVo-Scan, developed by IGD, CRCG’s sister institute in Darmstadt, will also be incorporated to provide real-time, unconstrained, 3-D Ultrasound data acquisition.

5.2. Telecommunications Future versions of TeleInViVo wilI have support

for modem hook-ups using PPP/SLIP and ISDN. UDP will also be available alongside TCP for efficient implementation of certain real-time opera- tions such as telepointing. It might be necessary, however, to make allowances for communication between agencies that have “fire walls”. Our

experience has been that some institutions provide “holes” for TCP traffic, but will not ahow UDP traffic. CRCG, in collaboration with IGD, will undertake to develop a set of middleware tools that will allow for optimal performance in scalable networking environments. When completed, Tele- InViVo will be transitioned to operate with this toolset. It is expected that the implementation will be CORBA compliant.

5.3. Graphics and imaging TeleInViVo, which currently displays only grey

level values, will be extended to support RGB data. New volume visualization techniques (e.g. translu- cent clouds) and graphic elements [e.g. mirrors) will augment existing capabilities. Also in the works is a version that will support the use of multiple datasets and their registration. The incorporation of ad- vanced tools for volume analysis and diagnostics, some of which have already been developed at IGD and others that are currently under development at CRCG, will increase the power and utility of TeleInViVo. Wavelets will be explored as an alter- native to our current volume representation. This might also lead naturally into rendering methods based on successive refinement algorithms. Rather than building a full resolution image on one pass, a line at a time, the entire image would be displayed at higher resolution on each successive pass. The ability to extract geometric representations from volumetric data would be a useful addition. It is also desirable to add further orientation clues that allow the user to properly position a scanned volume with respect to the full body. This feature may employ a detailed anatomical representation of the human body, maie and female. CRCG’s strengths in virtual reality may also be employed in this context to create a truly immersive environment [7, 81. The UI will be further simplified by providing normal and expert modes and the use of “parameter bundling” or macros for common operations. Help will be expanded with a tutorial and trouble-shooting notes.

5.4. cscw Rules governing the interaction of more than two

users might need to be revamped. It remains to be seen how well the current model of floor control can be extended in this multiuser environment. A means must be developed for indicating which user has control, the degree of synchronization, et<:.

5.5. Application domains Because TeleInViVo is a general purpose 3-D

volume visualization tool, it clearly can be applied outside of the medical domain. For example, we are currently investigating educational, geophysical (e.g. mine detection, multispectra data), underwater sonar, nondestructive testing, web browser exten- sions, and volumetric database applications.

TeleInViVoTM: towards collaborative volume visualization environments 811

Acknowledgements-This work has been sponsored, in part, in collaboration with Battelle Pacific Northwest National Laboratories (under Battelle subcontract number 261582-B- A7), with funding from the Defense Advanced Research Proiects Agency (DARPA) and the US Army Medical Advanced Technology Management Office (MATMO). We’d like to thank Dr.-Ina. Georaios Sakas at Fraunhofer IGD for the invention and use \f InViVo, Mr Richard Littlefield at Battelle PNL for his overall program leader- ship and, specifically for his helpful suggestions on interfacing the Immersion Probe, and CPT Christian Macedonia (U.S.A.), MD, for the vision, guidance, and imagination that got this effort underway. Also, thanks to Chris McGowan, Jason Murphy, and Uxiel Schaefer for documentation and testing support of TeleInViVo. Finally, special thanks to Dr Michael Macedonia, VP for Global Work Environments at CRCG, for his vision, encourage- ment, and in-house support.

REFERENCES

1. Sakas, G., InViVo-interactive visualizer of large scalar voxel fields. Computer Graphik Topics, 1991, 4, 12-13.

2.

3.

4.

5.

6.

7.

8.

Sakas, G., Interactive volume rendering of large fields. The Visual Computer, 1993, 9, 42H38. Sakas, G. and Walter, S., Extracting surfaces from fuzzy 3D-ultrasound data. In SIGGRAPH ‘95 Con- ference Proceedings, Los Angeles, CA, 1995, pp. 465- 474. http://www.igd.thg.de/www/igd-a7/Projects/Invivo/En- glish/invivo.html. Elvins, T. T., Volume visualization in a collaborative computing environment. Computers & Graphics, 1996, 20,219-222. Liu, P. W., Chen, L. S., Chen, S. C., Chen, J. P., Lin, F. Y. and Hwang, S. S., Distributed computing: new power for scientific visualization. IEEE Computer Graphics and Applications, 1996, 16(3), 42-51. Macedonia, M. R., Brutzman, D. P., Zyda, M. J., Pratt, D. R., Barham, P. T., Falby, J. and Locke, J., NPSNET: a multi-player 3D virtual environment over the intemet. In Proceedings of the 1995 Symposium on Interactive 30 Graphics, Monterey, CA, 1995. Macedonia, M. R. and Zyda, M. J., A taxonomy for networked virtual environments. In Proceedings of the 1995 Workshop on Networked Realities, Boston, MA, 1995.