Original ArticlepISSN 1738-2637 / eISSN 2288-2928J Korean Soc Radiol 2015;73(5):312-322http://dx.doi.org/10.3348/jksr.2015.73.5.312
INTRODUCTION
Similar to other disciplines of medicine, radiology training re-lies mainly on apprenticeship in the workplace, which may be supplemented with various educational media such as offline or
online textbooks, journal articles, lectures, and case collections. However, the conventional media have not evolved in parallel to several important changes that modern radiology practice has experienced recently. First, in illustrative cases, conventional me-dia usually relies on a limited number of key images. They rarely
Development of an Online Radiology Case Review System Featuring Interactive Navigation of Volumetric Image Datasets Using Advanced Visualization Techniques고급 영상기법을 이용한, 체적 영상 자료의 네비게이션이 가능한 온라인 영상 교육 시스템의 개발
Hyun Kyung Yang, MD1, Hyunsik Woo, MD2*, Jaemin Jo, BS3, Min Hee Lee, MD4, Bohyoung Kim, PhD1, Jinwook Seo, PhD3, Ju Hyun Jung, RT1, Heung Sik Kang, MD1, Kyoung Ho Lee, MD1,5
1Department of Radiology, Seoul National University Bundang Hospital, Seongnam, Korea2Department of Radiology, SMG-SNU Boramae Medical Center, Seoul, Korea3Department of Computer Science and Engineering, Seoul National University, Seoul, Korea4Department of Radiology, Soonchunhyang University Bucheon Hospital, Bucheon, Korea5Program in Biomedical Radiation Sciences, Department of Transdisciplinary Studies, Graduate School of Convergence Science and Technology, Seoul National University, Suwon, Korea
Purpose: To develop an online radiology case review system that allows interactive navigation of volumetric image datasets using advanced visualization techniques.Materials and Methods: Our Institutional Review Board approved the use of the patient data and waived the need for informed consent. We determined the follow-ing system requirements: volumetric navigation, accessibility, scalability, undemand-ing case management, trainee encouragement, and simulation of a busy practice. The system comprised a case registry server, client case review program, and com-mercially available cloud-based image viewing system.Results: In the pilot test, we used 30 cases of low-dose abdomen computed tomog-raphy for the diagnosis of acute appendicitis. In each case, a trainee was required to navigate through the images and submit answers to the case questions. The trainee was then given the correct answers and key images, as well as the image dataset with annotations on the appendix. After evaluation of all cases, the system dis-played the diagnostic accuracy and average review time, and the trainee was asked to reassess the failed cases. The pilot system was deployed successfully in a hands-on workshop course.Conclusion: We developed an online radiology case review system that allows in-teractive navigation of volumetric image datasets using advanced visualization techniques.
Index termsRadiologyEducationInternet
Received March 17, 2015Revised June 27, 2015 Accepted September 19, 2015*Corresponding author: Hyunsik Woo, MDDepartment of Radiology, SMG-SNU Boramae Medical Center, 20 Boramae-ro 5-gil, Dongjak-gu, Seoul 07061, Korea.Tel. 82-2-870-2177 Fax. 82-2-870-2899E-mail: [email protected]
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0) which permits unrestricted non-commercial use, distri-bution, and reproduction in any medium, provided the original work is properly cited.
jksronline.org J Korean Soc Radiol 2015;73(5):312-322
allow trainees to interactively navigate volumetric computed to-mography (CT) or magnetic resonance image datasets (1-3), limiting training in lesion detection at unpredictable locations (e.g., the detection of pulmonary nodules) or in understanding complex 3-dimensional configurations (e.g., the local staging of abdominal cancer invading adjacent organs). Recent studies re-ported that volumetric educational media could overcome these limitations of conventional media (1, 4). Second, conventional media rarely utilize new visualization or quantitative imaging techniques. The incorporation of these novel techniques into dai-ly practice necessitates education of the radiology community on the use of appropriate media (5). Third, conventional media pay little attention to the busy practices where radiologists are re-quired to improve clinical productivity (6).
Future educational media should ideally overcome these limi-tations (7). Internet and web-based educational media are used increasingly in radiology education (8-13). A teaching file soft-ware on mobile devices is also available (14). Several investigators recently attempted to integrate interactive 3-dimensional visual-ization of volumetric image datasets into radiology education (15, 16). One recent study (1) reported increase in recognition of pul-monary embolism among novice residents after using a teaching file of CT angiography. Another report (15) indicated that the in-tegration of interactive 3-dimensional image post-processing software into undergraduate radiology education effectively im-proved radiological reasoning, diagnostic skills and visual-spatial ability, and thereby even diagnostic skills for imaging modalities. However, we were unable to find any practicable platform for on-line case review that allows the navigation of volumetric image datasets using advanced visualization techniques, such as multi-planar reformation, volume rendering, and fly-through. There-fore, we developed an online case review system that allows in-teractive navigation of volumetric image datasets using advanced visualization techniques.
MATERIALS AND METHODS
The Institutional Review Board of Seoul National University Bundang Hospital approved the use of the patient data and waived the need for informed consent.
Development Team
Between September 2012 and April 2013, the case review sys-tem was developed by a team that comprised 2 teaching radiolo-gists (K.H.L. and M.H.L. who had 10- and 2-year experience, respectively), a practitioner radiologist (H.W. with 5-year experi-ence), a radiology resident (H.K.Y.), and 2 computer scientists (B.K. with 15-year experience in medical image processing and J.S. with 10-year experience in human computer interaction). The practitioner radiologist led the system development. A computer science student (J.J.) and a radiology technologist (J.H.J.) assisted in system implementation. The radiologists and radiology resi-dent also served as case authors by preparing the case data for a pilot test of the system. All members of the development team par-ticipated in developing the system requirements and implemen-tation (Table 1).
System Requirements
The system requirements were determined as follows. First, the system should allow the trainees to navigate through volumetric image datasets using advanced visualization techniques such as multiplanar reformation, volume rendering, and fly-through. Second, the system should be accessible from trainees’ desktop computers, although volumetric navigation typically requires a high-end computing system. Third, the system should be scalable to accommodate an increasing amount of case data and increas-ing number of trainees. Fourth, management and updating the case data should not be demanding, even after the distribution of the software program for case reviews to many trainees. This was
Table 1. System Requirements and ImplementationSystem Requirements System Implementation
Navigation of volumetric image datasets Commercially available cloud-based image viewing systemAccessibility from trainees’ desktop computers Commercially available cloud-based image viewing system
Scalability to accommodate an increasing amount of educational data and an increasing number of trainees
Commercially available cloud-based image viewing system
Undemanding management of case data Case registry server with a web-based case management toolEncouragement of participation by trainees Client case review program in an interactive case challenge formatSimulation of a busy practice Client case review program measures the review time
314
Online Interactive Case Review System
jksronline.orgJ Korean Soc Radiol 2015;73(5):312-322
to cover a variety of fields in radiology and to keep pace with the rapid changes in knowledge and technology. Fifth, the system should encourage trainees to participate throughout the case re-view course. Sixth, the system should simulate a current busy practice environment.
System Implementation
The system comprised an image viewing system, case registry server, and client case review program (Fig. 1). We selected a commercially available cloud-based image viewing system be-cause it met the important system requirements of volumetric navigation, accessibility, and scalability. We developed the case registry server to facilitate the maintenance of the case data. We used an interactive case challenge format for the client case re-view program to encourage participation by trainees (17); and a timer display was used in the client case review program to simu-late a busy practice environment.
Cloud-Based Image Viewing System
The image viewing system comprised a cloud image server (iNtuition CLOUD; TeraRecon, San Mateo, CA, USA) located in Seoul, Korea and a thin-client image viewer (AquariusNET v4.4.11.5; TeraRecon). The cloud image server stored the volu-metric image datasets and processed the images as requested by
the trainees. The client image viewers in the trainees’ computers displayed the final processed images received from the server. The trainees could utilize all the advanced visualization tech-niques that were available in the image viewing system (18), in-cluding multiplanar reformation, volume rendering, and fly-through.
Owing to the use of a cloud-based image viewing system, we could avoid the hardware burden and an arduous installation process on many computers of trainees, and the cumbersome distribution of large volumetric image datasets to the trainees’ computers, all of which might have discouraged the trainees from participating. In addition, the cloud image server provided a platform where the case authors could collaboratively upload, maintain anonymity, and annotate the image datasets via the In-ternet.
Case Registry Server
The case registry server was developed on the Ruby on Rails 3.1 platform (Rails Core Team, http://rubyonrails.org/core/). The server ran on a Linux system (Ubuntu 12.04 LTS; Canonical, London, UK) equipped with a 3.1 GHz 4-core Xeon CPU (Intel, Santa Clara, CA, USA) and 16 GB of main memory. The case reg-istry server interacted with a web server (Apache 2.2.22; Apache Software Foundation, Los Angeles, CA, USA) and a database man-
Fig. 1. Diagram showing overall architecture of the system. The system comprised a case registry server, cloud image server, client case review program, and client case image viewer.
Case registry server
Case data User response Image request Image data
Trainee’s computer
CommandClient case review program Client image viewer
User interface
Image display
Trainee
Cloud image server
315
Hyun Kyung Yang, et al
jksronline.org J Korean Soc Radiol 2015;73(5):312-322
agement system (MySQL 5.5.34; Oracle, Redwood Shores, CA, USA), which also ran on the Linux system. The case registry serv-er stored the case data, including brief clinical findings, case ques-tions, correct answers, and the identification numbers of the im-age datasets stored in the cloud image server, as well as the account information and response data of the individual trainees (e.g., their submitted answers and the review time for each case). The case registry server provided a web-based case management tool that allowed the case authors to upload and maintain the case data. The case registry server also provided a download link for the client case review program.
Client Case Review Program
We developed a client case review program, which provided user interfaces for the trainees on the .NET Framework 4 platform (Microsoft, Redmond, WA, USA). In order to encourage the train-ees to participate, the client case review program was developed in an interactive case challenge format. For each case, trainees were asked to answer several case questions, after which they were provided with the correct answers and key images, as well as annotations of important findings. A timer measuring the re-view time for each case was displayed to prompt the trainee to review the case as rapidly as possible, thereby simulating a busy practice environment.
RESULTS
In this section, we describe a pilot test of the developed system.
Preparation of Case Data
In the pilot test, we decided to use cases of low-dose abdomen CT for the diagnosis of acute appendicitis. This application was considered suitable for demonstrating the advantages of the de-veloped system. Appendicitis is a very common medical emer-gency and the preoperative CT reports are often made by non-expert radiologists (19, 20), but the reports play a critical role in patient disposition (21). An educational program that systemati-cally familiarizes radiologists with the noisy low-dose CT images would facilitate the dissemination of the low-dose CT technique, which is promising (22, 23), although not yet widely accepted as standard practice (24, 25). While the image viewing system com-prised many advanced visualization techniques, we selected mul-
tiplanar (26) sliding-slab average intensity projection (27, 28) in the pilot test, as the technique is known to help the diagnosis of appendicitis (29-31).
We randomly selected 30 low-dose (estimated effective dose of 2 mSv) CT cases, including 14 with confirmed appendicitis and 16 without appendicitis, from the database of a previous study (22). The case authors prepared the case data, including brief clinical findings, case questions, and correct answers, and then uploaded them to the case registry server. Four questions based on a report form used in recent studies (22, 23) were provided for each case: the visibility of the appendix (on a 3-point Likert scale), likelihood of appendicitis (on a 5-point scale), likelihood of ap-pendiceal perforation (on a 3-point scale), and any potential al-ternative diagnosis (as an open question).
Each CT image dataset had a section thickness of 2 mm and a reconstruction interval of 1 mm. The image datasets were anony-mized and uploaded to the cloud image server in the Digital Im-aging and Communications in Medicine format. A second copy of the image dataset was generated and stored in the cloud image server to provide feedback to the trainees. In the second copy, the case authors selected the set of transverse and coronal images that best depicted the appendix, and they annotated the selected images with arrows on the appendix and other important image findings. The case authors could also define the zooming, pan-ning, and windowing as appropriate. This information was stored in the cloud image server, which allowed the trainees to review the annotated image datasets by starting from the selected images with the predefined setting.
Task Flow in Case Review
The pilot system prompted trainees to review the 30 cases. The order of the 30 cases was randomized for each trainee. For each case challenge, the client case review program briefly displayed the clinical findings to the trainee. The system led the trainee to navigate through the volumetric image dataset using the multi-planar sliding-slab average intensity projection technique in the client image viewer. After the trainee submitted their answers, the client case review program displayed the correct answers and key images that were retrieved from the case registry server. The trainee was then asked to review the annotated image dataset to identify the appendix and other important image findings on the client image viewer, again using the multiplanar sliding-slab aver-
316
Online Interactive Case Review System
jksronline.orgJ Korean Soc Radiol 2015;73(5):312-322
age intensity projection technique. After the trainee finished all the 30 cases, the client case review program displayed the train-ee’s diagnostic accuracy and average review time per case. The accuracy was defined as the percentage of correct diagnosis or exclusion of appendicitis based on the likelihood of appendicitis with grade 3 or higher as a positive response. The trainee was asked to reassess his or her failed cases until the revised accuracy exceeded 95% (Fig. 2, Movie 1 in the online-only Data Supple-ment). The detailed task flow during a case review cycle in the pi-lot system was provided in Fig. 3.
Deployment of the Pilot System
The pilot system was deployed as a hands-on workshop course in the Advanced Imaging Multimodality Schools and Seminars of European School of Radiology (32), which was held in Seoul, Korea in July 2013. In total, 49 radiologists with limited previous experience in low-dose abdomen CT participated in the course by using their own laptops to run the client case review program. Due to time constraints (40 minutes assigned to the hands-on workshop, including program installation), only 5 (10%) partici-pants could complete all the 30 cases during the workshop course.
Fig. 2. Screenshots of the client case review program. A. A question page shows brief clinical findings (red underline), case questions (green box), diagnostic criteria for appendicitis (blue box), and the time spent reviewing each case (orange box).
A
317
Hyun Kyung Yang, et al
jksronline.org J Korean Soc Radiol 2015;73(5):312-322
The median number of cases reviewed by individual participants was 13 (interquartile range, 8-21) and the median per-case re-view time for individual participants was 91 (interquartile range, 67-119) sec. The course was rated as very good (mean ± standard deviation, 4.7 ± 0.7) according to the participants’ responses re-garding the satisfaction of the workshop course using a 5-point Likert scale (1, very poor; 2, poor; 3, satisfying; 4, good; 5, very
good). The workshop course is available for public use, and inter-ested readers can try it without charge after registering at http://www.locat.org/users/new?locale=en&no_phone=1&no_hospital=1&no_career=1&no_career_year=1&no_account=1&group_id=33.
Fig. 2. Screenshots of the client case review program. B. A feedback page shows the submitted answers (red box), correct answers (green box), and annotated key images (blue box). The trainee can open the annotated image dataset in the client image viewer by clicking the ‘Open Annotated Images’ button (orange box).
B
318
Online Interactive Case Review System
jksronline.orgJ Korean Soc Radiol 2015;73(5):312-322
DISCUSSION
We developed an online radiology case review system that fa-cilitates the interactive navigation of volumetric image datasets. The system was designed to meet the following 6 requirements: volumetric navigation, accessibility, scalability, undemanding management of case data, encouragement of participation by trainees, and simulation of a current busy practice environment. The system was deployed successfully in a hands-on workshop course as a pilot test.
We designed the system with the following features to simulate traditional workplace apprenticeship. First, the trainees were asked to integrate the image findings and clinical findings such as
a brief clinical history. Second, in order to help the trainees to ex-perience a wide spectrum of a disease, the case pool could in-clude positive and negative cases (33), and typical and atypical cases. Third, the system had several features to provide feedback that focused on the weaknesses of individual trainees, including an annotated image dataset for the second image review, report-ing the diagnostic accuracy of trainees and their average review time per case, and encouraging the trainees to reassess failed cas-es. Further investigation to individualize the case based on each trainee’s performance may lead to more efficient education.
The application of our system is not confined to a specific dis-ease or visualization technique. It could be expanded to many other diseases and advanced visualization techniques. Potential
Fig. 2. Screenshots of the client case review program. C. A training record page shows the trainee’s diagnostic accuracy and average review time per case (red box). The trainees can reassess their failed cases by clicking ‘Review’ buttons (blue box).
C
319
Hyun Kyung Yang, et al
jksronline.org J Korean Soc Radiol 2015;73(5):312-322
applications may include the detection of colonic polyps with fly-through in CT colonography, segmentation and volume mea-surement of pulmonary nodules in low-dose lung CT, and vessel analysis such as stenosis and soft plaque evaluation in coronary
CT angiography. Furthermore, the application of our system is not limited to educational purposes. The system can also be used to accredit radiologists involved in a multicenter clinical trial to ensure participant safety.
Fig. 3. Sequence diagram showing interactions during a case review cycle. In each case review, the client case review program retrieved the case data, including brief clinical findings, case questions, and the identification number of the image dataset (the first copy without annotations), from the case registry server and displayed the brief clinical findings to the trainee. After receiving the trainee’s request, the client case review program sent a command that included the identification number of the image dataset to the client image viewer, and the client image viewer then displayed the processed images received from the cloud image server for the trainee to review. During the review process, the client case re-view program displayed the time spent reviewing each case. The review time and answers submitted for each case were stored in the case regis-try server. After the trainee submitted the answers for each case, the client case review program received the correct answers and key images from the case registry server, and displayed them to the trainee as feedback. Following the trainee’s request, the client case review program sent a command containing the identification number of the annotated image dataset (the second copy with the annotations for the appendix and other important image findings) to the client image viewer, and the trainee could navigate through the annotated image dataset using the client image viewer. The boxes with dashed vertical lines represent lifelines of the four components of the system and a trainee. The vertical gray bars on the lifelines represent executions. The solid horizontal arrows represent messages between the lifelines and dotted horizontal arrows represent reply messages. ID = identification number
Loop
Trainee
Trainee’s computer
Client case review program
Client image viewer
Case registry server
Cloud image server
Start case
Display clinical finding
Start image reviewID of image dataset
Submit answersand time records
ID of second copy of image dataset
Stop timer
Submit answers
Proceed to the next case
Start second review
Display feedback
Start timer
Request image processing
Processed imagesDisplay images
Display images Processed images
Request image processing
Correct answers and key images
[Until the last case]
Request case data
Case data
320
Online Interactive Case Review System
jksronline.orgJ Korean Soc Radiol 2015;73(5):312-322
Our study had limitations. First, we did not comparatively ana-lyze the educational advantages of our online case review system vs. conventional educational media. Further investigations are required to assess the advantages of our system. Second, the edu-cation program was developed as a downloadable program run-ning on the operating system of Microsoft Windows. This choice was inevitable because the selected commercially available client image viewer ran on Microsoft Windows. As the computing en-vironment becomes more diverse and mobile, the online educa-tion system of the future should be web-based and compatible with multiple devices including smartphones or tablets. Third, a commercially available cloud-based image viewing system was used to implement our system. While this decision greatly saved our time and efforts in developing the case review system and provided the trainees with familiar user interface used in their practice, additional client program (client case review program) and 2 separate servers (case registry server and cloud image serv-er) were unavoidable. A more simplified system with 1 single cli-ent program or web-based case review system would be more practical.
In conclusion, we developed an online radiology case review system that allows the interactive navigation of volumetric image datasets using advanced visualization techniques.
Acknowledgments
This study was supported by grants from the Korean Health Technology R&D Project, Ministry of Health & Welfare, Repub-lic of Korea (HI13C0004), the Basic Science Research Program through the NRF funded by the MEST (NRF-2013R1A1A2006851), Seoul National University Bundang Hospital Research Fund (No. 14-2014-038), and Bracco Imaging Korea Ltd. (SCHBC 2013-01-007). TeraRecon Korea supported the image viewing system used in the study. The authors are grateful to Dr. Nicholas Gourtsoyi-annis and other staff of European School of Radiology for their support in deploying the case review system in the workshop.
SUPPLEMENTARY MATERIALS
The online-only Data Supplement is available with this article at http://dx.doi.org/10.3348/jksr.2015.73.5.312.
REFERENCES
1. Williams J, Sato TS, Policeni B. Pulmonary embolism teach-
ing file: a simple pilot study for rapidly increasing pulmo-
nary embolism recognition among new residents using in-
