v ;"4
dr
NASA Technical Memorandum 107563
Quo Vadimus?Multimedia
The 21st Century and
Allan D. Kuhn
NOVEMBER 1991
(NASA-IM-lOT563) QUO VADIHUS? THE
CENTURY AND MULTIMEOIA (NASA) 18
G3182
N92-22663
ASA
https://ntrs.nasa.gov/search.jsp?R=19920013420 2018-05-02T08:36:51+00:00Z
- =
NASA Technical Memorandum 107563
Quo Vadimus?Multimedia
The 21st Century and
Allan D. Kuhn
Prepared for the Scientific and Technical Information ProgramWashington, DC
N_A National Aeronautics and Space AdministrationScientific and Technical Information Program
Washington,DC 1991
E
QUO VADIMUS? THE 21STCENTURY AND MULTIMEDIA
"The computer display screen will be mankind's new home."- Ted Nelson, Dream Machines, The Flip Side of
ComputerLib, Microsoft Press, 1974. 1987.
Allan D. KuhnNASA Scientific and Technical Information Program
November 1991
In this year of 1991, we are in the 21stCentury.
Advances in adaptation of technology arehistorically accentuated, benchmarked, andpropelled forward, as distasteful as it is,by wars. A prime example is the 19thCentury American Civil War. It iscommonly accepted, in historical context,that this war was the first war of the 20thCentury simply through its rapidtechnology developments in weapons andweapons support systems. The repeatingcarbine, Gatling gun, iron clads, telegraph,i,e., electric communications, submarines,balloons, i.e., lighter-than-air craft,railroad transportation, weaponsmetallurgy and ballistics, photography,i.e., imaging technologies, are examples ofthe Civil War technology adaptations thatinfluenced the thrusts of warfare for mostof the 20th Century. Peacetime growth oftechnologies was in turn propelled forwardby the wartime developments, e.g.,expansion of electricity-drivencommunications, Image presentation,railroad and automotive transportation,distribution systems for commodities,manufacturing technologies, foodstuffspreservation, aircraft developments, andon and on.
In our common experience of today, thePersian Gulf War of January, 1991,
because of its unprecedented technology, isthe first war of the 21 st Century. It is thebenchmark war that has kicked us acrossthe threshold to the 21 st Century. Up untilthis war, a plethora of reports, books, andcommentaries coming out of the Federalgovernment, private industry, academia,and the press constantly questioned theworld position of the United States in itspredominance in technology. Thetechnology used in the Persian Gulf War,successfully proven under fire, confirmsU.S. technology prowess.
NASA had a major part in the success ofAmerican technology in the Persian GulfWar, as indicated in its summer 1991issue of NASA Update - State and LocalNewsletter.* A brief article titled"Advanced Technology Hastens Victory inGulf Conflict" (p. 3), states:
Victory in the Persian Guff conflict wasa triumph of advanced technologies.Many of those technologies had roots inaeronautics and space research anddevelopment that NASA and theU.S.military establishment haveconducted or sponsored over the years.
t
NASA Office of External Relations, Defenseand Intergovernmental Relations Division.NASA Update: State and Local Newsletter 5,1 (1991).
=
=
• .. Some of the types of systems thatcontributed to the speed with which thewar was terminated and that werepossible only because of advancedtechnology development by NASA and themilitary are:
• Visual and infrared imaging systemsand the related systems for interpretingand presenting information collected bythe imaging systems. Those systemscontributed significantly to the accuracyachieved in reconnaissance,surveillance, and targeting.
• Satellite-based communications
...we find ourselves living in a newenvironment .... New relationships andnew opportunities are resulting fromthe increased availability of informationand the stimulation and expandingopportunities it brings. There will benew individual and organizationalrelationships, new methods of doing ourwork, and new environments in whichwe will carry out our work.
A part of our 21st Century environmentand its "new methods of doing our work" iscomputer-driven multimedia, an ever-expanding means of conveying informationto people.
systems that provideand from remote locations. Essential to
the coordinated, effective ope_fations thatspee_ly provide(] victory, they had theside effect of turning the conflict into a"media event."
• Computers that are small, rugged, andportable. These ubiquitous marvels ofthe space age made possible any numbero_f improvements in planning andexecution not previously possible.
• Propulsion and guTdance systems,which enable "smart munitions" toattack defended targets with devastatingaccuracy and the Patriot missile tobecome the "Scud buster."
transmissions to
WHAT IS MULTIMEDIA?
Multimedia is computer integration andoutput of text, animation, audio, video, andgraphics.
You sit at your computer screen and watchand hear the action. You choose what youwant to ta_e place. You listen to whatsomeone has to sayas youwatch him speak;you e_r-music as bac_-ro-un-_-io- themultimedla presentation; with scientificexperiments and studies you witness theexperiment and its attendant results in alltheir forms - visual and auctorial; youexamine changes, shapes, and forms basedon those changes you yourself select.
The •bottom line • for advanced
technology_ in the U.S. arsenal is the A NA_SA engineer at Langley Researchspeed with which hostilities were Center reviews a wind tunnel test. _Thesuccessfully concluded-andtheavoidance engineer inserts an optical disc i_nto aof unnecessary injuries and deat_ls of _, an_serv-es the actual test as itU.S. personnel. .... took p_(__-_Tlii_, |ncrudes sound and
visuals, in addition to the manuals neededA good part of that technology involved to run both the wind tunnel multimediamachines making_ec[sions and conveying System, and theaCtua[ W_n_unnels_steminformation based on those decisions to itself. An embedded advisory system notpeople, vis.&-_,is-people making d-ec_sions 0nly stores experts' knowledge andbased on raw data provided by machines.This is the new environment, not only forwarfare, but now also for peacetime. Inthe first NASA STIP Quo Vadimus documentof May 1991 (BLAD91), the author, Mr.Walter Blados states:
experiences, but can be added to by theengineer. The engineer proceeds withvisualizations of computerized model datafrom the test -- pressure distribution inboth actual test data and computersimulation data. The test platform is re-
z z
- 2 -
oriented on the screen to see the effects ofthe test from differing vantage points. Hechanges test factors to see what changeswould take place in the test results. Bothsound and color, in addition to the videorepresentation, convey data to theengineer. The engineer, in reviewing thisearlier test, makes decisions on how toproceed with a new test.
A NASA employee, after a long day's work,goes home - and to relax, turns on hismachine. The machine has downloaded theday's soap operas. In fast-forwardingthrough the programs, he selects clips, andstores them separately on the hard disk. Inreviewing the clips, he further selectsportions of them, and then joins the clipsinto a media document that he is creating.The media document is a "paper" he iscomposing for his Doctoral thesis on TheEffect of Human Dramatization ofIntersocietal Problems of People asPortrayed by Cable Television Services.The thesis will exposit the creator'spremises through contextual visual andauditory information; text will beminimal, and that probably will be spokenby the creator, likely as voice-overs, lessin lecture form. The creator will alsocreate interactive elements in the thesis,for the referee/viewers to explore andcompare variances to the premises.
A medical doctor assistant on a NASA spacestation is faced with a crewman accidentemergency, a sliced off thumb, and needs toreview texts and procedures beforeperforming a needed reparative operation.There are several progressive multimediascenarios that can be imagined, all based ontechnologies available and emerging today.
• Scenario A - The doctor assistantrefers to his onboard, miniaturizedmedical reference system, comprised ofthe major medical references in U.S.practice. This system is replete withpage images of the original texts, withthe original drawings and photographicillustrations, including their color.The texts are rapidly searchable
("hypertext") simultaneously, andtherefore easy to cross-correlate.
There is also voice and motion pictureinstruction on medical procedures.After conferring with the system andwith "medical emergency central" --Kennedy? Johnson? -- the doctorassistant proceeds with the reparativeprocedure. This may take severalhours.
• Scenario B - In conjunction with theabove, operation is closely monitoredby earth-based medical staff followingan analysis of televised images of theinjury condition, with an automatedprogram run-through of the reparativeprocedure based on those images andvirtual reality technology, andretransmitted to the doctor assistant on
the space station. All this happens inless than an hour.
• Scenario C - Again in conjunctionwith all the above, all images of thesituation are fed into a robotic
cyberspace technology system thesystem in processing the images andmedical programs, monitored by thespace station doctor assistant and theearth-based medical staff, willconclude the needed procedures for theoperation and perform it. Cyberspacetechnology "is a type of interactionsimulation which includes humans asnecessary components. A cyberneticsimulation is a dynamic model of aworld filled with objects that exhibitlessor or greater degrees ofintelligence." [WALS91] With thiscombination of imaging-cyberspace-robotic technologies, the reparativeprocesses take place within tenminutes.
INCREDIBLE, YOU SAY?
Multimedia is here today. All the mediaelements involved in the above scenarios
are alive and well -- "cyberspace"technology, for example, has been around
3
and in development since the 1960s.Listen to what major movers in thecomputer industry are saying --
"Multimedia will change the world in the1990s as persona/computers did in the1980s." - John Scully, Apple
"Multimedia will be bigger thaneverything we do today."
Bill Gates, Microsoft
"We're going to put a computer in everyhome...with multimedia."
- James Cannavino, IBM
Do you get the feeling that there issomething highly evolutionary going on Inthe realm of information? I think the verybasic idea of multimedia is that it is thestage in computer and Informationtechnologies that will give us access toexperience. This may be seen moreeasily within the context of the stages ofinformation access:
• 1 _ Numeric, calculatory• 2 Access to Information Textual results
via coordination of terms• 3 Access to Knowledge Reviewing
and reformatting electronic text• 4 Access to Experience Cogn/tive and
sensory processing
Each of these access capabilities haveexpanding sets of tools underlying them toprovide the capabilities, with each accesscapability folding into the next level to givegreater expanded_s-_._ :_-i- _i_-_
It is generally accepted that the seminalarticle setting down the Ideas of what arenow multimedia processes was VannevarBush's "As We May Think," appearing In
Atlantic Monthly,* July 1945. In thisarticle Bush, in giving form to ideaspossible at that time, suggested creatinglinkages between documents, which tn turn
Vannevar Bush. "As We May Think."
Atlantic Monthly 176, 1 (July 1945); 101-8.
would be brought to a screen as rapidly aspossible according to those linkages. In histime, he was thinking about microfilmimages. For our time, read automated text,pictures, movement, and sound, alldocuments in our expanded concepts.
WHAT'S HAPPENING TODAY?
The forces of the market place are the bestIndicators for the use and success of
multimedia programs. The following us.es._of private sector multimedia, shown hereas examples of commercial multimediaapplications;_has_been compiled andprovided by Mr. Thomas A. Wheeler ofMultimedia Design Corporation [WHEE9t]:
• Northern Telecom is usingmultimedia to train their 20,000
employees at about 1/10th the cost oftraditional training methods. (MacWeekarticle).
• America_n Airlines has amultimedia project on the boards toprovide initial an-d-refresher training for.50,000 employees. They expect "50%.reduction in training time per employee',•reducing 300 hours of lessons to 152"hours (MacWeek, August 14, 1990).
•--GTE North, Inc., is using.multimedia to teach workers how to fix
telephone cables. "The retention rate hasto be three or four times higher thanlectures and pencil testing" (BusinessWeek, October 9, i989).
• General Motors Bu[cl_ Division
has developed a multimedia presentationthat a|i6ws iSro_ve custom-ers tobrowse an electronic brochure, on a PC,through descriptions and animated picturesof cars, complete with engine sounds. Themarketplace_ results_ of suc_h a brochurewere that 12% of those who bought carsafter receiving the electronic brochureended up picking Buick : this ,_,/as-aboutdouble its usual market share (BusinessWeek, October 9, 1989). The electronicbrochure includes car models, model
47_
=
options like color, wheels and styling, andspreadsheet operations that calculatepayments and make price comparisons withcompetitor cars. This brochure isinteractive with the customer, and
consequently the customer spends moretime with that than with a handful of paperbrochures.
• Steelcase Inc. uses an electronicbrochure to launch a new line of office
furniture. Customers "can get toinformation more conveniently than bypawing through pages of [paper]brochures" (Business Week, October 9,1989). Customers and designers move 3Drenderings about a computer screen to seehow different furniture looks in a specificroom, while a voice provides productdetails against a musical background.
• The American Heart Association
is distributing a set of digitized interactivelessons to medical schools on how to manageblood cholesterol levels. The programgives students an audiovisual rundowndetailing how cholesterol moves throughthe blood and liver.
What other commercial effects does privatesector multimedia have in addition to those_noted above? A study done by Duthie andAssociates [WHEE91] has compared thecosts of different training methods, basedon cost per student for 1,000 students.Their cost figure study results are:
Classroom $ 8,9 0 0Videotape 482Cassette 4 5 9Workbook 409Videodisc 403Computer-based training 226Interactive Multimedia 132
TERMINOLOGY, OR,CONFUSION d go go
"Multimedia" and "Interactive Multimedia"
are now the catch phrases in the marketplace. There are several other terms,
though, that are also used to designate theseprocesses, or are incorporated in themultimedia processes. These terms are"Hypermedia," "Hypertext," "lntermedia,"and "Imaging." In the interest of somestandard acceptance of the terms, I wouldlike to pick up the following explanationfrom one of the papers in the book,Learning with Interactive Multimedia[AMBR90]:
A note about terminology: As is often thecase in new fields, no consensus existsin the multimedia field about whatvarious terms mean or what to callcertain items that clearly need names."Multimedia," "hypermedia, " and"intermedia" are often usedinterchangeably to describe the samething. In this [paper], I use the term"multimedia" to describe any softwareusing the four essential communicationsmedia -- text, audio, images, and logic-
whether on a media computer or ananalog video-computer hybrid. I use theterm "hypermedia" to describe a subsetof multimedia that has "hyper"characteristics: various modules linkedin a nonhierarchical, nonlinearstructure .... [OSBO90]
Muliimedia and Hypermedia at this pointare nearly interchangeable; any differencemay be particular to an application and inthe nuance. Interactive multimedia refersto the viewer making selections in themultimedia program that causes theprogram to go off in another direction.Intermedia seems to be used very little anylonger.
"Hypertext," in the words of the term'soriginator, Theodore "Ted" Nelson, means"non-sequential writing. Ordinarywriting is sequential But thestructures of ideas are not sequential.They tie together every which way."[NELS87] Mr. Nelson, who coined theterm in 1965, means here a system ofcomputer-supported, nonsequentialinformation processing, primarily, ofcourse, of text. One selects a term or
5
phrase, and sees how the positions of the NASA. There are a multitude of multi-term relate throughout a document or set of mediaJhypermedia developments going ondocuments. The underlying principle is in the transition from traditional linear"the ability to create computer-supported information retrieval to actual informationlinks or cross-references permitting viewing. This is happening in what hasrapid, easy movement" among related parts been termed the multidimensiona/of information in the text or document set fnformation space [SEPEC90]. We already[KAYE91]. see from above how this is changing the
way we store, retrieve, and use infor-"Imaging" is a widely used catch phrase mation. Hyper-branching applications arethroughout the market place. Currently being experimented with throughout thethere are two basic areas involving whole of the government, academia, andimaging technologies-business imaging and pi'ivate-|hdust_,- The_ollowihg-are but'atechnical imaging. Imaging in the business few examples representative of what iscontext refers primarily to automating taking place j_NASA. Included is anbusiness processes and activities. This example of Hypertext application, with themeans basically forms and correspondence, view of showing such activity as abut also extensively including any images cornp6nent _f multimedia support. Theand graphics that support the businessactivities. The forms and correspondenceappear on the screen "imaged" as theyappear in paper form. When c6fin_c_ tofax capabilities, company forms andcorrespondence, if not input into areceptor imaging system, fax out on paperin the originating company'scorrespondence letterhead and format.Technical imaging concerns computer-stored images to support scientific andengineering activities. It is comprised ofeverything ranging from satellite-originated digital data images of earth andspace, and remotely-sensed data, toscanned-in maps, X-ray images, magneticresonance images (MRIs), microscopicimages, you name it.
As a consequence of the multiplicity ofterms, in the various examples following,"Hypermedia" may be correlated to"Multimedia"; "Hypertext" and"lntermedia" may be considered as featuresor effects that are incorporated inmultimedia/hypermedia processes.
So. • •
WHAT'S HAPPENING IN NASA?
descriptive information of the activitiesbelow is extensively verbatim from thepapers cited [KAYE91].
The ExDeriment Documentation
Ii!formatlon _vstem (EDISI. EDIS is beingdeveloped by Houston Applied Logic,Houston, Texas, for the NASA Life SciencesProject Division at NASA Johnson SpaceCenter, Houston, Texas. It is a systemdesigned to prod_uce and control the Lifesciences Expe_riment--DoCument (ED)c0ntaining large a-rno_Jnts of text incombination with tables and graphs ofmathematic a! an_-=s-_ific data, makinguse of hypertext concepts throughMacintosh HyperCard. The ED defines allfunctional Objectives, inflight equipment,consumables, measurements, groundsupport, and test sessions, along with theexpected results of the experiments. TheED consists of 16 chapters plus appendices.There is a fixed, or boilerplate text insome sections that applies to any LifeSciences experiment and reference tableformats concerning experiment-specifictext and mathematical/scientific data.Other sections contain experiment datatailored for each experiment. The EDIS isforeseen as being the first step in theautomation of the process required for
A review of materials cited in the defining complete packages of Life SciencesNASA]RECON lnformatlon system shows experiments for the Shuttle missionsextensive interests across the whole of [MOOR90].
6
Life Sciences Interactive InformationRecall !LSIIR). This is a study in
hypermedia applications, being done by GEGovernment Services, Houston, Texas, forthe Life Sciences Project Division, JohnsonSpace Center. LSIIR, through interactivemedia technologies, provides onlineinformation aids as a "job performanceassistance." The technologies areintegrated into a computer desktopworkstation environment with which
mission or payload specialist, thescientist, the engineer, and support oradministrative people are familiar. TheLSIIR is foreseen as providing assistance inLife Sciences Project missions andactivities such as development and testing,science monitoring, technical labactivities, and mission testing. The systemuses Mac SEs for running integratedapplications of HyperCard, MacRecorderSound System, MacDraw, MacPaint,Canvas, and MacroMind Director.MacroMind Director enhances graphicsdisplay and animation. Clip art andscanned photos are part of the system'sinformation base. The system serves as a"trainer" or simulator. It provides theuser with different sets of information tochange variables during an exercise, ormake alterations to procedures andconfigurations. LSIIR has passed its proofof concept stage, and is envisioned as anonline system for electronic documentationand information, and electronic trainingand revlew in all areas of the NASA Life
Sciences Project activity [CHRIS90].
Knowledge Base Browser (KBB_.Currently under development at the NASAJohnson Space Center is a hypermediasystem for browsing CLIPS knowledgebases. CLIPS is C Language IntegratedProduction System, an expert system shellused in this case to create knowledge baseexpert systems of rules that control theprocesses of the Onboard Navigation(ONAV) flight control position at theMission Control Center (MCC). Theseexpert systems will support the ascent,rendezvous, and deorbit/landing phases of aShuttle misslon. The KBB, as a component
program of the MCC, serves to assist in theverification of the rule bases of the various
expert systems, and to augment thetraining of the flight controllers. Whencomplete, the KBB will verify and browsethe CLIPS rule bases. This system, whichin the view of its creators is a hypermediasystem, will include the capabilities ofautomatic creation of links based on the
CLIPS rule structure, querying the rulesand saving the results as a collection, andbrowsing the rule bases either sequentiallyor by using the links and collections[POCK90].
The Space Station Freedom User InterfaceLanguage !SSF UIL). SSF UIL is indevelopment at the Space Operations andInformation Systems Division of theLaboratory for Atmospheric and SpacePhysics, University of Colorado, Boulder.It is designed for use by the astronauts,ground controllers, scientificinvestigators, and hardware/softwareengineers who will test and operate thesystems and payloads aboard the spacestation. The IUL is object-oriented,English-like, supplements the graphicaluser interface to systems and payloads byproviding command line entry, and will beused to write test and operationsprocedures. Hypertext is used to providelinks between users of code (statements,steps, procedures, etc.) and associatedannotation and documentation, linking codeto object information, and linking stepswithin a procedure [DAVl90].
Artific!ally Intelligent Graphical EntityRelation Modeler (AiGerm_. AiGerm is a
relational database query and programminglanguage frontend for Germ (GraphicalEntity Relational Modeling) system. Thesesystems are being developed byMicroelectronics and Compute TechnologyCorporation, Software Technology Program(MCC/STP), Austin, Texas. There arethree versions of AiGerm in use: QuintusProlog, B!Mprolog, and MCC's Logical DataLanguage (LDL). AIGerm is intended as anadd-on component of the Germ system to beused for navigating very large networks of
7
information, harnessing Prolog or LDL'srelational database query capabilities. Itcan also function as an expert system shellfor prototyping knowledge-based systems.AiGerm provides an interface between theprogramming language and Germ. When auser starts up AiGerm, the system builds aknowledge base of currently loaded Germfolio. The knowledge base is a collection ofnode, link, and aggregate facts. The userqueries the database and runs programsthat select, create, delete, inspect, andaggregate the nodes and links appearing inthe Germ browser. To use AiGerm, the
user first starts up Germ and loads thedesired hypertext network folio into theGerm browser. In a knowledge base, forexample, for each hypertext entity - i.e.,node, link, and aggregate - AiGerm assertsa-fact (a prolog clause). AiGerm iscurrently used in MCC/STP's DESIRE(DESign Information REcovery) system toextract information on the design code forsoftware systems. Research staff areexperimenting with AiGerm in buildingIBIS (Issue Based Information Systems) -reasoning and decision support systems for_,o3_-waredesign and engineering. RockwellInternational, an MCC/STP shareholder,uses AiGerm in a simultaneous engineeringproject. MCC/STP states that users ofAiGerm can navigate Germ Networks ordevelop prototypes of knowledge-based
h_rmeclia s_,_tems [HASH90 ].
Climc_csl Practice L_rarv of Medicine
the spacecraft physician with nearlyinstantaneous access to the most completemedical references on Earth. With this
type of support system, the physician couldbe confident that he was making the rightdiagnosis. The demonstration CPLM systemthat is available now runs on an IBM PS2
Model 80 microcomputer with a highresolution 8514A Display and a 1 gigabytedisk drive. The system is programmed in Cunder Microsoft Windows. The systemcontains a variety of medical textsincluding the STI Program's specialpublication NASA SP-3006, the"Bioastronauiics Data Book." The CPLM
system is written to allow expansibility tothe full capacity of the available storageaevice. Both tra_al and hypermediaaccess to the information is permitted.Traditional Boolean search methods areenhanced by a parsings dictionary unique toeach book that holds current spellings androot word divisions along with a lexiconthat provides a book specific list ofsynonyms and abbreviations thatautomatically provides alternate searchterms to the user. Word and phraselinkage among all documents is providedinitially by the University of Floridaproject team with annotations to beeventually added by the physician endusers. The educational capability of theCPLM system may be one of its majorbenefits in addition to-its ability to delivercomplex information in a user-friendlyfashion.
L._. The CPLM was conceived in1979 by a team of medical and computer The University of Florida developmentexperts from the University of_Florida and team.... leader stresses that the plannedKennedy Space Center. Since itsonset,_t_he addition of voice ac_ation, animation, andsystem has evolved from a mainframe- interactive hardwai'e can make the CPLMbased text database to a microcomputer- system function-asa fully automatedbased hypermedia system that supports physician's assistant, in a few years aboth text and high-resolution medical miniaturized hypermedia CPLM system isimages. The design changes necessary to foreseen as being built into space suits andexpand the system to include sound and carriedby Earthly physFciahS in theiranimation are now being delineated, black bags [GRAM91].
The CPLM system is currently a Virtual Interface Environment Workstationcomputerized, rapid-reacting, medical _VlEWL VIEW is a NASA development withreference system that could be placed high visibility, that is based on Virtualaboard a long-term space flight to provide Reality technology. Virtual reality is "the
8
creation of highly interactive, computer- ACADEMIC ACTIVITIES: TWO MAJORbased multimedia environments in which EXAMPLESthe user becomes a participant with thecomputer in a 'virtually real' world"[HELS91]. In the words of the founder andfirst Director of the VIEW Project:*
In the Aerospace Human Factors
PROJECT EMPEROR-I. This is a well
known hypermedia project, mergingmicrocomputer and videodisc hybridtechnologies. It has been ongoing since1984. It is a major research and
Research Division of NASA's Ames development project which demonstratesResea_rc_hCenter, an interactiveVirtual how new technologies enhance betterInterface Environment Workstation _nderstanding and appreciation of a(VIEW) has been developed as a new kind subject, in this case Chinese humanities,of media-based display and control by delivering a large-scale online (real-environment that is closely matched to time) hypermedia, multi-formatted, andhuman sensory and cognitive multi-dimensional information simply notcapabilities. The VIEW system provides possible in sequential-formatted systems.a virtual auditory and stereoscopic The current hypermedia system includesimage surrounding that is responsive to an interactive information delivery modelinputs from the operator's position, for providing, at rapid speeds measured invoice and gestures .... this variable fractions of of a second, requested relevantinterface configuration allows an information in any format - visual, audio,operator to virtually explore a 360- textual - as selected by the viewers atdegree synthesized or remotely sensed their pace and choice, including at the pointenvironment and viscerally interact of need. The project now includes:
with the components.• Two 12 inch NTSC CAV videodiscs,
The current [VIEW] system consists of a entitled "The First Emperor of China: Qinwide-angle stereoscopic display unit, Shi Huang Di."glove-like devices for multiple degree- ° Interactive courseware, at both a layof-freedom tactile input, connected public and a serious researcher levels.speech recognition technology, gesture Prototype courses have been developed fortracking devices, 3-D auditory display Digital Equipment Corporation's IVISand speech-synthesis technology, and systems and for IBM PC compatibles. Latercomputer graphic and video image systems now include the Apple Macintoshgeneration equipment. Mac IIs.
• Electronic image databases for IBMWhen combined with magnetic head and compatibles and Mac IIs. Furtherlimb position tracking technology, the development efforts have taken place withhead-coupled display presents visual SOPHIATEC, Nice, France, and with theand auditory imagery that appears to Project Athena of Massachusetts InstituteCompletely surround the user in 3-D of Technology involving a powerfulspace .... [FISH91] multimedia image system using DEC's
proprietary MUSE software for high-endmachines such as DEC's MicroVax and IBM
RTs. The EMPEROR-I hypermedia systemhas also been looked at for use on Sun3s andSun4s.
• Scott S. Fisher, Project Director, 1985-90, ° High resolution imaging digitizationfollowing which he jointly founded Telepresence and electronic imaging has been performed
on a Sun3-160 using OASIS software.Research. Mr. Fisher holds an M.S. in Media ° Converting and creating large textualTechnology from MIT. (Source: Virtual Reality files with images and Chinese characters[HELS91])
9
using MicroTek's MSF-3000 imagescanner and INOVATIC's Readstar II Plusoptical character recognition software.Digital textual files are kept in the harddisks, but when the data approaches 400-500 megabytes, CD-ROM can be produced.
This project, housed at Simmons College,Boston, Massachusetts, and aided by manyinterested resources, both industry andacademic, is a masterful development. Thegoal of the project's director, Professor
storage sources of the information, theIntermedia development incorporated twonew concepts in the handling of theinformation, the "anchor" and the "proxy."The anchor concerns maintainingconsistency across the applications; ananchor is a specific selection of data, a partof a document, with the surroundinginformation used to understand itssignificance. When a user follows a link,the document window opens to the size andlocation on the screen most recently saved,
Ching-chi Chen, is to show that computer and automatically scrolls to the_section thatpower, storage technology, and software reveals the anchor with its surroundingare now all available, at affordable cost, to information. The proxy is an intermediaryprovide the opportunities for innovativeexperimenta_i0n of ideas in education,training, research and development innearly every subject field [CHEN88,CHEN90].
_. Brown University's Institutefor Research in Information and
concept used by the viewer for selecting ananchor in disparate data sources, e.g., text,graphics, sound. The use of the data proxyconcept allows the viewer to visualize non-graphical and conceptual media, to havesimplicity in linkin _ mecliaj and to extendsystem applications to related data types[CATL88].
Scholarship (IRIS), Providence, Rhode _/- _Island, has developed a powerful multi- 21st CENTURY FUTURES - THEY AREuser hypermedia software that allowsprofessors, students, and other knowledgeworkers to create and follow links betweenelectronic documents for different types.This system is named "lntermedia." Thisproject defines hypermedia as the dynamiclinking of data such that related data iseasily accessible although the actual piecesof data may be stored in different physicallocations. In theory the data can be anytype, such as text, graphics, spreadsheets,video, or audio. Intermedia provides adesktop environment similar to that foundon the Macintosh. The desktop contains
HERE - OR IF NOT, THEN NOT MORE
THAN 10 MINUTES AHEAD OF THETRUTH _
There is so much technology on the boardsand coming up that the subject of futures isin itself expansive, much more than just inthe imagination. Science fiction in today'sterms must be more than outlandish,because a good part of yesterday's sci fi ishere. Examples: the following appear inThe Futurist, Sep{emloer-October 1990,Column: "Tomorrow in brief" --
applications (or tools) such as a word _ • A natural lappearance prosthesisprocessor, a structured graphics editor, a (e.g., artificial leg) made of carbon fiberhistorical timeline editor, a scanned-image and contained in a soft continuous coveringview, an animation editor, a videodisc gives amputees greater mobility to thecontroller, and a viewer that displays androtates three dimensional models. Users
(now termed "viewer" or "authors"), withthe tools just enumerated, enter data andlink significant items of Informationtogether for a contextual viewing of thatinformation.
Because Of the extensive differences in the
extent that they may take part in strenuousactivities such as rock climbing, cycling,and squash.
• Customized computers are makingmore and more inanimate objects intosmart appliances, tools, and toys. Anexotic example is the "skidometer", amicroprocessor embedded in skis to
10
i
i
I
=
BK=
w
measure speed and elapsed time on theslope. It is expected that major futuregrowth of microprocessor applicalions willbe in household appliances. Suchapplications will cause the use of thesedevices to grow from about 300 million in1989 to 500 million in 1995.
• A recent OTA report providesrecommendations on reducing thetremendous costs of spacecraft.Alternatives suggested are:
Fatsats - spacecraft made of heavier,cheaper materials.
Lightsats - smaller spacecraft withfewer capabilities, thereby costing less tobuild and launch.
Microspacecraft small, ruggedsatellites containing very smallinstruments, which can be launchedseveral at a time from cannon-likelaunchers.
• The Paris Transit Authority - RATP- has begun testing a new train it hopes tobe the "metro of the year 2000." It is notsectioned off into single cars, but a long,single train, nicknamed "The Boa,"allowing its 890 passengers to walk fromone end to the other. It is expected toreplace all first generation, non-pneumatic trains in the Paris metrosystem between 1998 and 2010.
• Seiko Epson Co. Ltd. is producing ahand-held device, about the size of a TVremote control, that translates Englishwords into Japanese. You run it across apage; it looks up each word and matches theEnglish words to 30,889 Japanese words.(Product: Epson TRAN PRO-1000.)
• A compact, desktop-size robot armcould help students jot down notes andconduct chemistry experiments. Developedby Systems Control of Middlesborough,England, il can be operated through anIBM-compatible computer and usesstandard, pretaught sequences. Students
can teach the robot new sequences thoughthe robot's keypad or the computerkeyboard.
An item of note, of interest to NASA people,concerns the development of solarsailcraft. In the column "Future scope" ofthe same issue the following appears:
Solar Sailcraft Will Race to Mars - To
honor Columbus, racers set sights on a"new world" - Solar sailcraft from sixnations will compete in a race to Mars in1992 to commemorate the 500thanniversary of Christopher Columbus'sdiscovery of the New World .... Thesails, which fly on the flow of photonsfrom the sun, will be of various shapesand sizes .... Most sails will be made ofmaterial that is thinner and lighter thanplastic food wrap. Folded into smallcanisters, the sails will be launched byrocket into high Earth orbit, where theywill unfurl. Source: U.S InternationalSpace Year Association ....
Let's narrow the field to the mystique ofinformation and its conveyance throughmultimedia.
Electronic City Hall- You will register tovote, sign up for racketball, and lodge acomplaint with the mayor, at an electronickiosk. Currently, in Mercer Island,Washington, an electronic informationbooth, called "The 24-Hour City Hall," hasbeen set up near a grocery store to givecitizens 24-hour access to their municipalgovernment. Developed by PublicTechnology, Inc. (PTI), it uses an IBMmicrocomputer and multimedia technologyto give information on town meetings andlocal government services. Moreelectronic city halls are being set up inother U. S. communities. (The Futurist,September-October 1990, p.6)
Computer with "Brain Waves" - an IBMscientist and colleagues at ColumbiaUniversity designed a supercomputermodel for studying the area of the braincalled the hippocampus, an area associated
11
with epilepsy. The model unexpectedly developers program applications in the Cbegan producing its own electrical waves, language. (Verbum, p.44)surprisingly similar to the brain waves ofthe hippocampus. If this is an accurate • Video- ]_gLV.JJ;JP_ (for the Mac). Aappssimulation, "experiments can be done on Corp., Sunnyvale, CA. This color real-the model as if it were an organism in its time video board allow the user to displayown right." (The Futurist, September- and capture 24-bit images for actualOctober 1990, p.6) computer processing, whether in
teleconferencing or multimediaFrom the magazine Verbum,* Winter applications. The board-includes an9 0 / 91 • improved tuner and supports 98 channels.
It can also display in black a_d white mode• Mind Ware MC?,- and _ using 128 gr_)}_ievels. _fsb contained in ,!
the package is _ced Publisher's i(for the IBM). Light & Sound Exposure, a color image_grabber and color !
Research, Pasadena, CA and Synectic !Systems, Seattle, WA. These are two paint CDEV. (Verbum, p.47)bra_-wav-e Controilers, _ w_5 _ acombination of special glasses and ° HyperMedia - ADOmotion (for the Mac). °earphones that deliver strobe lights and Motion Works, Van£ouver, BC, Canada.sounds to entrain users' brain rhythms, ADDmotion adds modules to HvperCard 2.0The machines are based on the same that allow _b-sers-_t_create anfrnatedresearch that associated different brain sequences. Theso-ftware lets you edit andwaves with various states of consciousness. Create full--c-o[or pictures, animate them
Each can induce either the alpha state for and add four-track sound. Object-oriented __quick relaxation or a body-relaxed but and interactive animation allow flexible -_brain-alert theta state for enhanced animation production. Cues in ADDmotion
/earning via cassette tape. MC 2 by Light & are passed back to _, providingSound Research provides a microchip with complete interaction _ the _ten preset light/sound combinations built programming language. (Verbum, p.42)
z
into a controller box that its goggles andearphones hook into. It does not link with a ° Verbumalia: Cool Stuff To Orderpersonal Computer n " n r Verbum Interactive The first edition ofsyste_ a .d__/s __t use ................... --programmable. Synectic Systems' the Verbum CD-ROM. T:tequt:res _irFtSshMindsEve Syneraizer provides a PC board II with 5 MB RAM and CD-ROM drive.that fits into a slot on an IBM Inc_redible inte-raciive magazine in ColorPC/A T/XT/386 or compatible, along with and sound with-a_G-a=llery of animation andgoggles, headphones and software. The multimedia works . . . software demos,
specific light/sound combinations are ifiteractive article_ an_ _m_cfi morel iprogrammable by the user. In fact, each $49:95 (verbuml p.60} :The demoeye and ear may be programmed programs are in parLad_grtiser s in the -=
magazine. A two-disc set in interactiveindependently. A separate kit lets format. The music on-the discs wilialso -
play on a regular audio C D player.
Verbum 4.3, Journal of Personal Computer In addition to some "brave new world"Aesthetics, Winter 90/91. "This is the technologies I have tried to give a small _=
: magazine that helped spawn a revolution in the panoply of what's going on in thenew world =world of personal computers by proposing that of multimedia - and have barely scratched =the creative spirit could work its magic through the surface. Not touched here are =silicon, that the human hand and mind could use rnu|timedia technologies such asdigital media in the making of True Art." (Issue Microsoft's Windows and its extensions,cover) enduser-level authoring systems from
12
MacroMind, Authorware, and othercompanies, and IBM's magnificenteducational interactive multimedia
projects "Ulysses" and "Columbus."
AN UNDERSTANDING OF THE ROLE OFMULTIMEDIA IN SCIENTIFIC ANDTECHNICAL INFORMATION
Multimedia is not an end in itself. Its roleis apparent in Grand Challenges: High
will be images of the print material, andwill be textually searchable, i.e.,hypertext searchable. But the base will becomprised basically of images relevant tothe user's information need. The imageswill be both static and motion, viaanimation and video. The base is further
supported by sound -- vocal, noise, andmusical; and graphics programs willpresent, process, and enhance theinformation; graphics in itself is anexpansive and distinct universe of
Performance
Communications, a report issued in 1990by a committee supported by the NationalScience Foundation [CPME90]. Accordingto the committee, challenges facing theUnited States concern high performancecomputing systems, advanced software
Computing and computer processing.
It is thus foreseen that a user's primarysource of working information will be themultimedia system information base thatthe user creates to serve his needs. Thiswill be a local system; information and
technology and algorithms, the creation of a computing resources may be accessedNational Research and Education Network remotely, but such activity will be(NREN), and basic research and human subsidiary in a supporting capacity to theresources. These are the four major local multimedia base.components of the Grand Challengesprogram. They in turn break down further The NASA Scientific and Technicalinto more specific areas to be researched Information Program (STIP) sees the needand developed, for incorporating new technologies into the
handling of STI. As stated by the Program'sMultimedia is a result of and will be Director in the STIP strategic planeffected by the outcome of research and (STIP91), "As we approach the yeardevelopment in all four of the program 2000, there are many excitingcomponents. The very basic purposes of opportunities for revolutionizing the waythe high performance computing program STI is managed, thereby increasing itswill be creation of advanced_ technologies value to NASA, the U.S., and ourfor the collection and storing of international partners."information, the accessing andtransmission of the information, the In summary, I would like to paraphrase anavailability of computer resources (both earlier paper [KUHN89]: The NASAlocal and remote) to process and re- Scientific and Technical Informationprocess the information, and the collection Program is at the threshold of its entryand storage of tailored information locally into multimedia. Because the business ofto fill the needs of the local user. Theinformation will be constituted in all its
possible varieties -- text, numeric,graphic, sensory.
Multimedia means no longer asking forpaper or file output of text and numericdata. Multimedia means building aninformation base that suits its user. The
• base will be comprised ofinformation. If it includes text, the text
STIP is information, there is a vastopportunity for looking at multimediaapplications, assessing their relevance tothe handling and processing of scientificand technical information in the NASAresearch and development community, anddeveloping multimedia implementations fordoing so.
As NASA STIP explores this area moreintensively, we find an increasing critical
13
requirement to coordinate and interface all in this paper is by no means anSTIP multimedia development to assist the endorsement. The companies and productsNASA user in the quest for information, noted simply serve to illustrateMultimedia in all its forms is establishing pragmatically the many multimediathe contextual handling of user-selected, programs and systems in use in privateuser-relevant information. Contextual industry.handling of information, regardless of itssource and form, and approximating the REFERENCEShuman manner of handling information, isthe reason for implementing multimedia [AMBR90] Sueann Ambron and Kristinasystems. Hooper. Learning with Interactive
In doing this paper on the-su]hje(_t ofmultimedia, though, note an irony -- it'sp_per-_ it's without graphics, [trs teletype-style sequential. Yet this is the verge of
MulTi-rfiEc7Ta: Developing and UsingMulEmedia Tools 7n Education. MicrosoftPress, Redmond, Washington. 1990_
[BLAD91] Walter R. Blados. Quo Vadimus?Coming to Grilhs-witff t_ Information World.
the 21st Century. All the capabilities of National Aeronautics and Spacemultimedia expressed in this pa_per are -A_m|nistrati(_n°,--S_:]er_tl-ffic and Techr_Tcal
Information Program, Washington, DC. Maypresent today.
All we need to do is to make the jump, andtoday that's hardly a large one.
§§§
ACKNOWLEDGEMENTS
Ms. Gladys A. Cotter, Director, NASAScientific and Technical InformationProgram (Code JTT), is thanked for herprogressive outlook on the_rmafionworld today, thereby allowing our lookinginto the application of multimedia for theNASA Scientific and Technical Information
Program.
1991, NASA Technical Memorandum105138. --
[CATL88] T. J. O. Catlin, and K. E. Smith."Anchors for Shifting Tides: Designing a'Seaworthy' Hypermedia System." OnlineInformation 88, Proceedings, 1988 (See[ONLINE88]).
[CHEN88] Ching-chih Chen. "HypermediaInformation Delivery: The Experience ofPROJECT EMPEROR I." Online Information
88, Proceedings, 1988 (See [ONLINE88]).[CHENg0] Ching-chih Chen. "Online
Hypermedia Information Delivery."
Mr. Thomas A Wheeler, Multimedia DesignCorporation, Alexandria, Virginia, is
National Online Meeting Proceedings- 1990(See [ONLINEg0]).
[CPMEg0] Committee on Physical,Mathematical, and Engineering Sciences;Federal Coordinating Council for Science,Engineering, and Technology; Office ofScience and Technology Policy. GrandChallenges: High Performance Computing
thankedfor provFding planning and market and C-omEnunications._ The FY 1992 U.S.place information that otherwise would Research and Development Program. Tohave had to be extensively researched. Supplement the President's Fiscal Year
1992 Budget. [1990]. Available fromMs. Karen M. Kaye, of the STIP Office (Code Committee on Physicall Mathematical, andjTT), is thanked for leading the way with Engineering Sciences, c/o National Scienceher extensive knowledge in multimedia F_t[onlWas_ngton, DC.technologies. [CHRIg0] Linda A Christman, Nam V. Hoang,
and David R. Proctor. "Life Sciences On-CAVEAT Line- A- study in Hyperme_a. _Application.*
SEPEC Conference Proceedings, I990 (SeeThe mentioning of companies and products [SEPECgO]).
14
_=
Z
[DAVI90] Randal L. Davis. "Hypertext As a [NELS87] Ted Nelson. Dream Machines (...
Model for the Representation of Computer the flip side of) .C¢gJ3J2g._Z_. Tempus
Programs." SEPEC Conference Books of Microsoft Press, Redmond,
Proceedings, 1990 (See [SEPEC90]). Richmond. 1974, 1987.
[FISH91] Scott S. Fisher. "Virtual [ONLINE88] Online Information 88, 12Environments: Personal Simulations & International Online Meeting, London 6-8
Telepresence." Virtual Reality: theory, December 1988. Learned Information
practice, and promise. Ed. by Sandra K. (Europe) Ltd., Oxford, England.Helsel and Judith Paris Roth. Meckler [ONLINE90] National Online Meeting,
Publishing, Westport, Connecticut. 1991.
[GRAM91] Ralph R. Grams. Final Report forNAG10-O041. 1990-1991, Clinical
Practice Library of Medicine - Librarian
and Tutor. Volume I. University of Florida,
College of Medicine, Department of
Pathology, Medical Systems Group,Gainesville, Florida, 1991.
[HASHg0] Safaa H. Hashim. "AiGerm: A Logic
Programming Front End for Germ." SEPECConference Proceedings, 1990 (See
[SEPECg0]).
[HELS91] Sandra K. Helsel and Judith ParisRoth. Virtual Reality: theory, practice, and
promise. Meckler Publishing, Westport,Connecticut. 1991.
[KAYE91] Karen M. Kaye and Allan D. Kuhn.
"Hypertext and Hypermedia in InformationRetrieval." Bringing Down the Barriers toInformation Transfer/L 'Abaissement de
Barri_)res s'Opposant au Transfer de
Proceedings -1990, New York, May 1-3,1990. Learned Information, Inc., Medford,
New Jersey.
[OSBO90] Hugh Osborn. "Media Computers,Motivation, and Informal Education:
Gutenberg 2000?" Learning withInteractive Multimedia: Developing and
Using Multimedia Tools in Education.Microsoft Press, Redmond, Washington.
1990.
[POCK90] Tony Pocklington and Lui Wang. "A
Knowledge Base Browser Using
Hypermedia." SEPEC Conference
Proceedings, 1990 (See [SEPECg0]).
[SEPEC90] SEPEC Conference Proceedings.
Hypermedia & Information Reconstruction:
Aerospace Applications & ResearchDirections, December 3-5, 1990, Houston,
Texas. University of Houston-Clear Lake,
NASA/Johnson Space Center, Hypermedia
Working Group.
I'lnformation. Specialists Meeting, [STIP91] A Strategic Direction. NASATechnical Information Panel, Advisory Scientific and Technical Information
Group for Aerospace Research and Program (Code JTT), Washington, DC.
Development (AGARD), Held at the Instituto 1991.
Nacional de Industria, Madrid, Spain, 8-9 [WALS91] Randal Walser. "The Emerging
October 1991; pp. 6-1 to 6-5. Technology of Cyberspace." Virtual
[KUHN89] Allan D. Kuhn. Hypermedia: Our Reality: theory, practice, and promise. Ed.
Entry into the Intertwing/ing Zone. Defense by Sandra K. Helsel and Judith Paris Roth.
Applied Information Technology Center, Meckler Publishing, Westport, Connecticut.
Alexandria, Virginia. DAITC Hypermedia 1991.
Laboratory Report Number 5, August 1989. [WHEE91] Thomas A. Wheeler, Multimedia
Available from NTIS, AD-212 604. Design Corporation, Alexandria, Virginia.
[MOORg0] Jane Moorhead and Henry Brans. Private Communication, June 18, 1991."Experiment Document Information System Benefits of Multimedia and sample business
(EDIS) Evolution." SEPEC Conference plan for multimedia program.
Proceedings, 1990 (See [SEPECg0]).
15
=
±
_ 7 :
i
R
.L
REPORT DOCUMENTATION PAGEForm Approved
OMB No. 0704-0188
Pub ic repOrting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources,
gathering and maintaining the data needed, and completing and reviewing the collection of informatiOn, Send comments re<jarding this burden estimate or any other aspect of this
collection of information, including suggestions for reducing this burden, to WashingtOn Headquarters Services, Directorate tot information Operations and Reports, 1215 Jefferson
Oavi$ Highway, Suite 1204. Arlington, VA 22202-4302, and to the Office of Management and Budget. Paperwork Reduction Pro ect (0704-0188), Washington, DC 20503.
1. AGENCY USE ONLY (Leave blank) 2. REPORT DATE 3. REPORT TYPE AND DATES COVERED
Nov 91 Technical Memorandum
4. TITLE AND SUBTITLE 5. FUNDING NUMBERS
Quo Vadlmus? The 21st Century and Multimedia
6. AUTHOR(S)
Allan D. Kuhn
7. PERFORMING ORGANIZATION NAME_)AND ADDRESS(ES)Scientific and Technical Information Program
NASA Office of Management Systems and Facilities
NASA
Code JTT
Washington, DC 20546
g. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES)
8. PERFORMING ORGANIZATIONREPORT NUMBER
NASA TM-I07563
10. SPONSORING / MONITORINGAGENCY REPORT NUMBER
11. SUPPLEMENTARY NOTES
12a. DISTRIBUTION/AVAILABILITY STATEMENT
Unclassified - Unlimited
Subject Category 82
12b. DISTRIBUTION CODE
13. ABSTRACT(Maximum2_ words)This paper relates the concept of computer-drlven multimedia to the NASA
Scientific and Technical Information Program (STIP). Multimedia is defined here
as computer integration and output of text, animation, audio, video, and graphics.
Multimedia is the stage of computer-based information that allows access to
experience. This paper also draws in the concepts of hypermedia, intermedia,
interactive multimedia, hypertext, imaging, cyberspace, and virtual reality.
Examples of these technology developments are given for NASA, private industry,
and academia. Examples of concurrent technology developments and implementations
are given to show how these technologies, along with multimedia, have put us at
the threshold of the 21st century. The STI Program sees multimedia as an
opportunity for revolutionizing the way STI is managed.
114. SU_ECTTERMSMultimedia, Hypermedia, Computer integration, Computer output,
Technology, Experience, Information Management, Scientific andchnlcal InfoKmaCion STI, Scientific and Technical Informa-
ion _ro_ram t _i_17. SECURITY CLASSIFICATION 18. SECURITY CLASSIFICATION 19. SECURITY CLASSIFICATION
OF REPORT OF THIS PAGE OF ABSTRACT
Unclassified Unclassified Unclassified
NSN 7540-01-280-5500
15. NUMBER OF PAGES18
16. PRICE CODE
20. LIMITATION OF ABSTRACT
UL
Standard Form 298 (Rev 2-89)Prescrd3ed by ANS_ Std z3g-IB
298-102
7_:T
2 7
.......... k
