Information Enhancement of Synthetic Environments
Nicholas F. PolysVirginia Bioinformatics Institute
Virginia Tech Computer Science &
Center for Human Computer Interaction
Presentation
• Introduction
• HCI & Usability Engineering
• Integrated Information Spaces
• Research:– CML -> X3D– CML -> Snap + Diverse– Biological Simulation in PathSim
What is HCI?
• A multidisciplinary science of the interface: psychology, design & media, human factors, sociology, computer science
• Experimental methods to rationalize UI features, design, and software architecture
Usability Engineering
Where the rubber meets the road…• Scenario-Based Design:
– Activities– Information– Interaction– Claims analysis
See:ROSSON, M.B. AND CARROLL, J. 2002. Usability Engineering:
Scenario based development of Human-Computer Interaction. NY,
Morgan Kaufmann.
Problem: Integrated Information SpacesResearchers, designers, students, and analysts
need to explore, search, recognize, and compare patterns across a wide variety of data types and applications.
Some representations are better suited to different data types and questions…
Fundamental Information Types
– Abstract information • -> Information Visualizations
– Perceptual or spatial information • -> Virtual Environments
– Temporal Information
User Problem:
• Perceptual, environmental phenomena alone do not guarantee accurate mental models
• Interfaces for information and controls are consistently under-represented and poorly integrated in VEs
Developer Problem:
• Increasingly, objects rendered in VEs have metadata and annotations associated with them. These may change over time.
• Lack of design guidelines and information architectures for efficient management of Integrated Information Spaces
Approach: Information-Rich Virtual Environments (IRVEs)
IRVEs are at the union of VE and Info Vis:perceptual + abstract information is simultaneously accessible and linked
• Define the issues and the task space for this class of applications
• Define the design space to solve and support them• Apply usability engineering methods to identify
tradeoffs and enumerate design guidelines
Some Research Questions:
Where, how, and when should the abstract information be displayed in the VE?
In what form should the information be displayed?
How can interrelationships between the spatial and abstract information be represented?
IRVE Activity Design
See also : Bowman, Doug, North, Chris, Chen, J., Polys, N., Pyla, P., Yilmaz, U., (2003) “Information-Rich Virtual Environments: Theory, Tools, and Research Agenda”. Proc. Virtual Reality Software and Technology (VRST)
Activity Task Example
Exploring Overview data loading, choosing the data and display method
Finding Details-on-demand
pop-ups, queries, lenses, zooming via 3D navigation etc.
Pattern Recognition
Relating, Comparing
rendering associations; brushing and linking
Task-Knowledge Structure
e.g. for Situational Awareness and decision support systems…
[Entity Relationship diagrams per task and media]
• SUTCLIFFE, A. AND FARADAY, P. 1994. “Designing Presentation in Multimedia Interfaces”, Proc. CHI '94, 92-98.
Research Thrust:
Data Models behind IRVEs
CML via XSLT
CML via XSLT
CML is a good example of an IRVE database
POLYS, NICHOLAS F. 2003. “Stylesheet Transformations for Interactive Visualization: Towards a Web3D Chemistry Curricula”, Proceedings of the Web3D 2003 Symposium, ACM SIGGRAPH.
http://www.3DeZ.net/X3D/CML
Research Thrust:
• Interactions and Architectures for IRVEs
Snap2Diverse: Coordinating Information Visualizations and Virtual Environments
• DIVERSE is an multi-platform open-source VR toolkit [diverse.sourceforge.net]
• Snap-Together Visualization is a free toolkit for information visualization [infovis.cs.vt.edu]
Multiple, Coordinated Views for IRVEs
• Users can index to and from perceptual and abstract information through ‘brushing and linking’– Diverse as a Snap-able component– Unique Ids– Event communication (e.g. select, load)
Snap Coordination
Manager
É
Component 1
CAVE Adapter (RMI Messaging
Events)
DIVERSEAtomView application
Java Component
Adapter Component 2
System overview
Application: Cheminformatics
Snap2Diverse Video
POLYS, N. F., NORTH, C., BOWMAN, D. A., RAY, A., MOLDENHAUER, M., AND DANDEKAR, C. 2004. “Snap2Diverse: Coordinating Information Visualizations and Virtual Environments”.
Proc. SPIE Conference on Visualization and Data Analysis.
http://csgrad.cs.vt.edu/~npolys
Usability Evaluation Summary
• Snap2Diverse as tested is:– high-threshold for learnability– high-ceiling for functionality
• Users were able to brush&link to solve tasks• In most cases, users chose suitable visualizations to
recover the information required for the finding and comparing tasks
• Given a choice, users tended to recover information from the VE instead of InfoVis
• Xwand is a moded interface and switching between 2D and 3D navigation and selection was not intuitive.
Lessons Learned
Simple events provide ‘glue’ to support complex functionality:
• Benefits of exposing VE scenegraphs to external events; enabling technology must be extended
• Benefits of composable and integrated information spaces
• Designers must consider tasks and the knowledge required for completion (ie Sutcliffe and Faraday, Shneiderman)
Research Thrust:
Objects and Patterns for IRVEs • X3D Event Utilities
• MFSequencers
• Information Panels
• Semantic Objects
• HUDs
PathSim Demo
http://www.vbi.vt.edu/~pathsim
Paper to be presented at Web3D 2004:
Polys, N., Bowman, D., North, C., Laubenbacher, R., Duca, K., (2004). “PathSim Visualizer: An Information-Rich Virtual Environment for Systems Biology” Proceedings of Web3D 2004, ACM SIGGRAPH.
Summary
• There is an increasing demand for information-rich functionality across a wide variety of domains
• A systematic HCI approach to design problems of Integrated Information Spaces is required
• Goal is to understand tradeoffs of design and software architectures to build better interfaces
Future Work
• IRVE Testbed construction• Controlled experiments and usability
evaluations to identify the significant design dimensions and techniques for integrated information spaces.
• Identify tradeoffs, guidelines, and design patterns for this class of problems.
• Propose future X3D standard components:– Annotation (labels, metadata)– Compositing (HUD, ApplicationTexture)