PowerPoint Presentation

VISTED: A Visualization Toolset for Environmental DataAdvisor:Dr. Sergiu DascaluCommittee:Dr. Valerie FridlandDr. Fred HarrisDr. Yaakov VarolDr. Yantao Shen

Likhitha Ravi

IntroductionBackgroundRequirements ArchitectureResearch PlanConclusions

VISTEDIntroductionBackgroundRequirements ArchitectureResearch PlanConclusions

VISTEDCyberinfrastructure (CI) developments are part of an NSF EPSCoR project (2008-2013, cca $21.7 million)Focused on climate change (CC) research, education, and policy making in NevadaSix project components: climate modeling (air) water resources (water)ecological change (land) education cyber infrastructurepolicy, decision making and outreach

Nevada Climate Change ProjectThe projects major goals:

Create research capabilities to add value to the existing R&D resources Establish unique positions in focused research fieldsIncrease inter-institutional and interdisciplinary collaborations

Nevada Climate Change ProjectResearch focus: The effects of regional climate change on ecosystem resources

Major interdisciplinary science questions:How climate changes affect water resources and linked ecosystem services and human systems?How will climate changes affect disturbance regimes (e.g., wildland fires, insect outbreaks, droughts) and linked systems?

Nevada Climate Change ProjectCyber Infrastructure (CI) goals: Facilitate interdisciplinary climate change research, education, policy, decision-making, and outreach by using CI to develop and make available integrated data repositories and intelligent, user-friendly software solutions

Nevada Climate Change ProjectEnvisioned in the NSF EPSCoR project proposal 2008

Nevada Climate Change Project

CI outputs: Nevada Climate Change Portal (NCCP) Software tools for climate change research, outreach and education: software frameworksIntegration and interaction across project and among CI groups within the 3-State Western Consortium: facilitator of collaboration

Nevada Climate Change ProjectNCCP provides the climate data online to help researchers working on climate change all over the globe.Why do we need data visualization?Although most of the climate related data is easily available on the World Wide Web, it is a complex and demanding task to analyze very large datasets without the help of visualization.

NCCP10Uses of visualizationPresenting the results in a comprehensible manner for decision makers, stakeholders and general public. Evolution of climate models.Verification of hypotheses.Data exploration in order to find the trends and patterns. Data VisualizationVISTED mainly helps the climate researchers by visualizing the datasets over the web.

The users of the VISTED are researchers, educators, students, policy makers and general public.

VISTED12Research QuestionsWhat specific visualization techniques and displays can increase the efficiency of the environmental scientists?

What mechanism for integrating data extraction, conversion and visualization are most beneficial for the environmental scientists work?

What are the challenges facing researchers in the field of data visualization?

VISTEDSignificant features of VISTEDData VisualizationData DownloadData ExtractionData Conversion

Capabilities of VISTEDHandling several input data formats such as Network Common Data Form (NetCDF), Comma-Separated Values (CSV), American Standard Code for Information Exchange (ASCII) and Hierarchal Data Format (HDF5).Providing different kinds of visualizations such as line chart, bar chart, bubble chart, and many more.VISTED14New capabilitiesA web based tool for climate researchers, students, educators and general public.Uploading datasets from users machine.Reading input from several data formats such as NetCDF, CSV, ASCII and HDF5.Extracting NetCDF, CSV, ASCII and HDF5 datasets.Converting into different data format.Introducing new visualization techniques to the climate researchers.

VISTEDIntroductionBackgroundRequirements ArchitectureResearch PlanConclusions

VISTEDTable 1: Matrix representing the features of visualization tools

# Tool NameOperating system supportVisualization TechniquesProgramming/ Scripting languages# of variables1ArcGISMicrosoft Windows, Linux,Sun SolarisMap (MXD), Globe, Geoprocessing, Geocoding, Network Analysis,Geodata , MobileVBA, VB, .NET, Java, C++, COM, Python, VBScript, JavaScript, ASP, JSP, ColdFusion, Java, .NET, JavaScript, XML, FLASH, PHPMultidimensional data 2AVS/ExpressWindows, Mac OS X, Linux, Solaris, and HP-UX, IRIX and Alph Tru642D line field plots, Gamma plot, 3D shaded,contour, and arrow field plots, Animations, particle tracing using stream lines and streak lines, isosurfaces, Volume VisualizationC, C++, and FORTRAN.2D, 3D, univariate,multivariate data3FerretUnix systems, and on Windows XP/NT/9xGeophysical formatting, symmetrical processing. Ferret Scripts 3D, 4D, Multidimensional data4GGobiWindows, Mac, UnixHistogram, textured dot plot, barchart, spineplot, Scatterplot, parallel coordinates, time series plotGgobi scripting3D, Multivariate data5Google Visualization APIWindows, Mac, Unixpie chart , Scatterplot, Guage, geo chart, bar chart, tree map, bubble chart, line graph, stack graph, , combo chart, column chart, area chart, candlestick chart, word cloud generator, and maps.Javascript2DSample VisualizationsAVS/Express

Terrain and WeatherWind ModelingSource: http://www.avs.com/products/avs-express/gallery.htmlArcGIS

Sample Visualizations

Impacts of Sea Level RiseClimate changeSource: http://www.esri.com/library/ebooks/climate-change.pdfTable 1: Matrix representing the features of visualization tools

# Tool NameOperating system supportVisualization TechniquesProgramming/ Scripting languages# of variables6GrADSLinux, Mac OS X, Windows, Solaris, IBM AIX, DEC Alpha, IRIXline and bar graphs, scatter plots, smoothed contours, shaded contours, streamlines, wind vectors, grid boxes, shaded grid boxes, and station model plotsFORTRAN, GrADS scripts5-dimensional7Integrated Data Viewer (IDV)Windows, Linux, Solaris (SPARC and x86), Mac OS-XCharts, maps, radar displays, gridded data displays, isosurfaces, volume rendering, globe display, plan view, profiler windsJava3D, multi-dimensional data8MathematicaWindows, Mac, Unixpolar and spherical plots, contour and density plots, parametric line and surface plots, and vector, stream plots, candlestick charts, quantile plots, box whisker charts, Bode plots, histograms, 2D and 3D bar charts, pie charts, bubble charts, B-spline curves in 2D or 3DC++, Java, .Net, FORTRAN, CUDA, OpenCL2D, 3D9MatlabLinux, Microsoft WindowsLine, area, bar, pie charts, Histograms, Scatter/bubble plots, Animations, Direction and velocity plots, isosurfaces, Volume VisualizationC, C++, and Fortran.1D,2D, 3D visualizations10 OpenDXWindows, Mac OS X, Linux, Solaris, and UnixAnimations, Direction and velocity plots, isosurfaces, Volume VisualizationC, FORTRAN and Visual Basic2D, 3D, univariate,multivariate dataSample Visualizations

Grads Temperature ForecastIDV view of Hurricane CharlieSource: http://wxmaps.org/pix/temp5.htmlSource: http://www.unidata.ucar.edu/software/idv/docs/userguide/index.htmlTable 1: Matrix representing the features of visualization tools

# Tool NameOperating system supportVisualization TechniquesProgramming/ Scripting languages# of variables11PrefuseWindows, Mac, UnixArea chart, Bar chart, Pie chart, scatter chart, line graph, Tree map, network diagram and animationsJava2D12RWindows, Mac OS X, Linux and UnixGraphs, traditional statistical tests, time series analysis, linear & nonlinear modeling, classification, clusteringC, Python, Perl 3D13S-PLUSWindows, Linux, UNIX, SolarisGraphs, linear & nonlinear modeling, classification, clusteringFORTRAN,C, S3D14SPSSWindows, Mac, and LinuxTables, graphs, linear regression, cluster analysis, and non-parametric testsJava, Python, SaxBasic2D15TableauWindowsScatterplot, matrix chart, bar chart, area chart, bubble chart, stack graph, pie chart, link map and spatial mapsNo programming or scripting required2D, univariate, multivariate dataSample VisualizationsR-Statistical PackageTableau GallerySource: http://www.r-project.org/Source: http://www.tableausoftware.com/learn/gallery

Table 1: Matrix representing the features of visualization tools

# Tool NameOperating system supportVisualization TechniquesProgramming/ Scripting languages# of variables16UV-CDATMac, Linuxmulti-view visualization, Direction and velocity plots, isosurfaces, Volume Visualization, and parameter space explorationPython, C/C++,Java, FORTRAN3D, multi-dimensional data17VisTrails Windows, Mac, Linuxmulti-view visualization, Direction and velocity plots, isosurfaces, Volume Visualization, and parameter space explorationPython3D, multi-dimensional data18VisItWindows, Mac, Linux, Unix, AIZ, Solaris, Tru64, IRIZContour 3D, Pseudo color plot, Contour 3D, volume plot, vector plot, subset plot, molecule plot, parallel axis plot Python3D, multi-dimensional data19Visualization toolkit (VTK)Windows, Mac, Unixscalar, vector, tensor, texture, volumetric methods, implicit modeling, polygon reduction, mesh smoothing, cutting, contouring, and Delaunay triangulationC++3DSample VisualizationsVis Trails GalleryVisIt GallerySource: http://www.vistrails.org/index.php/File:Screen_Shot_2012-01-12_at_2.50.19_PM.pngSource: https://wci.llnl.gov/codes/visit/gallery.html

NASA (National Aeronautics and Space Administration)* Provides data extraction.* Data can be downloaded in several formats.- No data interaction.

NOAA (National Oceanic and Atmospheric Administration)* Supports data interaction.* Provides data extraction.- Data can be downloaded only in ASCII format.

Cal-adapt* Supports data interaction.- Cannot change visualization technique- Does not support data conversion.

Many eyes* Supports several visualization techniques.* Allows users to upload data -Supports only CSV and ASCII file formats.

Related WorkSample Visualizations

Source: http://mynasadata.larc.nasa.gov/NASASample Visualizations

Source: http://www.climate.gov/#climateWatchNOAA Sample Visualizations

CAL- AdaptSource: http://cal-adapt.org/temperature/decadal/Sample Visualizations

Source: http://www-958.ibm.com/software/analytics/manyeyes/page/create_visualization.htmlMany EyesLess learning timeNo programming knowledge requiredArcGIS, Tableau, Graphpad, Many eyesProgramming/Scripting knowledge requiredAVS/Express, VisTrails, VisIt, VTK, Ferret, UV-CDAT, GrADS, IDV, R, SPSS, Jquery visualize, D3Open SourceFerret, GrADS, IDV, R, UV-CDAT, VisTrails, VisItSupporting several input formatsArcGIS, GrADS, VisIt, Ferret, NCLSupporting several visualization techniquesVisTrails, UV-CDAT, VTK, IDV, Many eyesSupporting large and complex datasetsAVS/Express, IDV, VisIt, VTK, Ferret

StrengthsDegrading performance while working with large datasets VisTrails, VisIt, XmdvTool, IDVPoor data modeling capabilitiesVTK, Tableau, Not supporting data interactionArcGIS, VTKSupporting limited operating systems/ browsers/ hardwareUV-CDAT, OpenDX, Many eyes, Ferret

LimitationsNone of the tools fulfill the needs of climate researchers completely.Switching among the tools could be easier if there is a standard input data format.Support of interactive 3D/4D visualizations.Support of several devices such as touch pads, display walls, mobile devices, and desktops.Handling erroneous data and missing data values.

DiscussionOne-Dimensional histograms, normal distributionsTwo-Dimensional line graphs, bar charts, area charts, pie charts, maps, scatterplots, and stream line and arrow visualizations.Three-DimensionalIsosurface techniques , direct volume rendering, slicing techniques , 3D bar charts and realistic renderings.Multi-Dimensional scatterplot matrices, parallel coordinates, starcoordinates, maps, and autoglyphsA Simple Taxonomy of Visualization TechniquesVisualization techniques

Source: http://www-958.ibm.com/software/analytics/manyeyes/page/Visualization_Options.htmlIntroductionBackgroundRequirements ArchitectureResearch PlanConclusions

VISTEDVISTED shall allow user to select a climate variable. VISTED shall allow user to select a combination of climate variables. VISTED shall allow user to select a time period. VISTED shall allow user to select a particular location.VISTED shall accept input data in netCDF format. VISTED shall allow user to download data in netCDF format. VISTED shall accept input data in CSV format. VISTED shall allow user to download data in CSV format. VISTED shall accept input data in binary format. VISTED shall allow visualization of datasets that are loaded from users system.

Functional RequirementsVISTED shall allow user to download data in binary format. VISTED shall allow user to view the selected data. VISTED shall provide the links for the navigation across the website. VISTED shall provide some sample visualizations to the users.VISTED shall allow user to choose a visualization technique.VISTED shall allow user to view data as time series graphs. VISTED shall allow user to pick a location from the map. VISTED shall provide users with frequently asked questions and answers. Functional Requirements38VISTED shall be platform independent. VISTED shall support many browsers VISTED shall be developed using competitive technologies like HTML5, jQuery, and CSS3. VISTED shall be extensible and reusable. VISTED shall be fault tolerant. VISTED shall have high performance. VISTED shall have high reliability. VISTED shall support devices like tablets and mobile phones.

Non-functional Requirements Use case diagram of VISTED

TechnologiesHTML5D3 JavaScript LibraryC#IDEVisual studio 2012Technologies D3 is the winner! * Provides several visualization techniques. * Provides data interactivity.

Technology

Source: https://github.com/mbostock/d3/wiki/GalleryIntroductionBackgroundRequirements ArchitectureResearch PlanConclusions

VISTEDSystem level diagram of VISTED

Modeling Output

NetCDF Files

NetCDF FileArray Data

Activity diagramFlowchart of VISTED

Initial PrototypeIntroductionBackgroundRequirements ArchitectureResearch PlanConclusions

VISTEDExploration of the current state-of-the art methods and technologies/tools used for the presentation and visualization of environmental data.Research and design of a new web-based software toolset for processing and visualizing transect data (these activities will lead to advanced data processing capabilities for the NCCP).Development, experimentation, and integration of the new processing and visualization software into the Nevada Climate Change Portal.GoalsTask 1: Survey existing methods and supporting tools used for the presentation and visualization of environmental data. Identify strengths and limitations. Outputs: survey report.Task 2: Elaborate conceptual design and operational approach (method) for a new web-based software toolset dedicated to presenting and visualizing NCCP environmental data.Outputs: conceptual design document; documented method.Task 3: Create software specification and architectural design of the new software toolset.Outputs: Software requirements specification document; design document (high-level sign, detail-level design, data design, user interface design, interface design).Task 4: Implement web-based software solution. Outputs: Implemented software; documented code.MilestonesTask 5: Integrate web-based software toolset into the Nevada Climate Change Portal and prepare user manual. Outputs: Integrated software, executable through the NCCP; tutorial and user manual.Task 6: Perform usability tests on the data portal and process results. Output: usability test report.Task 7: Based on user feedback, revise and improve web-based software toolset for data presentation and visualization. Output: improved web-based, NCCP-integrated software toolset for environmental data presentation and visualization.Task 8: Disseminate research and development results. Outputs: Journal or conference paper; one or two poster presentations.

MilestonesAs per GRA tasksPerformed survey on existing data visualization tools and techniques for environmental data. (Task 1)Gathered the requirements and created the concept and specification document. (Task 2)Created the detail design of the software toolset. (Task 3)Designed the initial prototype of the toolset. (Task 4) ProgressIn addition to GRA tasks

Wrote chapters 2 and 5 of the dissertation.

Presented a paper at CATA-2013 in March 2013.Likhitha R., Qiping Y., Dascalu M. S., Harris F. C. Jr., A Survey of Visualization Techniques and Tools for Environmental Data, CATA, March 2013.

Presented a poster in NSF EPSCOR Annual Climate Change Conference in March 2013.Likhitha R. An overview of visualization approaches for environmental data, Tri-State EPSCoR Climate Change Workshop, March 2013.

Coauthor on another paper and poster.Qiping Y., Michael M. Jr., Dascalu S., Harris F. C. Jr., Likhitha R., Community Metadata ISO 19115 Adaptor, CATA, March 2013.Richard k., Michael M. Jr., Eric F., Sohei O., Likhitha R., Ivan G., Jigarkumar P., Adrew D., Ershad S., Shahram., Dascalu ., Harris F. C. Jr., Communicating Climate Change on the Web: The Nevada Climate Change Portal, Tri-State EPSCoR Climate Change Workshop, March 2013.

ProgressTo doGet additional input from scientists.Finalize proposed approach and web-based solution. (Task 4)Integrate with NCCP. (Task 5)Perform user tests. (Task 6)Revise VISTED and compare with related toolsets. (Task 7)Disseminate research. (Task 8)Finalize and defend dissertation. (Task 9)

Work PlanIntroductionBackgroundRequirements ArchitectureResearch PlanConclusions

VISTEDThe main goal of the VISTED is to help the climate researchers in visualizing datasets using new capabilities. It provides a new approach and supporting tools. It gives users the flexibility in choosing the data of their interest.The toolset allows users to upload files for the visualization.

ConclusionsMain contributionsNew approach that integrates data extraction, conversion, and visualization (with possible extensions for data analysis).Associated web-based toolset for data manipulation and visualization.Support provided for several data formats.Flexible data extraction capabilities.Mechanisms for efficient visualization of climate data.

ConclusionsI would like to thank all my committee members.Dr. Sergiu DascaluDr. Valerie FridlandDr. Fred HarrisDr. Yaakov VarolDr. Yantao ShenI am also thankful to CSE R&D facultyMr. Eric FritzingerDr. Richard Kelley

AcknowledgementsNevada Climate Change Portal, available at http://www.sensor.nevada.edu/NCCP/.Graphical Forecasts, Nation Oceanic and Atmospheric Administration, available at: http://graphical.weather.gov/.UNR Valley Road Weather Station, Western RegionalClimate Center, , available at: http://www.wrcc.dri.edu/weather/unr.html.Snow Pack: Decadal Averages Map, Cal-adapt ExploringCalifornias Climate Change Research, available at: http://caladapt.org/snowpack/decadal/.Pavlopoulos G. A., Wegener A., and Schneider R., "A survey of visualization tools for biological network analysis", BioDataMining, November 2008.Aigner W., Bertone A., and Miksch S., "Comparing Information Visualization Tools Focusing on the Temporal Dimensions," 12th International Conference on Information Visualization, pp. 69 - 74, July 2008Mozzafari E. and Seffah A., "From Visualization to Visual Mining: Application to Environmental Data", IEEE Confererence on Advances in Computer-Human Interaction, pp.143-148, February 2008.Aigner W., Miksch S., Schumann H., and Tominski C.,Visualization of Time-Oriented Data, Springer, May 2011.Selected ReferencesArcGIS - Mapping and Spatial Analysis for Understanding Our World, ESRI, available at:.ArcGIS QGIS Faceoff, blog.donmeltz.com, available at: . AVS/Express Data Visualization Software, AVS/Express, .GrADS Home Page, Grid Analysis and Display System, .Unidata | IDV, Unidata, .UV-CDAT, UV-CDAT, available at: .VisTrailsWiki, VisTrailsWiki, available at: .VisIt Visualization Tool, visIt, available at .VTK - The Visualization Toolkit, Visualization Toolkit, available at