Department of Science and Technology Institutionen för teknik och naturvetenskap Linköping University Linköpings Universitet SE-601 74 Norrköping, Sweden 601 74 Norrköping

LiU-ITN-TEK-A--09/063--SE

The Virtual Telescope inUniview

Mikael Karlsson

2009-11-13

LiU-ITN-TEK-A--09/063--SE

The Virtual Telescope inUniview

Examensarbete utfört i vetenskaplig visualiseringvid Tekniska Högskolan vid

Linköpings universitet

Mikael Karlsson

Handledare Jonas LindqvistHandledare Carter Emmart

Examinator Anders Ynnerman

Norrköping 2009-11-13

Upphovsrätt

Detta dokument hålls tillgängligt på Internet – eller dess framtida ersättare –under en längre tid från publiceringsdatum under förutsättning att inga extra-ordinära omständigheter uppstår.

Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner,skriva ut enstaka kopior för enskilt bruk och att använda det oförändrat förickekommersiell forskning och för undervisning. Överföring av upphovsrättenvid en senare tidpunkt kan inte upphäva detta tillstånd. All annan användning avdokumentet kräver upphovsmannens medgivande. För att garantera äktheten,säkerheten och tillgängligheten finns det lösningar av teknisk och administrativart.

Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman iden omfattning som god sed kräver vid användning av dokumentet på ovanbeskrivna sätt samt skydd mot att dokumentet ändras eller presenteras i sådanform eller i sådant sammanhang som är kränkande för upphovsmannens litteräraeller konstnärliga anseende eller egenart.

För ytterligare information om Linköping University Electronic Press seförlagets hemsida http://www.ep.liu.se/

Copyright

The publishers will keep this document online on the Internet - or its possiblereplacement - for a considerable time from the date of publication barringexceptional circumstances.

The online availability of the document implies a permanent permission foranyone to read, to download, to print out single copies for your own use and touse it unchanged for any non-commercial research and educational purpose.Subsequent transfers of copyright cannot revoke this permission. All other usesof the document are conditional on the consent of the copyright owner. Thepublisher has taken technical and administrative measures to assure authenticity,security and accessibility.

According to intellectual property law the author has the right to bementioned when his/her work is accessed as described above and to be protectedagainst infringement.

For additional information about the Linköping University Electronic Pressand its procedures for publication and for assurance of document integrity,please refer to its WWW home page: http://www.ep.liu.se/

© Mikael Karlsson

AbstractThis report outlines a thesis project where new visualization funtionality was im-plemented into the existing Uniview visualization system - a product by Sciss AB.This functionality was ultimately presented as the capability to zoom in on faraway celestial bodies and formations in space using a virtual telescope. This zoomfunctionality can present parts of or the whole of 2D imagery with high resolutionand let the user of the system navigate through it.

SammanfattningDen här rapporten handlar om ett examensarbete där ny visualiseringsfunktiona-litet implementerats i det befintliga visualiseringssystemet Uniview - en produktav Sciss AB. Funktionaliteten presenterades som möjligheten att zooma in och utpå utomjordliga objekt och formationer långt borta i världsrymden genom ett vir-tuellt teleskop. Zoom-funktionaliteten kan visa delar av eller hela 2D-bilder medhög upplösning och låta användaren av systemet navigera genom det.

1

Acknowledgments

I would like to thank:

• Jonas Linqvist, Sciss ABfor his engaging enthusiasm and support for my work.

• Carter Emmart, American Museum of Natural Historyfor his great hospitality during my stay and his never ending flow of freshideas, realistic and visionary alike.

• Robert Hurt, Spitzer Space Centerfor his quick and friendly feedback on comments and questions concerningthe VAMP standard.

• Everyone else at Sciss ABfor putting up with my questions and helping me out with server problemsand system intricalities when they arose.

3

Contents

1 Introduction 31.1 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2 Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.3 Target audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.4 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 Background 52.1 Uniview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.2 VAMP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.2.1 Astronomy Visualization Metadata . . . . . . . . . . . . . . 62.3 Existing technologies . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.3.1 Google Earth and Sky . . . . . . . . . . . . . . . . . . . . . 62.3.2 World Wide Telescope . . . . . . . . . . . . . . . . . . . . . 72.3.3 Stellarium . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3 Problem Description 93.1 Zooming in on objects . . . . . . . . . . . . . . . . . . . . . . . . . 93.2 Searching for and fetching data . . . . . . . . . . . . . . . . . . . . 10

4 Implementation 114.1 Use-case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114.2 Front-end . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

4.2.1 Telescope view . . . . . . . . . . . . . . . . . . . . . . . . . 124.2.2 Sloan deep sky survey through WMS . . . . . . . . . . . . . 154.2.3 VAMP Search interface . . . . . . . . . . . . . . . . . . . . 16

4.3 Back-end software design . . . . . . . . . . . . . . . . . . . . . . . 164.3.1 Back-end interface . . . . . . . . . . . . . . . . . . . . . . . 164.3.2 Downloading imagery . . . . . . . . . . . . . . . . . . . . . 174.3.3 Threading . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174.3.4 Local caching . . . . . . . . . . . . . . . . . . . . . . . . . . 17

5 Result 195.1 Zooming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195.2 Implementation in an independant library . . . . . . . . . . . . . . 215.3 Local caching scheme . . . . . . . . . . . . . . . . . . . . . . . . . . 23

5

6 Contents

5.4 Graphical user interface . . . . . . . . . . . . . . . . . . . . . . . . 235.5 Data visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . 245.6 Summing up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

6 Improvements and future work 276.1 Drawbacks of the local cache . . . . . . . . . . . . . . . . . . . . . 276.2 Interacting with the visualization . . . . . . . . . . . . . . . . . . . 276.3 Extending the zoom window . . . . . . . . . . . . . . . . . . . . . . 28

6.3.1 Video and time-laps . . . . . . . . . . . . . . . . . . . . . . 28

Bibliography 31

List of Figures4.1 Wide field of view . . . . . . . . . . . . . . . . . . . . . . . . . . . 124.2 Wide field of view, camera moved . . . . . . . . . . . . . . . . . . . 124.3 Narrow field of view, camera stationary . . . . . . . . . . . . . . . 134.4 Screenshot of the zoom window, galaxy (M81) . . . . . . . . . . . . 144.5 Screenshot of the zoom window, Orion Nebula . . . . . . . . . . . 14

5.1 Undistorted image of Earth and Moon . . . . . . . . . . . . . . . . 205.2 Distorted image of Earth and Moon . . . . . . . . . . . . . . . . . 205.3 Orion in the night sky . . . . . . . . . . . . . . . . . . . . . . . . . 215.4 Zoom window turned on. . . . . . . . . . . . . . . . . . . . . . . . 215.5 Zoom window zoomed in on the Orion nebula. . . . . . . . . . . . 22

Chapter 1

Introduction

This thesis is, in a way, a continuation of earlier theses conducted at Linköping Uni-versity in collaboration with the American Museum of Natural History (AMNH)in New York, USA. The first thesis ended up with a new company, by the nameof Sciss, being formed. This thesis was conducted partly at Sciss’ offices in Nor-rköping, and partly at the Hayden Planetarium at AMNH.

This thesis project builds on the software and thus the work of Sciss. Some ofthe implementation also directly uses work and results from [4].

1.1 PurposeThe purpose of this thesis is to implement new visualization functionality into anexisting astro-visualization system called Uniview. This functionality is meant toenable the user to explore two-dimensional data from observatories and satellites,using a telecope-like user interface within Uniview.

1.2 StructureChapter 2 - Background introduces existing technologies and standards that areused in this thesis. It also briefly presents some features found in similar productson the market.

A description of the motiviation for this project, and the problems behind it,can be found in Chapter 3 - Problem Description.

Final implementation of a separate back-end and front-end is outlined in Chap-ter 4 - Implementation.

Chapter 5 - Result then digs deeper into how the requirements were met,problems that arose during the project, decisions taken and what could have beendone differently.

Finally, Chapter 6 - Improvements lists a few things that deserve to be ad-dressed during future work.

3

4 Introduction

1.3 Target audienceThe intended audience of this report is primarily anyone directly related to theproject such as the thesis examiner, employees and collaborators of Sciss AB.Secondarily, it is intended for students and anyone else interested in visualization.It is preferable, however, that the reader already has some basic programmingknowledge and understanding of the most common problems and challenges ofcomputer graphics in general.

1.4 LimitationsThis project focuses on producing a proof-of-concept, leaving the finer implemen-tation details - such as graphical interface design - of an eventual productionimplementation out. Some effort will be put into researching what demands auser of this type of addon system might have, and this is presented as part of thisthesis’ discussion.

Chapter 2

Background

2.1 UniviewUniview1 is a realtime, interactive visualization platform designed to bring itsaudiences a truely immersive experience. It features the ability for the audienceto take the role of everything from astronauts, orbiting a planet like Earth, tointergalactic travelers, flying through the stars.

It is written in C++ and uses OpenGL for its rendering capabilities.

2.2 VAMPThere are several large problems in reaching out to the public with astronomicalmultimedia data. One of these is the lack of a standard with which to categorizeand catalogue images. There is also a vast number of places in which one can lookfor such data, none of which are coherent in, amongst other things, user interfacefunctionality.

VAMP stands for Virtual Astronomy Multimedia Project, and is a standardthat addresses these very problems. A more elaborate presentation of the problemsmotivating the VAMP standard can be found in the VAMP Project Description[2].It is currently under collaborative development at Spitzer Science Center2, Esa/Hubble3,California Academy of Sciences4 and IPAC/IRSA5

The aim of the VAMP project is to provide a standard through which anyonewho is interested can search for and download high quality astronomical imageryand other multimedia. The media is to be uniformally tagged with all the informa-tion that is relevant for the specific case, and thus specified in the standard. TheVAMP database system is not meant to contain any data in itself, but is insteadsupposed to act as a reference database for anyone who looks to find this data.

1http://www.scalingtheuniverse.com - June 20082http://ssc.spitzer.caltech.edu - June 20083http://hubble.esa.int - June 20084http://calacademy.org - June 20085http://irsa.ipac.caltech.edu - June 2008

5

6 Background

The idea is to let data sources register their data with the VAMP servers,thereby giving the general public access to it. In this way, the many sources areconnected together in one searchable database.

2.2.1 Astronomy Visualization Metadata

The VAMP standard supports itself on a well defined system of tagging of mul-timedia. This system is specified in the AVM standard [1]. AVM, and thereforealso VAMP, supports not only observational imagery from telescopes and satel-lites, but also other type of media like simulation videos and artwork. As for thisthesis, however, observational imagery is the focus. Extending the use of imageryto other types requires minimal effort.

Observational imagery tagged using the AVM standard contains all informationone needs to use it in an astronomical visualization. By looking at the tags, onecan, for instance, find whether the image is an actual observatory image, artworkor an animation, as suggested above. If it is an observatory image, the tags mustalso include the position in the sky from where the image was taken. This makesit possible to put images up on a planetarium dome, at the position where theyactually originated.

The AVM tags are stored not only in the VAMP database, but also as eX-tensible Multimedia Platform[3] (XMP) tags in the header of the multimedia filesthemself. At the time of writing, the server interface for the VAMP servers areyet to be specified and no test servers are up and running. Thanks to the use ofXMP tags in the imagery itself, enabling access to observation parameters of theindividual images, development of the visualization itself could proceed.

2.3 Existing technologies

There are a few technologies - in development or already available - that presenta user with something close to the functionality described in chapter 3.

2.3.1 Google Earth and Sky

The most recognized - and indeed the most widely used - is Google Sky. GoogleSky was released as a feature within Google Earth, version 4.2, in August 2007and is essentially Google Earth with a few modifications. Google Sky utilizes therendering functionality that Google Earth already has, but inverts the earth globeand puts the camera in its center instead. The data is then streamed and updated- depending on camera parameters such as zoom and direction - in the same wayas with Google Earth.

In March 2008, a web based version of Google Sky was also released. It is basedon Google Maps and features more or less the same functionality at a glance.

2.3 Existing technologies 7

2.3.2 World Wide TelescopeMicrosoft Research’s contribution in this area is called World Wide Telescope(WWT). It was first officially announced at TED 6 in February 2008, and releasedin May 2008.

WWT features high quality visuals with high resolution datasets. The data isalso bought and owned by Microsoft, which means they’re free to edit it. Obser-vational imagery in WWT has been tailored to suit the application’s estethicallypleasing approach. Ongoing work with eliminating beauty flaws such as sharpedges around images, lets the images melt in seamlessly with the mostly blackbackground.

2.3.3 StellariumStellarium is an open source software, featuring a planetarium-like experience, onanything from a desktop home computer to a planetarium. It is quite similar toGoogle Earth in many respects, and the point of observation is not required to beEarth, the sky can also be viewed from the surface of Mars for example. The ideais for it to emulate backyard sky gazing, and it does this quite well, giving the userthe possibility to find planets, stars and other formations of interest in the nightsky.

6http://www.ted.com/index.php/talks/view/id/224 - June 2008

Chapter 3

Problem Description

There are two main problems that are addressed in this thesis project.First and foremost, the goal of this project is to give a user of the Uniview

system a tool that resembles a telescope, through which it is possible to zoom inon distant celestial bodies and astronomical formations. The idea is to present theuser with an analogy to real life backyard star gazing.

The possibility to look far into deep space, however, means nothing if there isnothing there to see. The dataset - called Digital Universe 1 - that Uniview usesto visualize the stars does not - at the time of writing - contain high resolutiondata of such things as nebulae or other galaxies. This calls for a new data source.

3.1 Zooming in on objectsUniview is a software based on three-dimensional datasets. Until the beginningof this thesis project, there has been no possibility in Uniview to look closer atcelestial formations such as nebulae, for which no 3D data exists.

Sources for two-dimensional data do, however, exist and sometimes in abun-dance. This data is collected and processed mainly from Earth based observatoriesand satellites, and is presented in the form of images. An image of a nebula hasno true meaning from any other point of view than from where it was taken. Thereason for this is that an image is just that; an image. It is a two-dimensionalprojection of the nebula which is three-dimensional. The nebula could be severallight-years across, and we have no detailed idea of its structure when it comes toits depth - its distance away from us and size in that direction.

Moreover, in Uniview it is considered important that the data is visualized cor-rectly. One could, of course, make a 3D model or volume rendered representationof a nebula, and make it look right from Earth’s point of view, and present theaudience with a cloud of colored dust to fly through. It would, however, not be acorrect and true representation.

1http://www.digitaluniverse.net/ - June 2008

9

10 Problem Description

3.2 Searching for and fetching dataA visualization using images manually fetched from online resources is never com-plete. It needs the capability to be updated once there is new, interesting dataavailable or when existing data is updated. To make this possible, a search in-terface implementation needs to be made towards a server that offers this kind ofservice.

To avoid having to fetch data from a remote host over a network every timethe software is started, data should be stored and re-used beteen runs.

Chapter 4

Implementation

First of all, a very short use-case is presented to highlight what the goal from auser’s point of view is. This is done in the form of an example.

4.1 Use-caseA client program, in this case the Graphical User Interface (here-

after refered to as the GUI) in Uniview, should supply a visual repre-sentation of the Horse Head nebula. The software therefore needs tofind and display imagery depicting it. The user submits a search queryin the form of parameters and tells the program to submit the search.

Once the program is done with its query and has gotten its resultsback from a media source, it acesses the result set and uses it as visu-alization data. The result set contains imagery and any data connectedto each image. The user can then analyze the returned data and choosewhat to use and how to use it, and what to discard.

Following this example, it is logical for the software design to be divided intoa back-end and a front-end part, with a simplistic interface inbetween.

The back-end is made up of functionality hidden from the user. Everythingit does can be done in the background, without any direct user interaction. Thefront-end on the other hand includes both a hook into the back-end, and thegraphical user interface for navigating the return media or data, which requires aconnection into the rendering pipeline of Uniview.

4.2 Front-endThe GUI is divided into two separate parts. Using VAMP as the media source,one is the VAMP search tool, which enables the user to submit search queries toa VAMP database. The other is the telescope tool inside the visualization, whichenables the user to zoom in on and navigate among far away objects of interest inspace.

11

12 Implementation

4.2.1 Telescope viewFrom what is stated in 3.1 and described in 4.1, what is required is for the userto "stand" on Earth, or somewhere close-by, and look at the two-dimensional im-agery of nebulae and galaxies that we do have - as opposed to three-dimensionalrepresentations. The images are of course more often than not far too small to beviewed normally, and have to be zoomed in on. There are two obvious ways ofdoing this.

Figure 4.1. Camera and view frustum with a wide field of view angle.

Figure 4.2. View frustum is the same as figure 4.1, and the camera is instead movedcloser to the image to achieve the same grade of zoom as in figure 4.3.

1. One could simply fly there. By placing the image on a billboard at thecorrect distance in space and moving the camera out into space towards theimage, a type of zooming is achieved. This is demonstrated in figure 4.2.This would also make sense educationally, since it gives the audience a senseof distance and spatial position. Moving the camera out of the view frustum

4.2 Front-end 13

Figure 4.3. View frustum with a narrow field of view angle and camera at the samedistance from the image as in figure 4.1.

would also reveal that what is seen is indeed a billboard image, and thus thefact that it is all the data we actually have - an important point to make.

2. "Zooming", in computer graphics, usually means manipulation of the field ofview of the virtual camera. In this case, the camera is stationary, and worksjust like an ordinary, real-world camera lens. Figure 4.1 and 4.3 illustratethis.

In this implementation, a modified version of the second alternative was chosen.See chapter 5.1 for a discussion on this. The modification is that the data is notrendered in the Uniview spatial rendering context - or coordinate system - directly,but instead in its own spatial context. It is then presented to the audience througha "zoom window" or "telescope view".

This is depicted in figure 4.4, which is a screenshot from Uniview showingthe rectangular telescope view projected onto a separate, framed billboard withinUniview. In this example. it is aimed at a galaxy by the name of M81, which canbe found just north-east of the Big Dipper in the sky. The cross that can be seennext to the big, blue telecope view is a marker that shows the audience where inthe sky the view is currently aimed.

Figure 4.5 demonstrates a few features that were requested during the project.

• First of all: layering and blending. The rendering functionality behind thetelescope view is capable of rendering an arbitrary number of images. Thelimitation is the amount of texture memory and number of vertices the graph-ics hardware and the rendering platform can handle. The images can alsobe categorized into an arbitrary number of layers, rendered back to front,that can also be blended as the user chooses. This is a very important fea-ture, which enables overlaying of images that depict the same thing. It isnot uncommon to have a stack of images that are of different wavelength forexample, and the ability to layer and blend inbetween them can be useful. In

14 Implementation

Figure 4.4. Screenshot of the telescope view, aimed at M81 (149.18 ◦ ascension, 69.06 ◦inclination)

Figure 4.5. Screenshot of the telescope view, aimed at the Orion Nebula just belowOrison’s Belt (83.6 ◦ ascension, -5.6 ◦ inclination)

4.2 Front-end 15

figure 4.5, this is demonstrated by the larger dark red image of the nebula,and the brighter and more colorful image layered and blended on top of that.

• The second feature demonstrated here is the displaying of a view frustum.Normally, the user will want to display the telescope view when being posi-tioned at Earth’s location, since that is the source of the images. By showingthe view frustum like this, the audience can get a much better sense of lo-cation. If the Uniview camera leaves Earth and travels into the stars, thefrustum will show from where, and towards where, the telescope view islooking. This will also help the understanding of the vast distances and dif-ferences in scale that we are dealing with. See chapter 6.3.1 for a discussionon what else this could be useful for.

• Thirdly, figure 4.5 is divided into two parts by the white, horizontal line.It is composed of two different screen shots of the same view in Uniview,but with one difference which is that of the telescope view having had itscolors inverted in the top half. This is a simple feature which was requestedbecause astronomers sometimes like to look at a white sky with black starsrather than a black sky with white stars.

The visual experience became more pleasant due to the utilization of valueinterpolation when zooming and panning across the sky. The ascension, inclinationand field of view angle of the camera is controlled by changing values over time,giving the window the possibility of sliding across the sky and the zoom level tochange seamlessly.

4.2.2 Sloan deep sky survey through WMS

As mentioned in Chapter 1, this work is in some ways based on and supported bythe work of [4]. An extension to the originally aspired functionality is the use of aWMS backdrop, using the Sloan deep sky survey1 - hereafter refered to as SSDS.

In short, [4] presents a method of coating a planet with textures dynamicallyloaded depending on camera view parameters. With a small amount of extra work,such a planet was turned inside out and the camera was positioned in its center.This works in much the same way as Google Sky, described in section 2.3.1 Theinside of the sphere (planet) was then textured using data from the SDSS. Oncethe orientation of the planet’s coordinate system has been alligned with that of theVAMP imagery and the world coordinate system, the VAMP imagery was layeredon top of the multi-resolution background of the SSDS data. In this way, the useris supplied with an extra reference in the zoom window - outside the area coveredby VAMP imagery - making it easier to navigate the data. Furthermore, it givesthe audience a better understanding of what they are seeing; that the imagery isactually depicting a portion of the sky.

1http://www.sdss.org/background/science.html

16 Implementation

4.2.3 VAMP Search interface

The implementation of the search interface inside Uniview was constricted to thatof a simple text based command interface. The commands were then passeddirectly into the back-end. The idea from the beginning was to implement a GUIthat looks much like an image catalogue browser, with search fields to specify.The reason why this has not been done is that the VAMP server interface has notbeen - and will, according to the developers, not for some time be - completelyspecified. Since the interface isn’t specified, there isn’t much of a point in makinga GUI, whereas the GUI should reflect possibilities and functionality allowed bythe server interface.

4.3 Back-end software design

Apart from what the user actually sees and is able to interact with, most of thework that goes into the product development lies in the functionality the GUIcalls upon to perform its tasks. This is the back-end of the product. It should,transparantly, provide functionality to submit a search query to a multimediadatabase and present the result back in a format that makes sense.

Implementing a service that can be used by a client to query a VAMP databaseis simple enough. In this implementation, however, there are a few things the back-end has to be able to do apart from simply passing on a query and returning theresult.

According to the VAMP specification [2], what comes back from a search queryto a VAMP database is a set of tags conforming to the AVM standard [1]. Theresult does not contain any image data, but instead a reference, a URL, to it. Thismeans that the image data has to be downloaded from somewhere other than theVAMP database server itself.

4.3.1 Back-end interface

To minimize the need for understanding the Uniview system in detail, and alsoto minimize dependencies on Uniview functionality, the back-end was written in astand-alone library. This way, the front-end - the Uniview GUI - can simply loadthe library and use it. This also means that any other application could use thelibrary if need be.

The interface should naturally be made as simple as possible. The actualobjective of the back-end - from the front-end’s point of view - is to take a numberof search parameters as input, and be able to submit this as a query to a VAMPdatabase. Once the query comes back, the back-end should perform any necessaryoperations transparently and the result is to be returned to the calling instance,which in this case is the front-end.

4.3 Back-end software design 17

4.3.2 Downloading imageryIn order to fetch the imagery, a download manager within the back-end is calledto initiate the download. The download manager stores the image in a temporarylocal folder and returns upon completion. Once the imagery has been downloadedsuccessfully, it is passed through an image processor. In this stage, a low resolutionthumbnail copy is made of the full image, and stored separately. XMP data is alsoextracted from the image to aquire the AVM tags that are stored in the header.These tags are then cross-checked against the AVM tags provided by the VAMPquery result. If they are equal or a tag is missing from one of the sources, it isstored together with the image and thus made available to the front-end once theresult is returned from the back-end. If the tags are different - although it shouldnot happen - the tag from the VAMP query result is chosen for storage, since thesehave been reviewed before being put into the database.

4.3.3 ThreadingThe back-end has - according to what has this far been described - many tasksto perform before a query is completely processed, and thus takes some time tocomplete its operations. Among these is the task of downloading images froma remote server (Chapter 4.3.2). This task in particular has a highly variabletimeframe in which to complete depending on bandwidth capacity, remote serverload, local resources and other parameters.

This all means that once the client program has called the back-end, the back-end will most likely be busy for quite some time. No matter how fast it all fallsthrough, it’s still too long a time for a realtime visualization software to wait toget its execution thread back.

Considering all this, it makes sense to give the back-end its own executionthread. This way, the client program can continue with whatever it is doing whilethe back-end is busy. Once the back-end has finished, it can call the client througha callback function with the results. It is also possible to let the client continuouslypoll the back-end to check if it has finished its operations, and once it has, get theresults through an explicit call. Since it might, in different implementations, besuitable to use both of these solutions, both were also implemented.

4.3.4 Local cachingOnce the client has received the results from a query, and the user has had achance to review them, it should be possible to use or discard any of the results.The back-end therefore contains a connection to a local cache. The reason for itsexistance is to avoid future searching and downloading of material that has alreadybeen fetched and imported into the visualization. With a local cache of earlier usedimagery, some query results can simply be taken from a local resource instead ofa remote one, thus drastically decreasing bandwidth and processing requirements.Since the visualization also has to get its imagery from somewhere locally in orderto use it, it makes sense to let the local cache of the front-end and the back-endbe one and the same.

18 Implementation

The local cache is a simple local SQLite 2 database. SQLite is a lightweight,serverless, file based database which takes standard SQL queries as input. TheSQLite library is already in use in Uniview, so there is no compatability issues orextra library dependencies introduced by the use of SQLite in the back-end.

The database files contains AVM tags with information such as coordinates,imagery type and purpose. The tags also carry information about reference pathsto actually imagery data stored locally. It also stores a flag which saves - betweenprogram executions - whether the image should be displayed in the visualizationand what layer level it adheres to.

2http://www.sqlite.org/

Chapter 5

Result

A lot of things were done apart from the initial requirements of the thesis project.Most of these features were implemented as a consequence of discussions, requestsor needs that arose during development. Below follows a summary of each indi-vidual part of the implementation and how they turned out.

5.1 ZoomingThe first possible choise between zooming approaches oulined in Chapter 4.2.1would be a bit unnecessary to implement, because recently Digital Universe hasstarted to include images of nebulae and galaxies. These are this far of low res-olution and not meant to be looked at up close, but the functionality is alreadythere and really only needs polishing.

With the second choice, there are some problems. As long as the user is infront of a flat computer screen, this would look good. It would work just like usingan ordinary telescope. By decreasing the field of view angle, and thus narrowingthe view frustum, the center of the image will seem to come closer because asmaller part of it will fill the same amount of screen area. This is illustrated in thedifference between figures 4.1 and 4.3. In computer graphics, this merely meanschanging the perspective projection transformation. The problem arises when thescreen is no longer flat. On a curved screen, a planetarium dome or - in worst case- in a cave visualization model, this doesn’t work.

Figure 5.1 depicts an illustration of the Earth and the Moon in a dome pro-jection model, where the circles are - when projected on the insides of the dome -horisontal lines similar to the circles of latitude on our planet. As can clearly beseen in figure 5.2, in such an environment everything outside the center is distortedonce the field of view starts to deviate too much from its normal value.

The solution that was chosen to cope with this, was to present the zoomed-in view on a planar quad - a square billboard - within the rendering context ofUniview. This way, using the field of view for zooming in is quite possible, anduniversal. The view itself is then first rendered in its own rendering context, and

19

20 Result

Figure 5.1. Undistorted image of Earth and the Moon with a reference circular gridfitting a dome projection model.

Figure 5.2. Zooming in on the moon by decreasing the field of view in a dome projectionmodel. The distortion is clearly visible both for the grid and the Earth, which seems to"melt away".

then displayed on the planar quad within the normal Uniview rendering context.The result is something that looks and works a lot like a digital telescope.

Furthermore, this approach gives the audience a sense of proportion, directionand origin thanks to the fact that the context around the object of interest isretained, as demonstrated by the figures 5.3, 5.4, 5.5 below.

5.2 Implementation in an independant library 21

Figure 5.3. The constellation of Orion centered in the sky, with Earth in the foregroundto the right.

Figure 5.4. The zoom window has been turned on, aimed at the Orion nebula. Thefrustum helps with the insight that it is an image taken from Earth.

5.2 Implementation in an independant libraryThere are a few different ways in which to implement the new functionality intoUniview. The ones that were eligible in this case were

22 Result

Figure 5.5. Same as figure 5.4, but zoomed in a lot further. The frustum continouslysupports the notion that the nebula indeed covers a miniscule part of the sky.

• Direct implementation into the Uniview system

• Dynamic Link Library (DLL)

• Static Link Library (LIB)

In the first case, all code would be written directly together with the rest ofthe Uniview code. To make this practically possible, a thurough understandingof the existing components of Uniview would first be necessary. However, fromand during the first stages of development of the VAMP search functionality, thewhole Uniview system was planned to undergo an extensive refactoring process.This meant that a large portion of the understanding of the existing system would- for a new developer - be outdated and irrelevant only within a few days. Thiseffectively discarded the first alternative.

Choosing to develop the whole back-end functionality within a library circum-vents the whole problem of first having to get to know the Uniview system that isalready in place. Instead, focus could be put completely on building the back-endfunctionality itself, without having to take very many surrounding factors intoconsideration.

In both cases, the functionality is stored into an independant software library,which can then be linked into Uniview, and accessed from there. In the dynamiccase, a Windows DLL file is compiled and has to accompany the rest of Uniview.It is then accessed and used at run time. In the static case, the library file isincluded into the Uniview system at link time, and doesn’t need to accompanythe Uniview distribution as a separate file. Considering that the VAMP search

5.3 Local caching scheme 23

functionality is meant to be a part of Uniview, and not just an independant plugin,there is no real need to keep it a dynamic library. The Uniview system also usesthis approach today for other parts of the system. Consequently, the solution of astatically linked library was chosen.

5.3 Local caching schemeAs previously stated in 4.3.3, the time it takes for an asset to be downloadedand made ready for presentation is heavily dependent on external factors thatare not under the control of the user. Assuming, however, that a stable, highbandwidth connection is available and that the number of media assets that needto be downloaded is small, there is one more possibility, namely that of streaming.

Streaming media resources directly into the application would technically work.It would, however, put a lot of excessive load on the servers from which the datais fetched and it would introduce waiting times that might disturb the flow of thepresentation, especially if it is interactive and not following a previously set outschedule.

Furthermore, there is - at the time of writing - no support for thumbnails inthe VAMP standard. This means that previews of imagery cannot be made insidethe application without downloading the whole image. It could be faked if theimagery was always progressive. That is, however, rarely the case and there areno constraints for this within the VAMP standard. Ways of handling this weresuggested to the team that develops the standard, with positive feedback as aresult.

In the end, the local cache turned out to be a good solution. It minimizesload times from anything unknown to nearly going unnoticed. Giving the useran interface through which it is possible to dynamically load and control whatimagery is to be displayed in the visualization (see Chapter 5.4), also eliminatesthe problem of high load times on program initialization if the local cache is large.

5.4 Graphical user interfaceOne thing that is missing in the front-end implementation compared to the initialfeature vision is an interface through which the user can search for, browse andsort imagery to include in the vizualization. A desirable property for such aninterface would be simple features such as

• Free and specific keyword lookups

• A possibilty to select and distinguish between remote database searches andlocal cache searches

• A simple drag-and-drop interface to include and exclude imagery in the vi-sualization

• Scheduled property alterations and image inclusion/exclusion for show au-tomation

24 Result

A simple tool such as this was implemented outside of Uniview in a quick-and-dirty manner, merely to support development. It was in no way suitable forinclusion within Uniview, although it did carry some value as a side effect. Thestatic library that serves as the back-end turned out to prove itself quite well as anindependent extension to any software. This means that it is simple to use bothin an external tool or an internal interface in Uniview, making it possible for theGUI to meet any demands the user might have without altering the back-end theslightest.

A second point it brought forward, was that loading imagery is still heavy workeven when using a background worker thread. This is primarilly important whensearching for VAMP imagery from remote servers. Since a preview cannot be dis-played until the image has been downloaded, the user experience can be anythingfrom slightly slow to a complete waste of time. As mentioned in Chapter 5.3,there is no mention of thumbnails in the VAMP specification and an improvementof it whould be to have the inclusion of thumbnails incorporated in the VAMPstandard.

Finally, the back-end could also use a possibility for the user to interrupt aquery that is taking too much time, either interactively or through a timeout.

5.5 Data visualizationThe zoom window created in this project is a very good proof-of-concept type ofvisualization. It is more or less as simple an implementation as can be, while stilldemonstrating the potential of such an addition to the software. This also means,however, that it is also not much to look at for a real-life audience. It serves itspurpose, but no more. One improvement on the visual style could for example bemaking the zoom window round instead of square, reinforcing the idea that theaudience is looking at something representing an actual telescope.

5.6 Summing upIn implementing the functionality described in Chapter 3, there were two maingoals to reach. In order to present the user or audience of the software with atelescope view analogous to backyard star gazing, two things are needed apartfrom actually rendering the telescope view:

• The capability of panning and zooming in and out on imagery

• Classification and fetching of data

The result of the zooming approach turned out quite well. Thanks to the factthat it is rendered in its own coordinate space and only displayed in a flat window,no problems with object distortions were encountered - apart from the benefitof giving the audience an extra reference to grasp the context around the zoomwindow. Zooming was smoothly rendered and scalable both in size and resolution.

5.6 Summing up 25

The layering of different imagery and using a SDSS backdrop also helped in makingthe experience rich and graspable, thanks to the added visual context.

All in all, the main goals of the project were met. It is possible for the systemto download and classify data (imagery) by the tags they contain. While doingthis, the user is not disrupted and an ongoing show does not get interrupted. It isalso possible to render and present the data to the audience through an interfaceresembling a telescope, although the GUI could perhaps be beautified somewhatand the controls can definitely be improved upon.

As previously stated, there is one thing missing. That is the possibility tosearch through and fetch resources using a VAMP server, although the reasons forthis not having been done has been previously discussed.

Chapter 6

Improvements and futurework

There are a few parts of the work that can and should be given further attention.There are several issues not covered in Chapter 5, both techincal ones and thoseconcerned with ”softer” - but nontheless important - areas like human-machine-interaction and visualization control.

6.1 Drawbacks of the local cacheOne small snag when turning to the local caching scheme is that once imagery isdownloaded, it is not affected by changes introduced on the media source serversor information stored in the VAMP database. This, however, would be fairlystraightforward to handle. A suggested approach to this would be to periodicallylet the back-end update all entries in the database by checking them against theVAMP database. If an inconsistency is found, the local database would be up-dated. As for the caching of imagery, the images would have to be downloadedagain to check for differences. A nice addition to the VAMP standard would be torequire that information about updated media should be accessible in the VAMPdatabase itself.

6.2 Interacting with the visualizationExcept for the graphical presentation of the zoom window, there is also the human-computer interface with which it is controlled when running a show. Almostno time was put into implementing this interface, only researching it. Duringdevelopment, the window was simply controlled either by the use of a mouse ora set of keys on a keyboard, where control consisted of rotating and acceleratingthe window in one direction or another.

There are of course many better ways one could imagine controlling the zoomwindow. First out is through the means of a device similar to a Wii Remote or

27

28 Improvements and future work

other type of wand device. With such a device, the narrator could litteraly point atthe dome and make the zoom window pop up at that location. For highly zoomedout views especially, a useful feature might be to use the device to "draw" an areaon the sky, which the zoom window opens up and displays, allowing for furtherexploration. This type of initial selection of an area to explore could, however,also be implemented through a more traditional mouse interface with only a visualrepresentation shown to the audience.

A simpler interface which is easier to implement is simply adding the possibilityof the user to specify targets, and let the visualization "move there", by bringingup the zoom view in the correct location. This is closest to the developmentimplementation, where the imagery could be loaded and located through commandline entry. The parameters of the window could then be input through anothercommand, and the window would interpolate its values toward those submitted -”moving” into place.

6.3 Extending the zoom windowThrough discussions with existing and potential users of the software, several ideasfor new uses of the zoom window were brought up. These all generalized the wholeidea of the zoom window prupose and leans more towards something like a mediafusion functionality, fusing several types of media views together into one.

6.3.1 Video and time-lapsOne of the most interesting ideas that surfaced was the possibility to view anythingflat. The use of video (or a series of images after one another, making a time-laps)had already been there from the start, since the VAMP standard allows for videoto be included. Therefore, after allowing the window to show video, it would be asmall step to extending the possibility of showing VAMP video to any other typeof video.

There are several interesting examples of uses for this during a live or pre-scheduled space show.

• The audience might be flying in to look at a vulcano on Earth and get tosee a video of an actual helicopter fly-by.

• When looking at all the satellites going around Earth, one - for example aweather or mapping satellite - potentially providing a live feed can be usedto present what it is actually seeing at the moment. This video could beeither live for real, or a recently recorded one.

• An expedition to Saturn shows the trajectory of the Cassini satellite aroundthe planet. Cassini has taken a lot of periodic imagery, which could bepresented as a time-laps to make some point.

• Uniview supports remote control which means that a controller can controlseveral shows at once, giving the same talk to several audiences at the same

6.3 Extending the zoom window 29

time. While watching a show directed by a speaker who is not geographi-cally present, the window could show live video the speaker, a Powerpointpresentation or some other supporting imagery.

Again, in any of these cases, the view frustum from the viewer position is auseful feature. It clearly showes where the video or imagery is taken from. Ifthe audience is looking at a video taken from satellite, the view frustum shouldoriginate from the position of the satellite. Also, when viewing VAMP imagery,the frustum can originate from the observatory or satellite from which the imagewas taken.

Bibliography

[1] Lars Lindberg Christensen, Adrienne Gauthier, and Robert Hurt. Astronomyvisualization metadata (avm) standard. International Virtual Observatory Al-liance, Version 1.1 Final Candidate, January 5 2008.

[2] Lars Lindberg Christiansen, Adrienne Gauthier, Robert Hurt, and Ryan Wy-att. Vamp - enabling global access to astronomy multimedia resources. virtu-alastronomy.org, September 2007.

[3] Adobe Systems Incorporated. Xmp specification.www.adobe.com/devnet/xmp/, September 2005.

[4] Johan Ostergaard. Planet rendering using online high-resolution datasets. De-partment of Science and Technology, Linkoping University, (LiU-ITN-TEK-A–08/034–SE), February 2008.

31