SnapWorks! Project Report

Krimy Amichandwala (krimya@gatech.edu)

Nihar Gadkari (nihargadkari@gatech.edu)

Samantha Misra (smisra6@gatech.edu)

Urja Shah (urja.shah@gatech.edu)

Introduction

With the increasing use and easy accessibility of high resolution cameras with no restrictions on

memory, we introduce SnapWorks - a novel concept in sharing a digital image library with your

near and dear ones, or for that matter anyone who cares. SnapWorks introduces several features

which prove to be very useful to the average user who seeks to share his images with others. In

addition to sharing of photos, SnapWorks provides for several convenient features such as

clustering based on event and locations, overlaying of images and clusters on the map. The user

can view photographs in sorted order based on the ‘Event and Location’ or divided by the

locations the pictures have been taken at or in groups divided according to the event.

Motivations & Objectives

We decided to make this application to help users share their photos and also along with that be

able to view them in different contexts depending on where and when they were taken to help

them organize and sort the photographs. We used the idea of clustering from some of the

recommended reading for the class.

Using the clustering on events and locations, users can see their photographs categorized based

on locations which have sub categories of different events taken place at that location. We cluster

the photographs based in the latitude and longitude and timestamp information stored as

metadata in each of the digital images. They also have an option to see their pictures sorted only

based on the location at which they were taken e.g. all the pictures taken at Atlanta come under

one category. For this, we cluster based on the latitude and longitude information of the

photographs. Another option is to see the photographs grouped according to the events when

they had taken place. We cluster the photographs based on the timestamp information in the

digital images here.

Architecture

The application is a three-tier architecture having a backend database which stores the user

information and a storage unit for all the photographs of all the users, a middle layer of business

logic where all the server pages, class libraries, tools and controls are present and the client side

where the user actually logs into his account to access his photographs.

It can be shown as:

Functional Components

We have five main components in our system. They are the database, metadata extractor,

clustering algorithm, overlaying of photographs on the map and photo sharing.

Database

The database stores all the user information like user name, email address, password, albums of

the user, the photo IDs of the photographs. It also stores the metadata of each of the photograph.

It stores the timestamp of when the picture was taken, the latitude and longitude of the location

where the picture was taken and the ID, album and user it belongs to. This information helps us

to render the photographs and also helps while clustering them.

Metadata Extractor

Digital photographs come with metadata about the picture in them. This data is stored in the

form of Exif tags. It stores normal information like the make of the camera, the model, date,

time, additional camera details like resolution, image width, height and also the GPS

information. However, the GPS information isn’t available with all the current cameras. Only

few cameras have the functionality of identifying the latitude and longitude. We have built our

system assuming that such cameras will be more popular in the future and that the GPS

information will be embedded in the photograph in the exif tags. Since we did not have a camera

with this functionality, we entered the latitude and longitude information manually in the

photographs using an Exif editor available at http://www.opanda.com/. We used the ‘MetaData

Extractor’ class library available at http://renaud91.free.fr/MetaDataExtractor/.

Clustering

The Clustering algorithm we have designed basically comprises of three steps:

STEP1:-

Fetching the metadata related to the images from the database and populating the ‘Photograph’

object and ‘Location’ object related to the location of the photograph. In the location object we

store the latitude and longitude of the photograph and by reverse geo-coding set the name of the

location where the photo was taken. This enables us to perform clustering on it effectively in the

later steps. Once the ‘Photograph’ objects are populated they are then logically partitioned into

different segments by adding the ‘Photograph’ objects in the appropriate ‘Segment’ object.

Every time a photo is added in a new segment the start time of the segment is updated to the time

of the first photograph added inside that segment consequently forcing the end time of the

segment to be twelve hours away from the start time .This step is basically implemented to

logically separate out the photographs based on time. Note that the twelve hour delay is an

arbitrary threshold which is taken into consideration. It can be changed to get varying results.

Cluster Based on Event Page:

STEP2:-

In the second step we make location based clusters in which the segments are added to different

clusters based on certain conditions. For this initially we find the means of the segments which is

nothing but the finding the mean latitude and longitude values of all the photos present inside the

segments. Then we make clusters based on these segment means based on the nearest or closest

points which are present to these segment means .Note that these nearest points are obtained by

geo-coding the segment means. Now all the segments having the same nearest points are added

in to the same ‘Cluster’ object. Therefore the nearest points are treated as cluster centers in this

step. This completes the first part of the second step .To get efficient cluster the clusters are

merged until we achieve a good spread of clusters. Initially the clusters are sorted according to

there cluster centers .This ensures that when merging the clusters based on certain user defined

thresholds we don’t merge certain spurious clusters .We merge two clusters when the distance

between the cluster means is below certain user defined threshold. Note that selecting an

accurate threshold is pivotal in getting logical and well defined clusters. This will ensure that

certain spurious clusters are not merged. Every time a cluster is merged we recalculate the cluster

centers which would be accordingly shifted. Therefore the output of the second step will be

obtaining appropriate event clusters based on location.

Cluster Based on Location Page

Step 3:-

In step three we merge the segments in the clusters based on certain user defined thresholds.

These thresholds are basically present to define an event. The user could fine tune this threshold.

In the algorithm we have calculated the threshold based on the number of segments inside a

cluster .Every time we find two segments inside a cluster where the difference between the end

times of the first segment and the start time of the second segment is less than the threshold the

segments are merged. The merging in this step basically indicates that the photos belonging to

two different segments though twelve hours apart (As discussed in the first step) belong to a

same event and hence belong in the same segment. Therefore the threshold could be altered here

to suite the event definition of a particular user.

To perform clustering we have maintains a class library called as ‘ClusterOnLocation’ which

primarily contains the entire clustering framework for the project. Any page which wants to

render these clusters simply has to call the ‘Driver()’ from the ‘ClusterManager’ class which is

entrusted with driving the entire cluster related activities. All the clusters are added in a

collection present inside a cluster hierarchy object .For rendering the photographs and clusters

one has to traverse the different collections inside the cluster hierarchy object.

Cluster by Event and Location Page

Overlaying

We have allowed the user to overlay the images from an album on a map to depict the location

where the images were taken. As mentioned earlier each image has a latitude and longitude

associated with it which is stored as metadata in the database. When the user chooses to overlay

the images on the map this latitude and longitude position of the image is utilized. To implement

the overlay feature, we have used a control that enables us to use Google maps API with

ASP.Net. Google maps API enables us to overlay these images at a specified latitude and

longitude on the map.

Overlay Photos on Map

Further, we have also provided the user with the option to overlay a photo cluster on the map in a

similar manner. Each cluster is represented by a cluster center which is then placed as a marker

over the map. On clicking the marker a window opens which contains all the images which

belong to that cluster. When displaying the individual photos over the map we are also reverse

geo-coding to find out the name of the location from the metadata of the image. This information

is included in the information window with the picture. This gives us a better idea about the

formation of clusters in the clustering step.

Overlay Clusters on Map Page

Album Sharing

We have also provided the user with the option to share his albums with other people by sending

emails which includes the URL to the album which the user wants to share. To do this, we have

incorporated a mail sending feature in the project which allows the user of our system to send an

invitation to other people by sending an email. We have used a mailing package which enables

this specific functionality.

Upload & Album Display Page

Share Photos Page

In addition to the glimpses of the functionality supported by the project shown, additional

snapshots of the basic functionality supported are included in the Appendix section of this report.

Issues

Different types of cameras vary slightly on the tags that they contain along with a photograph.

For example, the ‘Image Description’ tag is not present in the Canon cameras whereas in the

Nikon cameras it is present. Common tags were needed to be found in order to insert the location

information. Also mashing up Google Maps API with .Net environment was an unexplored area.

We found a way to do it seamlessly using subgurim.net control.

Strengths

One of the prime strengths of our application is that we have made use of an EXIF extractor

which basically extracts the desired metadata from the user uploaded digital .This makes the

users work easy and makes his experience enjoyable. The metadata is further stored in the

database and used for clustering and overlaying of the images.

One of the other chief strengths of our application is the efficiency of the clustering algorithm.

As discussed earlier the implementation of the algorithm is divided into three steps at the end of

which we get location based clusters inside which there are event based segments present. Such

sort of clustering is very intuitive and based on the common perception of a user. The clusters

and the segments could be fine tuned by varying the merging thresholds. A user therefore could

typically vary the segment merging threshold .This would enable the user to alter the definition

of an event. Also for visualizing these clusters we have overlaid these pictures over a map .The

user interface of our application is attractive and simple to use. The user is given intuitive

options to select from at each step.

Extensions

Tuning the Clusters

One extension to our application is still fine tuning the clustering parameters to fit the user’s

requirements after testing and getting more heuristic data. Also, we could allow the user to

define some parameters to cluster. It may be that different users feel that different amounts of

time make an event or they may want to cluster pictures more specific to a city or even want to

make it as wide as a location.

P2P Sharing

The application can also be extended to serve as a UI for sharing of photos over a peer-to-peer

network.

Commenting on Pictures:

In addition to photo sharing among users, an extension would be to allow users to comment on

the shared pictures. The comments will be visible to the user who ‘owns’ the photo as well as

users who have been invited to view the pictures. This is a good way for users to interact and

share experiences with respect to a particular place or event.

Photo Crunching An extension of picture organizing would be to use the clustered pictures of a location to recreate

the view of the location at which the picture was taken. The application will be able to generate a

360o view of the location. It will provide a panoramic view of a place where a friend stays or

even familiarizing a user of a location, he plans to visit. If two users have shared pictures of a

monument, the system will stitch these photos together to give the user a panoramic view of the

same.

Conclusion:

As described in this report, SnapWorks focuses mainly in providing the user with a number of

functionalities to organize pictures, while maintaining simplicity in terms of usage and it merely

touches the surface with respect to the functionalities that can be implemented. As discussed in

the extension section, there are several avenues to choose from to further enhance this

application.

References

1. “Automatic organization for digital photographs with geographic coordinates”, Mor

Naaman, Yee Jiun Song, Andreas Paepcke, Hector GarciaMolina, ACM, 2004.

2. "Time as Essence for Photo Browsing Through Personal Digital Libraries", Adrian

Graham, Hector Garcia-Molina, Andreas Paepcke, Terry Winograd, ACM, 2007.

3. "Mining Sequential Data in Search of Segmental Structures", Niina Haiminen, University

of Helsinki, Finland, Academic Dissertation, April 2008.

4. http://www.drewnoakes.com/code/exif/

5. http://www.opanda.com

6. http://renaud91.free.fr/MetaDataExtractor/

7. http://www.albumonnet.com/Home/Index.aspx

8. http://googlemaps.subgurim.net/

Appendix:

Snapshots of Basic Functionality

Login Page

Registration Page

Album Page

Create New Album Page