Extracting Valuable Information Lazily

Shiry GinosarAdvisor: Professor Manuel BlumGraduate Mentor: Luis von Ahn

One picture is worth a thousand words

If you have never seen an image, a description

might help your imagination

Our task was to attach a description to every image on the web

Wait… Didn’t

already do that!?

Not really…

only provides word labels.

These are great for image search

but not enough for

accessibility

Labels: Man, Woman

Our task was to attach a description to every image on the web

Our task was to attach adescriptive paragraph to every image on the web

Following the EVIL tradition, this was done

in the form of a FUN GAME

Input Output behavior

imageDescriptive paragraph

The game is an internet race between multiple players all

aiming to find the same randomly chosen image.

They are the “seekers”

The only information they have about this image comes from a

written description that is given to them

We provide them with an image search engine

Because searching is cool

The seeker’s screen

One of the players is the “describer”. He is the only one

who gets to see the image.

He will provide us with thedescription we are looking for

The describer’s screen

Once a seeker finds the correct image, he becomes the next

describer and a new image is chosen

Gain Points

The seeker that finds the image first wins 100

pints

The describer wins 200 points if

someone finds his image

Lose Points

The seeker loses 10 points for each

wrong guess

The describer loses 5 points for every image he

passes

This FUN GAME will hopefully provide us with descriptions of all

images on the web

Thus contributing to theaccessibility of the web

One picture is worth a thousand words

www.peekaboom.org/phetch