Odd Leaf OutCombining Human and Computer Vision
Arijit Biswas, Computer Science and Darcy Lewis, iSchool
Derek Hansen, Jenny Preece, Dana Rotman-University of Maryland’s iSchoolDavid Jacobs, Eric Stevens-University of Maryland Computer Science
Jen Hammock, Cynthia Parr-The Smithsonian Institution
Refining Metadata Associated with Images
Existing Image Crowdsourcing Games
How our game is different
• Anyone can play and can provide us with useful information.
• No expertise necessary
• Capitalizes on strengths of humans and algorithms– Humans are better than algorithms at identifying
similarity of images
Game Mechanics
Game Mechanics
How Leaf Sets Are Constructed
• Designed to bring in useful data
• Not too easy or too hard
• Curvature based histograms used to get features from leaf shapes.– These features are used to find distance between
all possible pairs of leaves.
What’s in it for us if people play this game?
• Identify errors in the dataset
• Discover if color helps humans identify leaves
• Feedback on how enjoyable or difficult the game is
Game Variations
Before Leaf is Chosen
Multiple Guesses Skip
After Leaf is Chosen
Contest after Game is Finished Contest Previous Round
Feedback Mechanism
When Feedback Occurs
Mechanical Turk Trial
1 2 3 4 50
5
10
15
20
25
30
Enjoyment
Num
ber C
orre
ct
Mechanical Turk Trial
1 2 3 4 51
2
3
4
5
Difficulty
Enjo
ymen
t
Summary
• Anyone can help in Computer Vision research work.
• Games can be fun for players and useful for researchers.
• Humans are better than machines in judging the similarity of two images.
Funding
This work is made possible by National Science Foundation grant number 0968546
