Date post: | 28-Dec-2015 |
Category: |
Documents |
Upload: | brett-gregory |
View: | 216 times |
Download: | 0 times |
MODELING VISUAL CLUTTER PERCEPTION USING PROTO-OBJECTS
Chen-Ping Yu
Prof. Dimitris Samaras
Prof. Greg Zelinsky
Introduction• The goal: model human visual clutter perception.
• Visual clutter: A “confused collection”, or a “crowded disorderly state”. Increasing visual clutter.
• Set Size Effect: search performance decreases as set size increases. How to quantify set size?
Introduction• What are proto-objects?
• Regions of locally similar features, they can be objects, object parts, or just pieces that come together to form objects.
• What is the proto-object clutter model?• Segments an image into proto-objects, use the normalized number
of proto-objects as the clutter measure of the image.
Stimuli• 90 800x600 images of real world contexts
• Selected from the SUN09 Database• 6 object-count groups, each contains 15 images• Human labeled objects are provided with SUN09
31 32 33
36 37 39
31~40 objects
15 images
51 52 53
55 57 58
51~60 objects
15 images
3 5 7
7 9 10
1~10 objects
15 images
90 images total
Stimuli
17
48
3
57
Behavioral method• Subjects
• 15 human subjects age from 18 to 32
• Method• Rank order the 90 images from least to most cluttered• Using a Matlab GUI• Participants were told to use their own definition of clutter
• Practice• 12 practice images prior to actual testing
Behavioral method• 2 displays were used
• Images were shown at random
• Bottom monitor subtended a visual angle of 27° x 20°
• Had the option to correct the ordering
• A experiment lasted roughly 45~60 min
• Average pair-wise rater agreement: ρ = 0.692
Computational method• 1. superpixel preprocessing
• K = 600• Need to merge the resulting over-segmentation
Computational method• 2. Mean-shift clustering
Computational method• 2. Mean-shift clustering, more examples
Computational method• 2. Mean-shift clustering in HSV color space
• Median color of each superpixel
Computational method• 3. Merge neighboring superpixels that belong to the same
color cluster
600 superpixels 207 proto-objects (0.345)
Computational method• Proto-object visualization
• Fill each proto-object using the median of the member-pixel colors
Results• Spearman’s rank order correlation
• Ρ = 0.814, p < 0.001
Results• Robust to different parameter/color space settings
• Each correlation is computed using the optimal MS bandwidth
Results• Comparing to other clutter models
Results• More visualized proto-object results
Some further experiments• Does visual clutter perception change when viewing
images of different sizes?
• Experiment: Large images vs small images• Same 90-image dataset, large images = original 800x600 size;
small images = quarter size (200x150)
• Same behavioral setting• 12 practice images• Same Matlab GUI
Further experiments• Small images subtended a
visual angle of 6.75° x 5 °
• 20 undergraduate students from SBU
• Followed the same procedure as the large-image setup
• Average inter subject correlation: 0.58
Further experiments• Highlights
• Human’s small image rating vs large image rating: ρ = 0.953..!• Proto-object model’s small image correlation: ρ = 0.852
Further experiments• Comparing with other clutter models
• Proto-object model stayed the most consistent
Conclusion• Number of object (set size) is a poor predictor to visual
clutter
• Set size may be better quantified/represented by proto-objects• All segmentation-based methods outperformed the feature/non-
segmentation based models
• Can proto-object’s spatial density predict search performance, and/or number estimation?
• Can proto-object’s spatial distribution predict gaze?