Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes
IEEE TCSVT 2011Wonjun Kim
Chanho JungChangick Kim
OutlineIntroductionProposed MethodExperiment ResultApplicationConclusion
IntroductionProblem occurs when background is highly textured
Proposed Method
Feature RepresentationEdge orientation histogram (EOH)Color orientation histogram (COH)Temporal Feature
Self-ordinal MeasureSaliency MapScale-invariant Saliency Map
Edge Orientation Histogram (EOH)
1. Compute the edge orientation of every pixel in the local region center at the pixel
2. Quantized into K angle in the range of [,]3. Compute the histogram of edge orientation
m(x,y,n):edge magnitude(x,y,n):quantized orientation
𝑖
local region
Color Orientation Histogram (COH)
1. Quantize the angle in HSV color space in the range of [,] into H angles
2. Compute the histogram of color orientation
s(x,y,n):saturation value(x,y,n):quantized hue value
Temporal FeatureCompute the intensity differences between frames
Feature at the pixel of frame
P :total number of pixels in local regionj :index of those pixels in P :user-defined latency
Self-ordinal MeasureDefine a 1(K+1) rank matrix by ordering the
elements of EOH(COH) ex:
Self-ordinal Measure
Saliency Map of Edge and ColorCompute the distance from the rank matrix of
center region to surrounding regions
Saliency Map of Edge Saliency Map of Color
N :total number of local regions in a center-surround window
, :maximum distance between two rank matrices
Spatial Saliency MapCombine the edge and color saliency
Combining with Temporal SaliencyCompute the SAD of temporal gradients between
center and the surrounding regions
Combine the spatial and temporal saliency
Scale-invariant Saliency MapCombine 3 different scales of saliency Map
(3232, 6464, 128128)
3232 1281286464
Algorithm
Experiment Result
Static ImagesVideo Sequences
Experiment ResultStatic Image
Local region = 55center-surround window = 77K = 8, H= 6 = 40, = 24
Video Sequence = 49Speed: 23ms per frame (43 fps)
Static Images
Static Images
Video Sequences
Video Sequences
Application
Image RetargetingMoving Object Extraction
Image Retargeting
Image Retargeting
Moving Object DetectionG:the set of salient pixels in the ground truth imageP:salient pixels in the binarized object mapCard(A):the size of the set A
Moving Object Detection
ConclusionOrdinal signature can tolerate more local feature
distribution than sample values.The proposed scheme performs in real-time and
can be extended in both static and dynamic scenes.