Mega-boundariesTEMPORAL VIDEO BOUNDARY DETECTION

Mega-boundaries

Mega boundaries are defined between macro segments that exhibit different structural and feature consistency.

A good example of mega boundaries application is commercial detection

Commercial detection

Common method is detection of high activity rate and black frame detection coupled with silence detection

Lienhart 1997 Use of monochrome images , scene breaks , and action.

Blum 1992 Use of black frames and activity detector.

Iggulden 1997 Distance between black frame sequences.

Dimitrova 2002 Automatically spots repetitive patterns. Must be identified before

recognizing

Commercial detection

Nafeh 1994 Learning and discerning of broadcast using Neural Network

Bonner 1982

McGee 1999

Novak 1988

Y. Li 2000

Agnihotri 2003

Features for Commercial detection

Mega boundaries detection method’s are based on what features we have on the test video

Unicolor Frame for commercial break

High visual activity

Letterbox format

Dataset of 8 hours of video from TV programs

Feature data consists of 600000 frames

Triggers and Verifiers

Trigger - Features that can aid in determining the location of the commercial break

Verifier – Features that can determine boundaries of the commercial break

We use the time interval between detected unicolor frame as triggers

Presence of a letterbox change or high cut rate expressed in terms of low cut used as verifiers.

Constrains on commercial breaks are longer than 1 minute and shorter than 6 minute.

Bayesian Belief Network Model

Directed acyclical graph (DAG) The nodes correspond to variables

The arcs describe direct casual relationship between linked variables

The strength of these links is given by conditional probability distributions

P(x1,..,xn)=P(Xn|Xn-1,..,x1)*. . . *P(x2|x1)P(x1) Ω(x1,..xn) - Variables define as DAG

P(xi|∏i)=P(xi|x1,..,xn)

P(.|.) is a cpd (conditional probability density)

Using probability density function and chain rule

Bayesian Belief Network Model

Probability for the verification node using pdf and chain rule

Probability for potential commercial

Probability for separator

Bayesian Belief Network Model

Probability for sequence of black frames

Probability for key frame distance

Evolved Algorithm

Challenge to create algorithm for all countries broadcast difference

Genetic algorithms implement a form Darwinian evolution.

Uses chromosome etc..

Eshelman’s CHC algorithm

CHC is general algorithm with 3 features Monotonic

CHC prevents parents from mating if their genetic is too similar.

CHC uses soft restart

CHC for our Experiment

Default Parameters. 50 chromosomes, divergence rate of 35%.

Each parameter was coded as a binary string.

Each chromosome was decoded into set of parameters for the commercial detector and this detector was given a test video stream.

Correct label for the video frames were detected by human

Highest precision and recall was achieved with precision + recall

Results

First data set 8 hours of TV broad cast consisting of 13 TV programs

1.5 hours of 28 different commercials

Second dataset 4 hours of TV broad cast consisting of 11 TV programs

1 hour of 35 different commercials

FN,FP,TP,TN

Recall=TP/(TP+FN)

Precision=TP/(TP+FP)

Results

Using First data set

Results from first 4 experiment (recall and precision) 80.8% and 92.6% , 80.8% and 92.6% , 79.7% and 87.4% , 81.3% and 94.3%

Experiment 5 used experiment 4 and result was 88% and 90 %

Using second data set

This dataset was acquired after the algorithm

Test to the Genetic Algorithm

Results are shown in Figure.

Conclusion

Boundary segmentation in videoVisual scene segmentationMultimodal story segmentationCommercial detection

Questions ?