Multimedia Surveillance Data Mining for Analytics Jai sri ganeshay namah sri saraswatyiy namah om namah shivay jai bajarang bali dada jai sai baba jai gaytri mata jai tirupati dada jai mahalaxmi mata jai kuldevi ma chamunda mata jai surya chandra mangal budh guru sukra sani rahu ketu devta santi bhavantu jai mariyamma mata jai anjnaiya dada jai mahavir swami bhagvan jai swami swarupanand saraswati ji bhagvan jai dakor na thakorji jai servdevodevi namah stute om shanti bhavantu
Transcript
Multimedia Surveillance Data Mining for Analytics
Jai sri ganeshay namah sri saraswatyiy namah om namah shivay jai bajarang bali dada jai sai baba jai gaytri mata jai tirupati dada jai mahalaxmi mata jai kuldevi ma chamunda mata jai surya chandra mangal budh guru sukra sani rahu ketu devta santi bhavantu jai mariyamma mata jai anjnaiya dada jai mahavir swami bhagvan jai swami swarupanand saraswati ji bhagvan jai dakor na thakorji jai servdevodevi namah stute om shanti bhavantu
Outline
Motivation Introduction Problem Definition Proposed Approach for Evacuation Scenario Statistical data mining Model Results Obtained on VAST challenge dataset Future Work
Motivation
Wide use of surveillance system for monitoring the behavior of people, vehicles
Objective: To detect suspicious behavior based on available multimodal data
Strong need for automated or semi automated means for suspicious behavior detection and prediction
Introduction
Video Surveillance SystemsExpensiveRich amount of information
RFID Surveillance SystemsNot very expensiveLimited amount of informationTherefore can use appropriate sensory data for the task at hand and can even use multiple modalities for redundancy and cost-savings
Introduction
Suspicious movement detection scenariosExplosion event followed by evacuationOpen firing event followed by chaos Even a small accident in office or street leads
to considerable change in normal movement pattern
Need quick way of analyzing and also the way of predicting suspicious behavior
Introduction Video Surveillance
Systems Observing large volume of
data by a few observers Suspicious patterns may
not be explicitly visible to observer
RFID Surveillance Systems Suspicious patterns are not
visible to observer
Therefore some automated pattern analysis or data mining is required
Problem Definition
To build an intelligent surveillance system’s tool that can, Help investigate suspicious behavior for different
scenarios, Automatically or semi automatically incorporating the
intuitions that are similar to the one that security officer can have.
Where investigation should give answers to when?, where?, who?, what? etc.
Evacuation Dataset of IEEE VAST Challenge 2008 In 2007 an explosive device was set off at a Miami,
Florida DOH building, resulted in casualties and damage Employees & visitors wore badges (RFID) Data provided
Time: Ticks, representing intervals between tag readings Person Id: Tag identification of all employees and visitors Xcor: the location x-coordinate Ycor: the location y-coordinate
The file includes data before and throughout the incident.
Input Trajectory Data Trajectory of 82
people over total Time Duration of 837seconds on building map of 91x61 grid space.
Making sense of this data seems extremely difficult
Questions for the Evacuation Scenario
Where was the device set off? Identify potential suspects and/or
witnesses to the event. Identify any suspects and/or witnesses
who managed to escape the building. Identify any casualties.
Proposed Approach
Gather intuitions (hypotheses) for the scenario Compute the possibly useful parameters like
average speed in certain time interval, average traversed area in certain time interval
Build a statistical model using the computed parameters combined with the hunches
Perform Analysis
Intuitions for the Evacuation Scenario Evacuation Scenario in office environment
where explosion event is followed by evacuation.
Intuition 1 [Normal Behavior]:Usual movement of people will be low before
explosion event and it will increase drastically afterwards to evacuate the scene.
Intuitions for the Evacuation Scenario Intuition 2 [Suspicious Behavior]:
Suspicious persons would try to run away from explosive device location before the explosion happens.
Intuition 3 [Victims Behavior]: Victims would have normal behavior before
the explosion event but will be injured or have fainted or be dead on explosion.
Formulation of Statistical Model
Parameters for Statistical Model:Time Window: The analyst needs to input
appropriate time window parameter for the statistical model to compute the following
Speed of each PersonArea Traversed by each PersonAverage Global Speed of PeopleAverage Global Area Traversed by People
When did the Explosion happen?
Obtain the Global Average Speed.
Find the Global Maximum value from
Based on intuition1 we can consider this GM as approximate start time of Explosion
Where was the device set off? Average speed and Average
area traversed by the Victims will be almost near to zero after explosion event.
They may not be able to reach to the Evacuation Area.
They will be found within or very near to the explosion area.
Location cluster of such people represents the area of explosive device.
Where is Evacuation Location?
Based on intuition1 people are trying to reach to evacuation place.
High density region at end times would be representing evacuation place.
Who are the Suspects? Average speed and Average
area traversed by the persons will be higher before explosion event.
Suspicious person should have visited Explosion location just prior to the explosion.
They might either reach Evacuation before others or will escape without entering Evacuation area.
INPUT DATATime & Location of
each person
Computing required Parameters ( speed, area
Covered within time window)
Finding the Start TimeOf Event (Explosion)
Analyzing the speedbefore Event (Explosion)
High speed people in this duration is set of
Suspicious people
Analyzing the speedafter Event (Explosion)
Low speed people in this duration is set of
Victims
Traversed through Event (explosion location) are
strong set of suspects
Clustered at event(explosion) location
Evacuation Model
Future work
Need to incorporate other data capturedVideo dataAudio dataFire Alarm, Temperature data etc.
Come up with a Mining/Analytics tool to facilitate such investigations.
Definition Data mining:
“is the process of automating information discovery” or “is the exploration and analysis by automatic or semiautomatic means,
of large quantities of data in order to discover meaningful patterns and rules”
“multimedia data mining” “knowledge discovery in a multimedia database” “extraction of implicit knowledge, mm data relationships or other
patterns not explicitly stored in multimedia files”
Motivation
Tremendous benefits of traditional data mining is proven for structured data.
Now its time for extending the mining techniques for unstructured, heterogeneous data.
MDM Challenges and Problems Feature Selection Dimensionality Reduction: for reducing the problem
size , enables learning algorithms to operate faster and effectively.
Feature construction / transformation: by constructing new features from the basic features set.
How to analyze the heterogeneous data that consist of text, graphs, images, sounds, videos and other kind of sensor data? Multimedia data has complex structures that can not be processed as a whole by available data mining algorithms.
Tokenizing textual document into words and phrases has proven to work reasonably well for retrieval but images, audio, video etc cannot be readily decomposed into such semantic units.
