© Fraunhofer
Smart Video Surveillance and Privacy
Dr.-Ing. Erik Krempel,
Fraunhofer IOSB, Karlsruhe, Germany
© Fraunhofer IOSB 2
What can we do with video analytics…
SMART VIDEO SURVEILLANCE
© Fraunhofer IOSB 3
From Visual to Symbolic Video Surveillance
• The classic approach of video surveillance is visual videomonitoring by security staff.
• Today, video analytics are more and more in use as assistance systems. Mainly to attract and sustain attention of the staff to possible points of interst.
• Current research is going towards useage of advanced video analytics for symbolic situation awareness and „privacy-aware management by exception“
Classic Approach Intelligent Video Analytics
Privacy-aware Situation
Awareness Tools
© Fraunhofer IOSB 4
Video Analytics for Scene Understanding
Object Tracking
Activity RecognitionSemantic SceneUnderstanding
Scene Context Recognition
Object DetectionImage Enhancement
© Fraunhofer IOSB 5
Current Research: Advanced Objekt Detection, Tracking and Re-Identification
• Multi-camera multi-object detection and tracking of persons and vehicles
• indoor / outdoor applications with complex illuminationconditions
• self-calibrating cameras (automated geometric and colorcalibration)
• New approaches on dynamic learning of objects‘ unique features
© Fraunhofer IOSB 6
Towards Activity Recognition for (Global) Scene Understanding
• Classification of activities in videos.
© Fraunhofer IOSB 7
Person Retrieval
• Component that is looking for similar persons
in archive in a specific time interval
• Features of the person of interest (color features and texture features) are compared to already computed features of other tracks
• System generates a listof tracks, sorted basedon their similarity tothe person of interest
• Operator can verify the results visually
© Fraunhofer IOSB 8
Why do we need to certify smart video surveillance systems
CERTIFICATION
© Fraunhofer IOSB 9
Security
What is the error rate of video processing algorithms?
False positive rates
False negative rates
Is the system robust against environment conditions?
Rain on the camera
Light conditions
Is the system safe against attacks?
The systems keeps secrets for itself (Confidentiality)
The systems stays operational in the presence of an attacker (Availability)
The systems detects the same results in the presence of an attacker (Integrity)
© Fraunhofer IOSB 10
Trust / Transparency
© Fraunhofer IOSB 11
Trust
How can you achieve real transparency in such complex systems?
Is there a human preventing the system from making wrong decisions?
Does the systems prevent discrimination?
© Axis Communications
© Fraunhofer IOSB 12
Efficiency dimension
Is the system able to reduce crime / increase security?
Is the system easy to use?
Can you extend the system / use components of another manufacturer?
© Fraunhofer IOSB 13
Freedom Infringement
Does the system prevent discrimination?
Are video processing algorithms restricted to legal applications?
Does the system prevent misuse by the operator?
Security of the data processing…
Secure against data theft…
Purpose limitation…
Errors by the systems…
© Fraunhofer IOSB 14
RESEARCH WORKPrivacy in smart video surveillance
© Fraunhofer IOSB 15
Situation-dependent video surveillance
Alexander Roßnagel, Monika Desoi, and Gerrit Hornung (2011)
© Fraunhofer IOSB 16
Idea: Privacy-aware Surveillance Workflows
Default Mode: Optimized for privacy Assessment Mode:Event-specific, privacy preserving
Eventdetected
Eventconfirmed
Eventresolved
Investigation Mode:Event specific functions unlocked
© Fraunhofer IOSB 17
Default Mode: Optimized for Privacy
Computer vision algorithms in background
Abstract representation of observed environment visualized
No access to video data
No access to archived data
Exposes as little information about observed persons as possible
© Fraunhofer IOSB 18
Assessment Mode: Event-specific, Privacy Preserving
Eventdetected
Assessment view according to event type
Anonymized live video data released
Possibly anonymized access to a limited buffer of recorded (video) data
Only selected cameras
© Fraunhofer IOSB 19
Investigation Mode: Event-specific Functions Unlocked
Eventconfirmed
Eventresolved
Allows additional privacy intrusions for investigation purposes
I.e., retrieving the person who dropped a piece of luggage
Restrict additional analyses to persons related to the event under investigation
© Fraunhofer IOSB 20
IMPLEMENTATIONHow to build systems with this concept…
© Fraunhofer IOSB 21
Security Use-Case
© Fraunhofer IOSB 22
Safety Use-Case: Prototype NurseEye
© Fraunhofer IOSB 23
Operator interaction
Default mode:
No Access
Assessment mode:
Access to anonymized video
Investigation mode:
Full access to video to help in emergency handling
© Fraunhofer IOSB 24
Transparency
All cameras come with displays showing the current mode and data usage
Displays become monitor for chat in the investigation mode
Default mode Assessment mode Investigation mode
© Fraunhofer IOSB 25
Video NurseEye
© Fraunhofer IOSB 26
TOOLS
© Fraunhofer IOSB 27
User Study on Anonymization Techniques: Introduction
Which obfuscation technique should be used?
Regarding privacy (identity leakage), utility and perceptual video quality
ROI Anonymized person
Blurring Silhouette Edge detection PixelizationBlurring(gray scale)
© Fraunhofer IOSB 28
Results: Subjective assessments
1 = silhouette; 2 = pixelization; 3,5 = edge detection; 4 gray scale blurring; 6-8 color blurring
Error bars represent one standarddeviation of the data sample
Perceived privacy protection and perceived image quality
© Fraunhofer IOSB 29
Video anonymisation
Edge detectionPixelization
Gaussian blurring Silhouette
© Fraunhofer IOSB 30
Adaptive „Privacy-Masking“ for Pan/Tilt/Zoom Camera
• Advanced Privacy Masking for semi-stationary PTZ-cameras
• Given position of camera over ground (or an elevation model of the site) optical distance toobjects in the scene is estimated
• Given a parameter for „privacy-preserving resolution“ video is pixelized or blurredinhomoge-nously depending on distance to object.
• Highly senstive areas (e.g. buildings / windows) can be blacked out of the stream bydynamic adaptable polygons (depending on pan/tilt/zoom settings)
© Fraunhofer IOSB 31
Information Flow Tracking in NEST
Video &Observation
archive
PEP
HMIGeoViewer
Observations
PEP
Requests
HMIStreamViewer
PEP
Imageexploitationalgorithm
Imageexploitationalgorithm
Imageexploitationalgorithm
PEPIMG
PolicyDecision
Point
PolicyInformation
Point
Policy
Observations
Control
PEP
Acce
ss Co
ntr
ol
OOWM
Association
PEP HLS-Alarms
PEP Classification
PEP Fusion
© Fraunhofer IOSB 32
Discussion
“Data is not an asset, it’s a liability”-Marko Karppinen-
Contact Information:Erik Krempel
Fraunhofer IOSBFraunhoferstr. 176131 Karlsruhe, Germany
[email protected]+49-721-6091-292