Object Based Processing for Privacy Protected Surveillance

Karl MartinKostas N. Plataniotis

University of TorontoDept. of Electrical and Computer Engineering

February 6, 2008

2Confidential - Not to be reproduced without permissionFebruary 6, 2008

Overview

Introduction

Object Based Image/Video Processing

Object Based Coding/Compression

Privacy Protection via Object Processing

Smart Surveillance Technologies

3

Digital Signal Processing

Image and Video Processing

• Digital Camera Processing• cDNA Micro-array Imaging• Visual Object Compression• Universal Multimedia Access

Multimedia Security

• Authentication & Fingerprinting• Image and Video Encryption• Visual Secret Sharing• Secure Streaming• Watermarking

Biometrics & Surveillance

• Face Recognition• Gait Recognition• Visual Surveillance

Professor:• Kostas Plataniotis

(kostas@comm.utoronto.ca)

4Confidential - Not to be reproduced without permissionFebruary 6, 2008

Object Based Image/Video Processing

Semantic processing of frames/video sequences Objects:

– Face, body, suitcase, vehicle, etc.– Defined by:

Shape + Texture

Enables:– Privacy Protection– Event Detection– Etc.

5Confidential - Not to be reproduced without permissionFebruary 6, 2008

Object Based Image/Video Processing

Requires detection and segmentation– Rough box or accurate contour

– Computer vision and pattern recognition techniques

6Confidential - Not to be reproduced without permissionFebruary 6, 2008

Object Based Coding/Compression

Traditional frame/video sequence coding– MPEG-1/2/4, H.263, Motion JPEG, Motion JPEG2000– Content unaware

Object based coding– MPEG-4, ST-SPIHT Wavelet (UofT, Martin and Plataniotis 2006)– Content aware – enabling technology for object based processing

VideoCompression

EngineCompressedData Stream

FrameObject

CompressionEngine

Compressed Shape and TextureStream (Object 1)

Compressed Shape and TextureStream (Background)

Compressed Shape and TextureStream (Object 2)

7Confidential - Not to be reproduced without permissionFebruary 6, 2008

Privacy Protection via Object Processing

Object obscuration – irreversible

Object Encryption - reversible

Texture Removal (blackout)

Texture Blur

Decryption Key

8Confidential - Not to be reproduced without permissionFebruary 6, 2008

Privacy Protection via Object Processing

SecST-SPIHT Secure Object Coding (UofT, Martin and Plataniotis, 2007)– Secure texture OR texture+shape

Encryption can be reversed with correct key– Real-time or offline release of key based on authority– Allows investigation of original content

Original Frame Secured Texture SecuredTexture+Shape

9Confidential - Not to be reproduced without permissionFebruary 6, 2008

Privacy Protection via Object Processing

Example System Topologies

ObjectSegmentation

SecST-SPIHT(secure coding)

Recorder/Storage Server

AuthorizedKey Release

Local Real-time Monitor

CentralReal-time Monitor

Law Enforcement

PhysicalTransfer

KeyManagement

10Confidential - Not to be reproduced without permissionFebruary 6, 2008

Privacy Protection via Object Processing

Example System Topologies

ObjectSegmentation

Recorder/Storage Server

AuthorizedKey Release

Local Real-time Monitor

CentralReal-time Monitor

Law Enforcement

PhysicalTransfer

KeyManagement

SecST-SPIHT(secure coding)

11Confidential - Not to be reproduced without permissionFebruary 6, 2008

Smart Surveillance Technologies

Digital vs. Analog

Digital Analog

Operating Central operation Isolated, site-by-site

Managing Central management Difficult to manage large number of sites

Recording Flexible framerate/quality Fixed framerate/quality

Storing Flexible based on storage, compression parameters – highly scalable

Tapes requiring substantial storage space – potential video loss during tape change

Searching Single query search across multiple storage sites – near instantaneous

Manual, physical retrieval of tapes

12Confidential - Not to be reproduced without permissionFebruary 6, 2008

Smart Surveillance Technologies

General processing stages for smart processing

Detected Behaviours – Violence– Suspected objects– Robbery– Vandalism– Overcrowding– Trespassing

EventDetection

ObjectRecognition

TrackingBehavoural

Analysis

13Confidential - Not to be reproduced without permissionFebruary 6, 2008

Smart Surveillance Technologies

Integrated Approach– Technical infrastructure– Organizational human-led procedures– Ethical framework (privacy vs security)– Behavioural studies

Requirements for Implementation– Robust– Adaptable– Easy installation– Off-the-shelf hardware– Real-time operation

14Confidential - Not to be reproduced without permissionFebruary 6, 2008

Smart Surveillance Technologies

Example Commercial Solutions

Compression Other Features

MC Electronics MPEG-4 Audio, PTZ control, alarm inputs

Visiowave Visiobox 3D Wavelet PTZ control, motion detection 3

Baxall Analog and digital interfaces

Teleeye III+ NF Wavelet Motion detection, wireless

Sony SNCV10P Wavelet Multiple interfaces, embedded

LINUX, built in security

15Confidential - Not to be reproduced without permissionFebruary 6, 2008

END