Video Recognition Systems of NRC
http://iit-iti.nrc-cnrc.gc.ca/about-sujet/cv-vi_e.html
http://synapse.vit.iit.nrc.ca (www.videorecognition.com)
Leader: Dr. Dmitry Gorodnichy
DNI IARPA VACE, Phase III Meeting Washington DC,
1 November 2006
2. Video Recognition projects and interests (Dr. Dmitry Gorodnichy)
NRC - IIT
• From Space Shuttle (Canadarm &
Canadarm2), to Hollywood, to Fine
Arts, to Security applications
3. Video Recognition projects and interests (Dr. Dmitry Gorodnichy)
VRS
• Part of Computational Video Group (formed in 2001)
• Our Mandate: Develop Video Recognition
technologies for Canadian companies and OGD’s
• Higher mission:
Computer Vision allows computers to see.
Perceptual Vision allows computers to understand what they see.™
Test-bed and Criteria:
If you are able to recognize something, why computer can’t ?..
4. Video Recognition projects and interests (Dr. Dmitry Gorodnichy)
A bit of history
First: Canadarm2
Then: Nouse™, ACE Surveilance
• Canada Borders Services Agency (CBSA)
• Canadian Police Research Center (CPRC, RCMP)
• Ottawa Health Center
• U of Ottawa Music Dept.
• Several Canadian/Ontario private companies 5. Video Recognition projects and interests (Dr. Dmitry Gorodnichy)
Key applications
and partners
Domain 1: Security, Surveillance and Monitoring.
Domain 2: Visually-enabled computer-human
interaction (inc. hand-free user interfaces)
Domain 3: Visually-enhanced communication and
Intelligent video processing (eg annotation of
video)
(And partners)
6. Video Recognition projects and interests (Dr. Dmitry Gorodnichy)
Technologies developed
1. Neuro-associative memorization/ recognition
Face Recognition from (low-res) Video
(Better than correlogram) Object learning
2. (Multiple) object detection and tracking
Automated tele-operating, …
3. Nose (convex-shape) tracking
Hands-free interfaces
4. Critical Evidence Snapshot extraction
Intelligent Surveillance
5. Fingers, hands detection tracking
Piano playing annotation
7. Video Recognition projects and interests (Dr. Dmitry Gorodnichy)
1. Neuro-associative memorization/ recognition
Face Recognition from (low-res) Video
(Better than correlogram) Object learning
2. (Multiple) object detection and tracking
Automated tele-operating, …
3. Nose (convex-shape) tracking
Hands-free interfaces
4. Critical Evidence Snapshot extraction
Intelligent Surveillance
5. Fingers, hands detection tracking
Piano playing annotation
Research & Results
Technologies developed
(X,Y)
name
8. Video Recognition projects and interests (Dr. Dmitry Gorodnichy)
What we do
• Lots of research and … coding
– Working with companies: consulting, making
prototypes
– Working with OGD: consulting, joint proposals
• Lead IEEE-published workshops:
at CVPR, CRV, IJCNN
– on Face Processing in Video (2003-2005)
– on Video Recognition (VideoRec’07)
– Neuro-technology IJCNN demonstration workshop
9. Video Recognition projects and interests (Dr. Dmitry Gorodnichy)
#1:
Face Recognition in Video
Bad news: low-resolution (TV:320x260) data i.o.d = 12 pixels !
Good news: face detectors (Viola,CMU,MIT,Pitts) can detect such faces !
Solution: make use of temporal domain!
i.e. Learn & Retrieve face from sequences (using Recurrent Neural Networks)
10. Video Recognition projects and interests (Dr. Dmitry Gorodnichy)
#1. Efficient visual attention mechanisms (motion etc)
#2. Decision based on accumulating results over time.
#3. Three main principles of neuro-processing
1. non-linear processing,
2. distributed collective decision making
3. synaptic plasticity.
They allows the system:
a) to accumulate learning data in time by adjusting
synapses,
b) to associate a visual stimulus to a semantic
meaning based on the computed synaptic values
Keys principles
(from biological systems) (X,Y)
“Paul”
11. Video Recognition projects and interests (Dr. Dmitry Gorodnichy)
Demo
Solution:
- 12 pixel i.o.d. face model + accumulation over time while tracking (both in training and recognition)
- combination of neuro-biological and computer vision approaches
Tested: on TV shows, on computer user recognition