1

Computer VisionComputer VisionCS101: Wk 08 Topical LectureCS101: Wk 08 Topical Lecture

What is Computer VisionWhat is Computer Vision

““The goal of Computer Vision is to make The goal of Computer Vision is to make useful decisions about real physical useful decisions about real physical objects and scenes based on sensed objects and scenes based on sensed imagesimages””Image and Video Image and Video UnderstandingUnderstandingMIT Copy DemoMIT Copy Demo

2

Computer Vision

Image IN

Symbolic Decision or ModelOUT

ComputerGraphics

Model IN Image OUT

Relationship between Computer Relationship between Computer Graphics and Computer VisionGraphics and Computer Vision

Writing a Program to Detect FacesWriting a Program to Detect Faces

3

4

Face DetectionFace DetectionWhat is a What is a (human) face?(human) face?

Your Your description description should be should be invariant to invariant to 3D rotation, 3D rotation, illumination, illumination, facial facial expressionexpression

Viola/Jones Face Detector (2001): Using implementation in OpenCV

Results of Schneiderman/Kanade Face Detector

5

Taxonomy of Vision ProblemsTaxonomy of Vision Problems

1.1. Reconstruction:Reconstruction:–– estimate parameters of external 3D world.estimate parameters of external 3D world.

2.2. Segmentation and Tracking:Segmentation and Tracking:–– partition partition I(x,y,tI(x,y,t) into subsets of separate objects.) into subsets of separate objects.–– Given an object in Given an object in I(x,y,tI(x,y,t), find the same object in I(x,y,t+1)), find the same object in I(x,y,t+1)

3.3. Recognition:Recognition:–– action recognition: activity, gesture, expressionaction recognition: activity, gesture, expression–– face recognitionface recognition–– object recognitionobject recognition

Ref: Jitendra Malik, UCB

1. Reconstruction1. Reconstruction

To recover 3D geometric models from To recover 3D geometric models from images or videoimages or video

Inverse Problem of Computer GraphicsInverse Problem of Computer Graphics

Shape from XShape from X–– Shading, Motion, StereoShading, Motion, Stereo

Video simplifies the problemVideo simplifies the problemOptimization Optimization problemproblem

6

Shape from StereoShape from Stereo

Stereo ResultsStereo Results

7

8

Path Planning for Biped RobotsPath Planning for Biped Robotshttp://http://www.cs.cmu.edu/~Joel/footstep/pictures.html#planswww.cs.cmu.edu/~Joel/footstep/pictures.html#plans

Ref: Tsai and Shah, UCF

Shape From Shading

9

Shape From Texture

Ref: Debevec et. al., SIGGRAPH 96, UCB

10

Ref: Debevec et. al., SIGGRAPH 96, UCB

Movie:

11

Reconstruction from Old PaintingsReconstruction from Old Paintings

Reconstruction: Reconstruction: Recovering Camera TransformationRecovering Camera Transformation

12

RegistrationRegistration

Reference Image

Aerial Image

Qs: Where was the camera when aerialimage was taken

13

2. Segmentation and Tracking2. Segmentation and Tracking

SegmentationSegmentation–– Going from Images to ObjectsGoing from Images to Objects

IllIll--posed problemposed problem

Hard to define clearlyHard to define clearly……What is meant by an object?What is meant by an object?

14

•Manual segmentations by different people are also not the same•Transparency effects are not taken into account

15

Video SegmentationVideo Segmentation

16

TrackingTracking

Given an Object in Frame Given an Object in Frame tt, find that , find that object in frame object in frame tt+1+1

Ref: Khurram Shafique, Alper Yilmaz, Mubarak Shah, UCF

RecognitionRecognition

Activity RecognitionActivity RecognitionFace RecognitionFace RecognitionGender RecognitionGender Recognition

17

3. Recognition3. Recognition

Action RecognitionAction Recognition

Finite State MachinesMarkov ModelsStochastic Context Free Grammars