Visual Servoing in Non-Rigid Environments: A Space-Time Approachp pp

D Santosh & C V Jawahar

International Institute of Information Technology (IIIT),

IIIT

Hyd

erab

ad Hyderabad, India

The Problem

IIIT

Hyd

erab

ad

Santosh, Jawahar. Visual Servoing in Non-Rigid Environments 2

Desired Image I*

F

F*

Initial Image IF

Desired Pose F*

Challenges with Non-Rigidity

• Varying Pose• Non-rigid g

Correspondences• Model-free SolutionsModel free Solutions

IIIT

Hyd

erab

ad

4

Challenges with Non-Rigidity

• Varying Pose• Non-rigid g

Correspondences• Model-free SolutionsModel free Solutions

IIIT

Hyd

erab

ad

5

Space-Time Modeling

M d li D f ti• Modeling Deformations

Configuration of moving points

IIIT

Hyd

erab

ad

Santosh, Jawahar. Visual Servoing in Non-Rigid Environments 6Non-Rigid Motion

Space-time Modeling

Pk = [Xk Yk Zk Tk]k [ k k k k]

pk = M Pkpk k

e(t) = c(t) – c(t)*e(t) c(t) c(t)

IIIT

Hyd

erab

ad

Santosh, Jawahar. Visual Servoing in Non-Rigid Environments 7

Space-Time ModelingS*

s

S

sDesired Pose

R,t

Initial Pose

Desired Pose

IIIT

Hyd

erab

ad

Santosh, Jawahar. Visual Servoing in Non-Rigid Environments 8

Camera Motion Estimation

Desired Pose

Initial Pose

IIIT

Hyd

erab

ad

9

Degenerate Configurations

• Exploit the availability of p ymultiple planes

Hres = f (H1,H2, ..., Hn)

IIIT

Hyd

erab

ad

10

Degenerate Configurations

• Exploit the availability of p ymultiple planes

Hres = f (H1,H2, ..., Hn)

IIIT

Hyd

erab

ad

11

Final Control• Optimal Control (3D)

v α –t, ω α -uө • Feature Visibility• Hybrid Control• Hybrid Control

IIIT

Hyd

erab

ad

Santosh, Jawahar. Visual Servoing in Non-Rigid Environments 12

Summary

IIIT

Hyd

erab

ad

Santosh, Jawahar. Visual Servoing in Non-Rigid Environments 13

Experimental Analysis

IIIT

Hyd

erab

ad

Santosh, Jawahar. Visual Servoing in Non-Rigid Environments 14Objects used in the experiments

ResultsTracked Interest Points

ResultsII

IT H

yder

abad

15

Results

IIIT

Hyd

erab

ad

Santosh, Jawahar. Visual Servoing in Non-Rigid Environments 16

Results

Camera Trajectory

IIIT

Hyd

erab

ad

Santosh, Jawahar. Visual Servoing in Non-Rigid Environments 17

Summary & Future Work

• Enhanced VS to tackle non-rigid objects• Opens the door for various applications• Model the object using space-time features and

exploit the projective geometry between the views

l i i• Real time camera motion– Multiple view geometry in the space-time [Sato_ACCV06]

• Planarity Assumption

IIIT

Hyd

erab

ad

Santosh, Jawahar. Visual Servoing in Non-Rigid Environments 18

Planarity Assumption

Thank You!

D SantoshD SantoshIIIT Hyderabad

http://research.iiit.ac.in/~santoshhttp://cvit.iiit.ac.in/projects/roboticVision

IIIT

Hyd

erab

ad

Publications• ‘Visual Servoing in presence of Non Rigid Motion’• Visual Servoing in presence of Non-Rigid Motion ,

– Accepted at International Conference on Pattern Recognition (ICPR) 2006

• ‘Robust Homography based Control for Camera Positioning in• Robust Homography-based Control for Camera Positioning in Piecewise Planar Environments’, – Accepted at Indian Conference on Computer Vision, Graphics and Image

Processing (ICVGIP) 2006Processing (ICVGIP) 2006• ‘Visual Servoing in Non-Rigid Environments: A Space-Time

Approach’– International Conference on Robotics and Automation (ICRA) 2007

• ‘Visual Servoing in Non-Rigid Environments’– In Preparation for IEEE Transactions on Robotics (ITRO) 2007

IIIT

Hyd

erab

ad

Santosh, Jawahar. Visual Servoing in Non-Rigid Environments 20

p ( )