Exact and Distributed Algorithms for
Collaborative Camera Control
Dezhen Song*
A. Frank van der Stappen†
Ken Goldberg*
* UC Berkeley, USA
† Utrecht University, Netherlands
Dezhen Song
Webcams
n users 1 pan, tilt, zoom robotic camera
“ShareCam”
Example input: 7 requested frames:
One Optimal Frame
ShareCam Problem: Given n requests, find optimal frame
Taxonomy (Tanie, Matsuhira, Chong 00)
Multiple Operator, Single Robot (MOSR):
Single Operator, Single Robot (SOSR):
Single Operator, Multiple Robot (MOSR):
Related Work• MOSR systems
– Cinematrix (91)– Cannon, McDonald, et al. (97) – Goldberg, Chen, et al. (00, 01) – Goldberg, song, et al. (02)
• Internet robots– Tanie, K., Chong, N. Et al(01)– Jia, S. And K. Takase (01)– Hu, H., Yu, L., Tsui, P., Zhou, Q (01)– Safaric, R. Et al. (01)– Goldberg and Siegwart (02)
Related Work• Facilities Location Problems
– Megiddo and Supowit [84]– Eppstein [97]– Halperin et al. [02]
• Rectangle Fitting – Grossi and Italiano [99,00]– Agarwal and Erickson [99]– Mount et al [96]– Kapelio et al [95]
Related Work• Similarity Measures
– Kavraki [98]– Broder et al [98, 00]– Veltkamp and Hagedoorn [00]
• Distributed robot algorithms – Sagawa et al [01], Safaric[01]– Parker[02], Bulter et al. [01]– Mumolo et al [00], Hayes et al [01]– Agassounon et al [01], Chen [99]
Problem Definition
Requested frames: i=[xi, yi, zi], i=1,…,n
Problem Definition• Assumptions
– Camera has fixed aspect ratio: 4 x 3– Candidate frame = [x, y, z] t
– (x, y) R2 (continuous set)– z Z (discrete set)
(x, y)3z
4z
Problem Definition• “Satisfaction” for user i: 0 Si 1
Si = 0 Si = 1
= i = i
•Symmetric Difference
•Intersection-Over-Union
SDArea
AreaIOU
i
i
1)(
)(
)(
)()(
i
ii
Area
AreaAreaSD
Satisfaction Metrics
Nonlinear functions of (x,y)
• Intersection over Maximum:
),(
)(
),max(
)1,)/min(()/(),(
i
i
i
i
biiii
Max
Area
aa
p
zzaps
Requested frame i , Area= ai
Candidate frame
Area = a
Satisfaction Metrics
pi
Intersection over Maximum: si( ,i)
si = 0.20 0.21 0.53
Requested frame i
Candidate frame
),(),( yxpyxs iii
),( yxpi
Requested frame i Candidate
frame (x,y)
)1,)/min(()/(),( biiii zzaps
(for fixed z)
Satisfaction Function
– si(x,y) is a plateau •One top plane•Four side planes•Quadratic surfaces at corners•Critical boundaries: 4 horizontal, 4
vertical
Objective Function• Global Satisfaction:
n
iii
n
i
biii
yxpyxS
zzapS
1
1
),(),(
)1,)/min(()/()(
for fixed z
ShareCam problem: Find * = arg max S()
S(x,y) is non-differentiable, non-convex, but
piecewise linear along axis-parallel lines.
Properties of Global Satisfaction
ShareCam Algorithms
Bruteforce Algorithm– Compute S at each pixel (x,y)– O(whmn):
•w, h: width and height of panoramic image
•m: number of zoom levels•n: # users
Approximation Algorithm
spacing zoom :
spacing lattice :
zd
dx
y
d
Compute S(x,y) at lattice of sample points:
Approximation Algorithm
– Run Time: – O(w h m n / d2)
* : Optimal frame
: Optimal at lattice ~
: Smallest frame at lattice that encloses *
)ˆ()~
()( * sss
)(
)ˆ(
)(
)~
(1
**
s
s
s
s
ddz
z
z
min
min...
Exact Algorithm
• Define “Virtual corners” – Consider a pair of requested frames
and – Their critical boundariesy
x
Exact Algorithm• Virtual corner: Intersection between
boundaries– Self intersection:– Frame intersection:
y
x
Exact Algorithm
• Claim: An optimal point occurs at a virtual corner. Proof:– Along vertical boundary, S(y) is a 1D
piecewise linear function: extrema must occur at x boundaries
Exact Algorithm
Exact Algorithm:Check all virtual corners
(mn2) virtual corners(n) time to evaluate S for each
(mn3) total runtime
Improved Exact Algorithm• Sweep horizontally: solve at each
vertical – Sort critical points along y axis: O(n
log n)– 1D problem at each vertical boundary
O(nm) – O(n) 1D problems– O(mn2) total runtime
O(n) 1D problems
Distributed Algorithm
More users More computers available
Distributed Algorithm• At the Server
– Sort horiz. boundaries– O(n log n)
• At the Client– Solve 1D problem
for own vertical boundaries.
– O(nm)
• O(n(m+ log n)) Total Four 1D problems
Examples
Examples
• New Approx Algorithms:– With Har-Peled, Koltun– Stair-like
approximation– Dynamic segment tree– O(n log n)
• Weighted Requests
Current Work
Future Work
• Continuous zoom (m=)• Multiple outputs:
– p cameras – p views from one camera
• “Temporal” version: fairness– Integrate si over time: minimize accumulated
dissatisfaction for any user
• Network / Client Variability: load balancing
• Obstacle Avoidance
The Tele-Actor
Operators
Server
Summary• Satisfaction Metric: • Intersection over Maximum• ShareCam Problem : find * = arg max
S()• Critical Points at Virtual Corners• Exact Algorithms: • Distributed Algorithm:• tele-actor.net/sharecam
O(mn2)
O(mn)
Summary
• A collaborative camera control system
• Satisfaction metric• Virtual corner based
algorithms• Distributed algorithm• www.tele-actor.net/
sharecam
Introduction
• Regular web-cam • Collaborative camera control
Queue
Internet Interface:
Results & Discussion
• Speed of naive (B) and fast (V): – AMD K7 950Mhz– 1.2 GB memory– JDK 1.3.1– For a fixed z