1 Horst Bischof Redundancy for aerial computer vision
Exploiting redundancy for reliable aerial computer vision
Horst Bischof ���������� ����������������
2 Horst Bischof Redundancy for aerial computer vision
Digital Image Increase
3 Horst Bischof Redundancy for aerial computer vision
Images Worldwide
� �������������� ��� ������������� ��������������� ������������� ��������������� �� !�� ��������������
�"��#�����$!��������%�����
4 Horst Bischof Redundancy for aerial computer vision
Terrestrial Image Acquisition
� �������& ��� �%�'�� (���������!�����)���*� ��� �+� �,����!� ���� �����*�� ��-����� �����
5 Horst Bischof Redundancy for aerial computer vision
Aerial Photogrammetry
� -����������.�� �)#������ � �,� �������+� �&/� �01������� �� ��-�!����2�#����3���)���!��&,+�� (����!�����)(���*� ��� �+��
6 Horst Bischof Redundancy for aerial computer vision
New Sensor Platforms
� -�� ������� �� 4��3����)�� ���� ����������5����5��� �
��� ������5� ��*&�'����� ����� ��+�
6 Towards Fully Automatic Photogrammetric Reconstruction Using Digital Images Taken From UAVs./7�$�������"�%78�1�!���"�#7�8������ �"�97�:�����"���5�47�;�����7�����<�����$&*=&�����7>��
7 Horst Bischof Redundancy for aerial computer vision
Airborne vs. MAVs
#������ ��� ��-�!,�
������3�?����
#/%�*�� �3�/@��@����3�������
A���7���� A����
�&,B�(C�! �3���9��B�(2��!�
�&,B�(�7��! �3���9��B�(��!�
8 Horst Bischof Redundancy for aerial computer vision
Applications
9 Horst Bischof Redundancy for aerial computer vision
Application: Virtual Habitat
[Leberl et al. IEEE Computer 2010]
10Horst Bischof Redundancy for aerial computer vision
Application: Architecture and Cultural Heritage
[Zebedin 2010], [Irschara 2010]
11Horst Bischof Redundancy for aerial computer vision
Application: Construction Site Monitoring
12Horst Bischof Redundancy for aerial computer vision
Application: Mining
13Horst Bischof Redundancy for aerial computer vision
Outline
D 3D Reconstruction �E & �1� 1������!�#� ���E ,�����& �����E =�51�5�����
�
D Semantic Classification �E *3���������������� ���E =��1����� ���E 41����
�
D Discussion & Outlook
14Horst Bischof Redundancy for aerial computer vision
3D Reconstruction
15Horst Bischof Redundancy for aerial computer vision
Structure from Motion
6,7�F� ��"�G/1 �!� ��5����������� �1� ������!�1������� �5�5�����H1�����I7�*�7,7"�����>�6#7�*���������� ���7"�G%�1���!�5�����' �������5���5���!���I7�$J-%"�����@>�6F7�&������� ���7"�G*�� ����1��!B�03�������*�� ��-����� �������,I7�&$��=/*9"�����>�6&7�/���'���� ���7"�G:1�5���=�!������,��I7�-%*="����2>�6J7K#7�4���!�� ���7"�G:1�5���=�!�������-��15�����,��I7�0--%"�����>�
5�����H1����� �1��������5�!�����
�,�!�5�����!�������� � ����L��,�� �1� 1���
$F*��� M��*���
16Horst Bischof Redundancy for aerial computer vision
Structure from Motion
-�!�����M��� � ����
�,�&������& �1� 1���
$!�����-��������5������
17Horst Bischof Redundancy for aerial computer vision
Structure from Motion
X1
X4 X
3 X2
X6
X7
X5
-�!������R2, t2, [K2]
minimize g )R"t"[K]"X+�
-�!������R1, t1, [K1]
-�!������R3, t3, [K3]
6F7�&������-%*=�N�2� 1 ����>�
�7�7��������!��������7�����,���� ���
����7�����1����'���
18Horst Bischof Redundancy for aerial computer vision
�*�� �-*��&���51����K��3�
Structure from Motion
Feature Extraction
Coarse Matching
Detailed Matching
Geometric Verification
Images Pose Prior
Geometric Estimation
Epipolar Graph
Image Overlap
Local Descriptors
Camera poses 3D points Matches
Calibration Pose Prior
-�!�1 ������5�,����/��� �� 1���)-�,/+�
19Horst Bischof Redundancy for aerial computer vision
Aerial Photogrammetry
� �����������!������ �����(�����!O�� C�P��������� �@"@���3�2"@����3���� (����!��&,�
20Horst Bischof Redundancy for aerial computer vision
Global Depth Map Optimization
5� �� ��!����1��� �� ��!�
���
��� ��������������
���
)),(()(
)),(( ���� 77�!� �������� ��)F--+�
�������
)( 77��� ���%��� ���)�%+����1����� ��B�
� ������������� ����!�� ������ �5���� �1 �����������
6�7�*����� ���7"�G/�����3����!1�� ��������� �1�1��!1� K������������!�I7�0--%"����C>�
-�� �1�1��0������#�!�� ���/�������
21Horst Bischof Redundancy for aerial computer vision
KKEY view
22Horst Bischof Redundancy for aerial computer vision
DEPTH map
min
5� �� ��!����1��� �� ��!�
���
��� ��������������
���
)),(()(
23Horst Bischof Redundancy for aerial computer vision
DEPTH map
min
5� �� ��!����1��� �� ��!�
���
��� ��������������
���
)),(()(
24Horst Bischof Redundancy for aerial computer vision
25Horst Bischof Redundancy for aerial computer vision
Primal-Dual Optimization D �������1� �������3��� !�� ���������!����������5���
����� K��5�����!��K51�������� �!�6-��!�����"�*��������>�D 4�� 1���������������� !����������������������3�
������!��' �����'��� �1� 1���D -������������ �������������5����!����������������
���5'�����1�������*���
26Horst Bischof Redundancy for aerial computer vision
Distributed Visual SLAM
Set of updateable, geo-referenced virtual cameras
PPose
Low framerate, for map extension and relocalization
Full framerate, “cheap” features, for visual servoing
Very little data needed, provide only data necessary for the
current environment Maintain global map,
use “expensive” features!
27Horst Bischof Redundancy for aerial computer vision
Dense Reconstruction On-the-Fly
28Horst Bischof Redundancy for aerial computer vision
Dense Reconstruction On-the-Fly
29Horst Bischof Redundancy for aerial computer vision
Semantic Segmentation
30Horst Bischof Redundancy for aerial computer vision
Motivation – Semantic Interpretation
Water ?
Building ?
Tree ?Street ?
10 cm/pixel
31Horst Bischof Redundancy for aerial computer vision
Motivation
�… a lot of overlapping images
3D information
32Horst Bischof Redundancy for aerial computer vision
Fusion - Model
D �%K;��5�������!�5���6F�������@>�����
- Robust against outliers - Preserves sharp edges
33Horst Bischof Redundancy for aerial computer vision
Fusion - Model
D 03 ������ ��!1� ���������� ����6Q����?>��
D ��
Some observations
34Horst Bischof Redundancy for aerial computer vision
Fusion – Semantic Interpretation
D =����!�� ����� ��������������!�� ��D -�� �1�1�����!1�� ������ ���*� ��!�5���6*����2>�
� aggregated
refined
35Horst Bischof Redundancy for aerial computer vision
Fusion - Color and Height
Color: Wavelet-Fusion Height: TV-Fusion
36Horst Bischof Redundancy for aerial computer vision
��
Some observations in ortho-view
37Horst Bischof Redundancy for aerial computer vision
Evaluation – Redundant Interpretation
Remember evaluation on single images: Graz 89.5%, Dallas 92.5%
38Horst Bischof Redundancy for aerial computer vision
Large-Scale Results
Building
Green Area
Waterbody
Tree
Streetlayer
Dallas, 4 tiles, each 240 x 240 m, 15 cm
39Horst Bischof Redundancy for aerial computer vision
Building
Green Area
Waterbody
Tree
Streetlayer
SF 10 x 3 tiles 2500 x 700 m 15 cm
40Horst Bischof Redundancy for aerial computer vision
Large-scale Results
Building
Green Area
Waterbody
Tree
Streetlayer
Graz, 7 km2, 155 images, 20 x 10 tiles, 8 cm
41Horst Bischof Redundancy for aerial computer vision
Conclusions & Future Work D /���������5��,������� �1� ������� ������5�!����
� ���� ��������� ����D /������������5��,�����!� ��������
�D =�51�5����������B�8��� ���1 �!� �� ���
�D =��1� ���!�1 � ��������semantic 3D model
D 41 1��B��������#/%��E 4��3����E -�����
�
42Horst Bischof Redundancy for aerial computer vision
Videos/Code/Papers see
'''7��1@���7���������7��7 1����7� �
43Horst Bischof Redundancy for aerial computer vision
Acknowledgments 41�5������5�5���B��D 4R4�1�5���*��.�� �9�����M�5���%��� �����#� ��5������-�!�1 ���
%����D 4$�K$��*�����!��1�5�5����:#%$��1�5���*��.�� �%#�*�"�*0�/&�&"�
-MF&�=�-�"�9M;$&�$-�D F%5���
���