Date post: | 04-Jun-2015 |
Category: |
Technology |
Upload: | culturainnovazione |
View: | 524 times |
Download: | 1 times |
01/07/2010 15:36 1
Pieter Jonker Professor of Vision based Robotics
Dept Bio-Mechanical Engineering, Faculty 3MEDelft University of Technology, Delft
Augmented Reality
Yolande KolsteeLector of Innovative Visualisation Techniques for Art & Design
Royal Academy of Art, The HagueThe [email protected]
01/07/2010 15:36 2
Het volgen van een enkele hypothese (toegepast in AR)Augmented Reality vs Virtual Reality
01/07/2010 15:36 3
Sorts of Augmented Reality
Screen BasedHead-Set Based
01/07/2010 15:36 4
Sorts of Augmented Reality
Video see through AR2 cameras + 2 displays + video in video
Optical see through AR1 camera + 2 half translucent mirrors + 2 displays
01/07/2010 15:36 5
History of our headsets
01/07/2010 15:36 6
The Measurement Problem
How to hold the virtual objects in a perfect position and orientation in the physical world if I move my head?
By searching and tracking (camera + image processing) one or more known landmarks!
01/07/2010 15:36 7
How to anchor onto a marker?
Grab an image Edge detectionSelect closed
contoursSimple corner
detection
Keep contours with 4 corners
Split contour into 4 lines
Sub-pixel fit of the 4 lines
Intersect 4 lines for corners
Calculate pose of marker
Determine the ID
Calculate pose of camera
Grab an image Edge detectionSelect closed
contoursSimple corner
detection
Keep contours with 4 corners
Split contour into 4 lines
Sub-pixel fit of the 4 lines
Intersect 4 lines for corners
Calculate pose of marker
Determine the ID
Calculate pose of camera
1
32
1024
2
64
2048
4
128
4096
8
256
8192
512
16384
161
32
1024
2
64
2048
4
128
4096
8
256
8192
512
16384
16
01/07/2010 15:36 8
Why not tracking at natural features?
This costs too much time fort now, but soon…
01/07/2010 15:36 9
Is only a camera & image processing enough?
No, we must know where the head will be in 30 msecsbecause rendering a virtual image costs time. We have to predict where the head will be bij measuring the motion of the head.
That is why we have to fuse the data of the camera and image processing with the date from a so called inertia tracker to track fast head movements.This is comparable with the yes and the inner ear / vestibular organ of a human.
An inertia tracker has 3 accelerometers to measure linear acceleration in 3 dimensions (x, y, z) and 3 gyroscopes to measure 3 rotational velocities (roll, pitch, yaw).This gives relative displacements at 100Hz that are combined with the 25Hz absolute position and orientation of the camera.
01/07/2010 15:36 10
Fusion of camera and inertia tracker data
100 Hz
60 Hz
Camera measurement
Inertia measurement
Data Fusion
Rendering Imagetime
Image processingImage acquisition 15 Hz
01/07/2010 15:36 11
So how accurate can one place virtual objects in the physical world?
Statical misalignment< 0.6 ° (< 0.03° = eye) Dynamical Misalignment< 2.3° (< 0.5° = eye)
With a 20 x 20 cm marker at 5 m and a 1MPix camera
Own and object motion will camouflage this
01/07/2010 15:36 12
Future elektronics to loose the backpack and headset
01/07/2010 15:36 13
Conclusions so far
• Augmented Reality is not Virtual Reality• Augmented Reality is screen or headset based• Headset based AR is video or optical see-through• The AR problem is dominated by the the allignment
problem of virtual and real world objects• This is solved by inertia and camera tracking• Due to the imperfect technology one can still see a
bit alignment errors, but far better than GPS! • Happily the human visual system compensates a bit• Future hardware will make it as small as iPods.• AR is a new and complex medium to discover!
01/07/2010 15:36 14
The Vision of Yolande Kolstee
Visualization in Art and Design
01/07/2010 15:36 15
Royal Academy of Art (KABK)
The Hague
The Netherlands
www.ARLAB.nl
Undergraduate / Bachelors
� Design: Textile, Fashion, Interior,
Furniture, Graphic, Interactive Media
� Fine Arts & ArtScience
� Photography
Graduate / Masters
� Type and Media
� ArtScience
� Interior Architecture
� Research in the Arts
� Industrial Design
01/07/2010 15:36 16
Delft University of Technology
Delft
The Netherlands
Undergraduate, Graduate & Doctorate
Faculties:
� Aerospace Engineering
� Applied Sciences
� Architecture
� Civil Engineering and Geosciences
� Electrical Engineering, Mathematics & Computer Sc.
� Industrial Design Engineering
� Mechanical, Maritime & Materials Engineering
� Technology, Policy and Management
01/07/2010 15:36 17
The fusion of perception of real and virtual imagesThe AR Lab is for all students, also outside students, who need help in the AR domain.
Institutes of cultural heritage come to us, like Museums:•Van Gogh•Boijmans van Beuningen•Stedelijk Museum•Kröller-Müller•Escher
Manifestations of:Multi-mediaModern-ArtICT
01/07/2010 15:36 18
For example we used AR to show animated chairs and tables in the 2008 exhibition Salone del Mobile in Milan, Italy
01/07/2010 15:36 19
AR was used to demonstrate the construction process
01/07/2010 15:36 20
AR can be used to demonstrate the possibilities of the product
Melanie Luchtenveld
01/07/2010 15:36 21
The virtual objects can be studied from all viewpoints
01/07/2010 15:36 22
Transportation and duplication of furniture is quite easy
01/07/2010 15:36 23
Connecting with the virtual world, experiments with interfaces
Audience likes AR & Interaction �Wii remote controller�RFID�Data glove: 5 flexor strips in fingers�Markers 'stand-alone' or incorporated�Fragments of pottery
01/07/2010 15:36 24
We use RFID technology to let (tagged) physical objects influence virtual objects
01/07/2010 15:36 25
We use a data-glove to manipulate virtual manipulated objects. Here to realize a virtual hand-puppet
01/07/2010 15:36 26
One can interact with virtual objects in several ways
One can us real physical objects to change behavior of virtual puppets e.g.
Bath-sponge � towel and foam
Pot-holder � cook
Fay-wand � blow fish is crowned
Cigarettes � blow fish blows
01/07/2010 15:36 27
Interaction can also be performed by moving the markers
01/07/2010 15:36 28
Interaction with a WII mote
� Escher in the Palace:� Wii remote controller:
throw a visual to thewall
01/07/2010 15:36 29
Headset Design
Students of the Royal Academy designed a new AR headset based on the VR/AR headset of the Dutch SME Cybermind who made a working version of the student’s mock-up
01/07/2010 15:36 30
Design requirements:� More ease to wear� Don't isolate the user� Don't block the reality� Contact with the environment� Unique experience / attractive
01/07/2010 15:36 31
Headset 'George'
� camera at the right side� inertia tracker left� open view below� wide screen� Side view
01/07/2010 15:36 32
Various other designs have been made, but not yet prototyped
Design Niels Mulder
01/07/2010 15:36 33
What to display?
Man made:There is a variety of 3D design programs and skilled and talented people who master this:Fine Art Artists, Animators, 3D Designers, Architects
Computer Generated:� 3D imagery (industrial) machines
especially from the bio-medical research� CT scan generated 3D images, yield an
inexhaustible amount of CT images
Data visualization:� There are various huge databases, e.g. geo
information, annual reports
01/07/2010 15:36 34
Art: Drops of White
Marina de Haas
01/07/2010 15:36 35
Geo Info
Data Visualization of Acid level in the first AR MAP on behalf of GSDI 11 World Conference, RotterdamJune 16-20 2009
Global Spatial Data Infrastructure association
01/07/2010 15:36 36
AR Art: first experiment for general public
Marina de Haas
01/07/2010 15:36 37
Cultural Heritage: Displaying computer generated data:Using bio-medical imaging equipment: CT scans for visualizing sgraffito-pottery:
Pre industrial original (1250-1450)Post industrial means Scanning- Digitalizing
3D printing � physical copy3D Archiving � virtual copy
01/07/2010 15:36 38
01/07/2010 15:36 39
Lessons learned
� Let the public view on a beamer what is seen by the HMD
� The “empty” third dimension is very useful for information display and interaction and detaches the application from gravity. One can have art or any info in the air ( e.g. drops of white)
� Motion of the virtual objects covers up for misalignment and jitter
� Design packages such as Cinema 4D make design with animated figures possible. For real 3D animated films with large plots, we are busy to incorporate game engines like Unity
01/07/2010 15:36 40
� Headset based AR is at its best when a full immersive experience is required and people can walk around or within larger objects; buildings, DNA, airplanes to be refurbished
� For outdoor AR it is necessary that the ambient light intensity and the intensity of the LCD displays on the headset is in balance (automatic sunglasses)
� Augmented Reality can be fruitfully used to attract a broad public to displays of cultural heritage. Its narrative power is huge.
� The collaboration between researcher in the area of image processing with artists, designers and curators appeared to be very fruitful and has lead to many amazing productions and exhibitions
01/07/2010 15:36 41
“It begins with a blessingand it ends with a curse;making life easy,by making it worse.”
‘Why more realities?’
the old man in Milan said:
‘Live is already so complicated’.