Post on 25-Dec-2015
transcript
Augmented Reality with .NET
casey chesnutbrains-N-brawn.com
Dallas C# SIGJanuary 2008
brains-N-brawn.com• Pervasive Computing
– Tablet PC (MVP 03)
– Compact Framework (MVP 04)
– Advanced Web Services (MVP 05)
– Media Center (MVP 06)
– Speech / Office Communication Server (MVP 07)
– Location Based Services– Artificial Intelligence– 3D– Image Processing– Augmented Reality
Source
• http://www.brains-N-brawn.com/wpfAugReal/• http://www.brains-N-brawn.com/wpf3dPart• Environment : 32-bit Vista, VS 2005, .NET 3.0,
Logitech QuickCam Pro Web Cam for Notebooks
Audience
• DirectShow?
• 3D?
• WPF?
• Augmented Reality?
Outline
• Intro to Augmented Reality
• DirectShow
• Augmented Reality (AR) Libraries
• WPF 3D
Introduction
• Augmented Reality (AR) - is the combination of real world data with virtual data.
Audience
• What are some current examples?
Current Examples
• Fighter Pilot HUDs• QuickCam video conferencing• PS3 Eye of Judgment• Sports : Football, Hockey, Nascar• Aircraft maintenance• ARQuake• Car GPS HUD• …• Star Wars chess!
Videos
• QuickCam
• Eye of Judgment
Steps
• Acquire real-world data (camera, HMD, mobile devices)
• Recognize real-world data (computer vision, face detection, fudicial markers, GPS)
• Determine relative position of real-world marker to input source.
• Display composite of real-world data and virtual object (monitor, HMD, mobile device, projector).
Acquire real-world data
• OpenCV / SharperCV
• DirectShow / DirectShow .NET
• Windows Media Foundation
• …
DirectShow
• Used by Windows Media Player, Media Center Edition, Windows Media Encoder, Movie Maker, …
• User for audio/video playback, DVD, TV tuners, web cams, media capture, media transcoding, …
DirectShow
• COM
• GraphEdt tool
• Graph, Filter, Pin, Configuration
• Filters (Source, Transform, Sink)
• DirectShow .NET (C# wrapper)– http://directshownet.sourceforge.net/
• Learning the filters is key
• Not WPF 3D friendly
DirectShow .NET
• C# wrapper
• Wraps a majority of DirectShow
• Many samples available, along with GraphEdt.exe, and MSDN documentation
• Also has beta wrappers for Windows Media Lib and Windows Media Foundation
Demo
• GraphEdt
• /dshowPlay
• /wpfArBasic - capture
Recognize real-world data
• Computer Vision
• Face tracking
• AR Libraries (fudicial markers)
• GPS
• …
AR Libraries
• ArToolKit
• ArTag
• ArToolKitPlus– http://studierstube.icg.tu-graz.ac.at/
handheld_ar/artoolkitplus.php
• Studierstube Tracker
• Goblin XNA (coming)
ArToolKitPlus
• Optimized for mobile devices
• C++ Lib
• GPL, no longer under development– Studierstube Tracker is the latest
• Not well documented, but ArToolKit is
• Single, Multi-config, Multi-distinct tracking
• Markers (ID, BCH, Patterns)
Videos
• ARTag Magic Lens
• Studierstube Mobile
Demo
• /wpfArBasic – AR tracking
• /wpfArTest
Composite real and virtual data
• GDI
• OpenGL / Tao
• Direct3D
• XNA
• WPF 3D
• …
WPF 3D
• Viewport3D
• MatrixCamera
• Light
• ModelVisual3D / Model3DGroup
• Material
• Transform3D
• HitTest
WPF 3D
• 3D Programming for Windows (Petzold)
• Windows SDK Samples
Demo
• Simple WPF 3D
• /wpfArBasic – 3D model
• /wpfArVideo
Future
• Augmented Reality @ Home (book)– Mostly OpenGL and C++, but at least one C#
and Tao sample. Based on ArTag library.
• Goblin XNA (AR library)– For PC use, not on 360 until XNA is extended
• …
Questions