04/19/23EEC492/693/793 - iPhone Application
Development 1
EEC-492/592EEC-492/592Kinect Application Kinect Application
DevelopmentDevelopmentLecture 1Lecture 1
Wenbing ZhaoWenbing Zhao
[email protected]@ieee.org
04/19/23EEC492/693/793 - iPhone Application
Development 2
IntroductionIntroduction Instructors
Wenbing Zhao, [email protected] Office Hours
Mondays 6-8pm Wednesdays noon-2pm
Course Web site http://academic.csuohio.edu/zhao_w/teaching/EEC492-S14/e
ec492.htm
04/19/23EEC492/693/793 - iPhone Application
Development 3
Quick Survey
Have you programmed in C#? Have you programmed in C++? Have you programmed in Java? Have you ever played Kinect games? Have you installed Visual Studio and done
Windows application development?
04/19/23EEC492/693/793 - iPhone Application
Development 4
Requirements
Prerequisite: Moderate programming background Basic knowledge of data structure and algorithms Basic knowledge of operating systems
04/19/23EEC492/693/793 - iPhone Application
Development 5
Textbooks and References
Main textbook Kinect for Windows SDK Programming Guide, by
Abhijit Jana, PACKT publishing, 2012. References:
Kinect Hacks, by Jared St. Jean, O’Reilly, 2013 Hacking the Kinect, by Jeff Kramer et al., Apress,
2012.Reference materials C# in a Nutshell, By Peter Drayton, Ben Albahari,
Ted Neward, O'Reilly, 1st Edition March 2002
04/19/23EEC492/693/793 - iPhone Application
Development 6
Required Hardware & Software Hardware
Windows PC or laptop Kinect sensor
Software Visual Studio 2010 Kinect SDK Other related libraries
04/19/23EEC492/693/793 - iPhone Application
Development 7
Kinect’s Capability Capturing real-time video using the color sensor Tracking a human body and then responding to its
movements and gestures as a natural user interface Generating depth map of the objects tracked Recognizing a human voice and developing hands-
free applications that can be controlled by voice
04/19/23EEC492/693/793 - iPhone Application
Development 8
Real-World Applications of Kinect Healthcare: exercise measurement, monitoring patients, their
body movements, and so on Robotics: as a navigation system for robots either by tracking
human gestures, voice commands, or by human body movements
Education: help them to learn subjects either by their gesture and voice commands
Security system: track human body movement or face and send the notifications
Virtual Reality Trainer: measuring the movements of human body joints,
providing live feedback to users if the joints are moving in an appropriate manner by comparing the movements with previously stored data
Military: build intelligent drones to spy on enemy lines
04/19/23EEC492/693/793 - iPhone Application
Development 9
Healthcare with Kinect Reflexion Health
Advancing physical therapy http://blogs.msdn.com/b/kinectforwindows/archive/2013/05/
09/reflexion-health-advancing-physical-therapy-with-kinect-for-windows.aspx
04/19/23EEC492/693/793 - iPhone Application
Development 10
Healthcare with Kinect ReMotion360: uses Kinect to measure and provide
on screen feedback for range of motion exercises either in a clinic or at home http://www.infostrat.com/solutions/Remotion360 https://www.youtube.com/watch?feature=
player_embedded&v=-jhpYslVygs
04/19/23EEC492/693/793 - iPhone Application
Development 11
Improving Safety at Workplace Siemens uses Kinect for Windows to improve
nuclear plant employee safety virtual 3-D human avatars integrated to model motion-
controlled actions input with a Kinect http://blogs.msdn.com/b/kinectforwindows/archive/2013/04/
19/siemens-uses-kinect-for-windows-to-improve-nuclear-plant-employee-safety.aspx
04/19/23EEC492/693/793 - iPhone Application
Development 12
Enabling Retailers to Drive Business Benefits:
Capture a customer's attention Educate customers about products Move a customer through the decision-making cycle to
close a sale Ski Shop scenario:
https://www.youtube.com/watch?feature=player_embedded&v=ORWYr1gx5eU
Toy store scenario: https://www.youtube.com/watch?
feature=player_embedded&v=wZfcDOsBs2E Retail clothing scenario
https://www.youtube.com/watch?feature=player_embedded&v=Mr71jrkzWq8
04/19/23EEC492/693/793 - iPhone Application
Development 13
Styku: Apparel Shopping Help find clothes that fit Virtually try on clothes before they buy To be offered by major retailers, such as Brooks
Brothers, IM-Label, and others
04/19/23EEC492/693/793 - iPhone Application
Development 14
Shoppers who approach the Kinect-based showroom are guided through an array of natural movements to activate hotspots on the Pathfinder model, allowing them to inspect the car inside and out https://www.youtube.com/watch?
feature=player_embedded&v=rQVycUmeqB4
Nissan Pathfinder Virtual Showroom
04/19/23EEC492/693/793 - iPhone Application
Development 15
Education Tool nsquared sky spelling
makes it fun for children to learn to spell in an interactive and collaborative environment
capture letters that complete the spelling of various words
04/19/23EEC492/693/793 - iPhone Application
Development 16
Hospitality Tool Nsquared Hotel Kiosk
augments the concierge function in a hotel by providing guidance to hotel guests through an intuitive, interactive experience
use a “grip” gesture to drag content across the screen and a “push” gesture to select content
04/19/23EEC492/693/793 - iPhone Application
Development 17
Ubi Interactive: Turn any Surface into a Touch Screen Instruct classes in an interactive lecture hall Access product information on a store’s window front Recipes can be projected onto kitchen countertops without
having to worry about getting anything dirty Use wall to play interactive games like Angry Birds Video: https://www.youtube.com/watch?
feature=player_embedded&v=Iu2XH5p_hMM
3D Scanning Kinect fusion: perform 3D scanning with a single
Kinect sensor! Nice match with 3D printers
04/19/23EEC492/693/793 - iPhone Application
Development 18
04/19/23EEC492/693/793 - iPhone Application
Development 19
New Development on Depth Cameras: PC Makers Bet on Gaze, Gesture, Voice, and Touch A combined 2-D and 3-D camera from Intel will
be built into laptops and tablets from a range of manufacturers, Intel announced at the 2014 International Consumer Electronics Show (CES)
04/19/23EEC492/693/793 - iPhone Application
Development 20
New Development on Depth Cameras: PC Makers Bet on Gaze, Gesture, Voice, and Touch Toshiba also demonstrated a dual-camera module for
smartphones, tablets, and laptops at 2014 CES Gesture recognition, refocus images, etc.
New Development on Depth Cameras SoftKinetic demonstrate a depth camera for 3D scanning
Need to move the camera around the subject
Apple will also come to the depth camera field Acquired Primesense, the company invented the technology
behind Kinect Filed a patent in 3D camera for iOS devices
04/19/23EEC492/693/793 - iPhone Application
Development 21
04/19/23EEC492/693/793 - iPhone Application
Development 22
Main Topics Understand the Kinect sensor and how they work Learn how to capture data streams from the Kinect
sensor Learn the uses of color, depth and IR data and how to
leverage them in your applications Use skeleton-tracking for interactive applications Learn basic gesture recognition methods and develop
gesture-enabled applications Build speech-enabled applications with Speech API, and
text to speech as well Learn 3D game development with Unity3D for Kinect
(virtual reality!) Applied computer vision with OpenCV and Kinect
04/19/23EEC492/693/793 - iPhone Application
Development 23
Assignments and Grading Assignments
In-class exercises Term project
Reading several research papers on the application of Kinect Develop a working prototype program using Kinect Project presentations and demonstration
Grading policy Class participation: 60% (any missed session will lead to
credit reduction) Project: 40% No exams
04/19/23EEC492/693/793 - iPhone Application
Development 24
Demos, Software Installation and
Q/A