ITCamp 2013 - Tim Huckaby - Kinect for Windows - Designing Software for Gesture & Voice Controlled...

itcampro@ itcamp13# Premium conference on Microsoft technologies

Kinect for Windows -Designing Software for

Gesture & Voice Controlled User Interfaces

Tim HuckabyInterKnowlogy Chairman/Founder

Actus Interactive Software Founder & CEO

Microsoft® Regional Director – Southern California

Microsoft® MVP – Kinect for Windows

E-mail: [email protected]@Actus-Software.com

Blog:team.interknowlogy.com/blogs/timhuckaby

FaceBook:www.facebook.com/Tim.Huckaby.Carlsbad

LinkedIn: www.linkedin.com/in/timhuckaby

Twitter: @TimHuckaby


Breakout

SessionHuge thanks to our sponsors!


A little bit about me and why I am here…

A little bit about me and why I am here…

Kinect for Windows - Designing Software for Gesture & Voice Controlled User Interfaces


Breakout

SessionAbout Tim, InterKnowlogy & Actus…

InterKnowlogy (www.InterKnowlogy.com)

Tim Huckaby, Founder/Chairman ([email protected])

◦ 13+ year old services firm – Custom Application Development

◦ focused on building NUI solutions

◦ Web, mobile, and desktop

◦ Windows 7/8, PixelSense, WP7/8, Azure, Kinect, XAML stack

- Largest Client: Microsoft

Actus Interactive Software (www.Actus-Software.com)

Tim Huckaby, Founder/CEO ([email protected])

◦ Software Product Company

◦ Interactive Kiosk and Active Digital Signage

◦ Multi-Touch, Gesture, and Voice Driven Solutions

mailto:[email protected]


Relevant Use Cases (And Use Cases that do not Make Sense)

Objectives And Agenda

Runway to get started

Technology Roadmap

Where We have Been and Where we are Going

Demos, Demos, Demos….

Gesture & Voice Driven User Experiences


NUI: The Natural User Interface

Multi-Touch Capable•Common Gestures of Touch•Physical Object Interaction•Virtual Object Interaction

Gesture Capable•Motion Based Interface•Tracking of full-body movement, facial expression, and voice with precision

Voice Capable•Multi-array Microphone•Uniquely Distinguish Audio & Voices

Nueral Capable•Conscious and non-conscience interfaces•Non Invasive


World's Fastest-Selling Consumer Electronics Device

… a Future Beyond Gaming

Combination of RGB camera, depth sensor and multi-array microphone– RGB camera delivers three basic color components

– Depth sensors “sees” in 3-D

– Microphone locates voices by sound and excludes ambient noise

Consumer Priced: $249.99

Tracks full-body movement, facial expression, and voice with precision

Kinect sensor bar scans and monitors 1 million points on the body

Kinect for Windows SDK– Skeletal Tracking

– Face, Gesture Recognition

– Audio Echo cancellation

– Audio Beam Forming

– Speech Recognition

– Machine Learning


Breakout

SessionInterKnowlogy – Picture Viewer

Tim HuckabyFounder / Chairman, InterKnowlogyFounder / CEO, Actus SoftwareMicrosoft RD & MVP


Breakout

SessionUse Case: Data Visualization & 3D Modeling

Interactive data visualization

3D modeling and design

Hands-off presentations

Kinect Fusion


Breakout

SessionKinect – 3D Anatomical Manipulation



Breakout

Session

A Relatively New Hardware Product A commercial HW SKU for B2B sales; developers can purchase via Amazon and similar channels

A Commitment Kinect for Windows gives a customer:

Fully supported hardware for Windows, plus a new “near mode” to allow innovation at 40cm from the sensor

Ongoing updates based on the software innovation that is part of Microsoft’s deep investment and commitment to NUI, in both speech and human tracking

A Future of InnovationKinect for Windows opens up a new wave of innovation for Human Computer Interaction on Windows platform

Puts Natural User Interface (NUI) into the public eye in a broad variety of venues: from car dealerships to malls to boardrooms to operating rooms

What is Kinect For Windows?


Breakout

Session

Use Cases: Exercise, Physical Therapy, Education, Training

Exercise and physical therapy

Senior wellness – exercise and monitoring

Interactive science experiments

Language/speech learning

Home Healthcare / Tele-Medicine


Breakout

Session

• Skeletal Tracking

• Near Mode

• Depth Mode

3 Modes of Operation


Breakout

SessionK4W Skeletal Tracking


Breakout

Session

The Kinect for Windows InterKnowlogy - Physical Therapy



Breakout

Session

The Good News:• The Kinect SDK for Windows

The Not So Good News:• Windows Only Device; You are not legally allowed to build

Windows apps for your Xbox Kinect;

• you cannot build apps for 18 million Xbox Kinect owners with it.

Kinect for Windows


How to:

United States - $249.99 - SKU: L6M-00001

Amazon Visit the Store

GameStop Visit the Store

Microsoft Store Visit the Store

Newegg Visit the Store

http://www.microsoft.com/en-us/kinectforwindows/purchase/

http://www.amazon.com/Microsoft-L6M-00001-Kinect-Sensor-Windows/dp/B006UIS53K/ref=sr_1_1?ie=UTF8&qid=1327348853&sr=8-1

http://www.gamestop.com/pc/pc-hardware/kinect-for-windows/99075

http://www.microsoftstore.com/store/msstore/en_US/pd/productID.244210600

http://www.newegg.com/Product/Product.aspx?Item=N82E16874103270&Tpk=kinect for windows


Breakout

Session

Raw sensor streamsAccess to raw data streams from the depth sensor, color camera sensor, and four-element microphone array enables developers to build upon the low-level streams that are generated by the Kinect sensor.

Skeletal trackingThe capability to track the skeleton image of one or two people moving within the Kinect field of view make it easy to create gesture-driven applications.

Advanced audio capabilitiesAudio processing capabilities include sophisticated acoustic noise suppression and echo cancellation, beam formation to identify the current sound source, and integration with the Windows speech recognition API.

Sample code and documentationThe SDK includes more than 100 pages of technical documentation. In addition to built-in help files, the documentation includes detailed walkthroughs for most samples provided with the SDK.

Easy and familiarThe SDK installs quickly, requires no complex configuration, and the complete installer size is less than 100 MB. Developers can get up and running in just a few minutes with a standard standalone Kinect sensor unit (widely available at retail outlets). Windows 7, Windows Embedded Standard, Windows 8 Desktop Apps C++, C#, or Visual Basic in Microsoft Visual Studio 2010.

Free!Free SDK download, free run time license!http://www.microsoft.com/en-us/kinectforwindows/develop/

What’s in the Kinect for Windows SDK?

http://www.microsoft.com/en-us/kinectforwindows/develop/


Breakout

SessionKinect for Windows SDK



Breakout

Session

Launch Kinect for XBox

K4W Team

Formed

K4W Non-commercial SDK beta

K4W

commercial

SDK TAP Initiated

New K4W

Hardware announced

K4W commercial

Release

Timeline

Nov, 2010 April, 2011 June, 2011 Sept, 2011 Nov, 2011 Feb 1, 2012

K4W v1.5 K4W v1.6 K4W v1.7 K4W Dev Center

May 21, 2012 Oct, 2012 March, 2013 May, 2013


Breakout

SessionUse Case: Interactive Media

Interact with customers in-store

Virtual fitting room – in-store or shop-at-home

Vending machines

Interactive window exhibits

Augmented Reality web chat


Breakout

Session

Perfect for Kinect: Not designed for Kinect

Windows 8 / Metro / Modern UI


Breakout

SessionActus Interactive Kiosk



Breakout

SessionUse Case: Hands-Free Control

Doctors in surgery operating medical imaging devices through gesture and voice

Airplane mechanics/engineering workers viewing instructions

Cooking and other household scenarios


Breakout

SessionInterKnowlogy: Touch-less Operating Room



Breakout

Session

Leap Motion– $70– https://leapmotion.com– In developer beta

SoftKinetic– http://www.softkinetic.com– $249– Distance and Near Mode– Multiple Products– In the space for over a decade

Panasonic– Works outside in broad daylight– $2,600

Kinect for Windows Competitors

https://leapmotion.com/

http://www.softkinetic.com/


Breakout

SessionGesturePak: Carl Franklin’s Gesture Recorder



Breakout

SessionCarl Franklin’s Gesture Recorder

GesturePak is a speech-driven gesture recording WPF app and a matcher class for .NET developers that uses the Kinect For Windows device and SDK.


SummaryKinect for Windows is not great in the majority of Use Cases; where it is good:• Hands-Free Control• Exercise, Physical Therapy, Education, Training• Data Visualization, 3D Modeling• Interactive Media, Retail, Digital Signage

The hardware, tools, plumbing and platform for Kinect for Windows Applications are powerful and available now.…and will just continue to get better, cheaper, powerful and more available

The concepts of Gesture and Voice Recognition change the way we design and build software in some Use Cases.…if the goal is better user experiences


More info on InterKnowlogy:

www.InterKnowlogy.com www.vimeo.com/InterKnowlogy

More info on Actus Software:

◦www.Actus-Software.comAbout Tim Huckaby…

◦Founder/Chairman, InterKnowlogy

◦Founder/Chairman, Actus Software

◦Microsoft® Regional Director – Southern California

◦Microsoft® .NET Partner Advisory Council Founder / Member

◦Microsoft® MVP – Kinect for Windows

◦INETA Speaker – International .NET Users Group Association

◦Board Member: Association for Competitive Technology

◦Board Member: Microsoft Healthcare Partner Advisory Board

◦Author / Speaker

Contact me:E-mail:

[email protected]@Actus-Software.com

Phone: • Office: 760-444-8640• Mobile:: 619 990 9200

Blog:http://team.interknowlogy.com/blogs/timhuckaby

FaceBook:http://www.facebook.com/Tim.Huckaby.Carlsbad

LinkedIn:http://www.linkedin.com/in/timhuckaby

Twitter: @TimHuckaby

Tim Huckaby




Q & A

Date post:	13-May-2015
Category:	Technology
Upload:	itcamp
View:	418 times
Download:	2 times