Date post: | 06-Apr-2018 |
Category: |
Documents |
Upload: | zhiyong-tsukuyomi |
View: | 226 times |
Download: | 1 times |
of 26
8/3/2019 Kinect Report
1/26
Table Of Contents
1.Brief overview of Kinect
2.History of kinect
3.Why develop kinect?
4.Advantages/problems with kinect5.How kinect works?
6.Kinect for windows
7.How to set up kinect with xbox 360 console
8.Technology in kinect
9.Projects on kinect
10. 1 of the most popular games involving
kinect (star wars)
8/3/2019 Kinect Report
2/26
KinectBrief overview of kinect
kinect is basically a motion sensing input device developed by Microsoft for the Xbox 360
video game console (and now for PC). Based around a webcam-style add-on peripheral for
the Xbox 360 console, players can navigate around and interact with the xbox 360 console
via hand/body gestures and spoken command, completely eliminating the need of a game
controller.
Kinect was launched in
North America on November 4, 2010
Europe on November 10, 2010
Australia, New Zealand and Singapore on November 18, 2010
Japan on November 20, 2010.
Kinect achieved the Guinness World Record of being the "fastest selling consumer
electronics device" due to 10 million units of the Kinect sensor have been shipped as of
March 9, 2011.
(Note : kinect was first known as Project Natal)
8/3/2019 Kinect Report
3/26
History of kinect
June 6, 2009Microsoft announces Project Natal at the annual E3 press conference. They
demonstrate the new motion sensing technology they have developed.
September 24, 2009 Microsoft provides more information about the games for Natal andwho will be making them.
June 13, 2010 Microsoft announces that Project Natal will now go by Kinect.
July 20, 2010 Microsoft gives Kinect a price and release date of November 4, 2010 with a
price of $149.99.
November 4, 2010 Microsoft launches Kinect in North America.
November 15, 2010 1 million units of Kinect have been sold less than two weeks after its
North American launch.
November 18, 2010 Kinect is launched in Australia.
November 20, 2010 Kinect is launched in Japan.
November 29, 2010 Microsoft has sold 2.5 million Kinect units in the 25 days since it was
released.
January 5, 2011 8 million Kinect units have been sold.
February 21, 2011 Microsoft announces a plan to release new software for Kinect on
Windows machines in the spring.
March 3, 2011 10 million Kinect units have been sold.
Why develop kinect?
A few important points :
Microsoft wanted people to start using their full bodies, rather than just their hands,while gaming
good way for people to interact without controllers, broadening the typical gamer base
encourages people to be active
released to compete with the Nintendo Wii and the Playstation eye and move.
8/3/2019 Kinect Report
4/26
8/3/2019 Kinect Report
5/26
in such a case , users must adjust the lighting condition as the RGB camera
embedded is unable to function in darkness comparable to dark skinned
peoples
How kinect works?
Motion sensor
o Invisible light source illuminates subject
o Sensor chip measures distance and light travel
o Unique embedded imaging software uses depth map to perceive & identify objects
o End user device reacts appropriately
MiddleWare
kinect firmware
can recognize people
distinguishing human body parts, joints, movements and faces
the camera knows who you are, once u step in
Xboxs processor
o translating movement captured by kinect camera to meaningful on screen events
http://www.wired.com/images_blogs/gadgetlab/2010/11/Canesta-howitworks1.jpg8/3/2019 Kinect Report
6/26
o context-specific
o easier time recognizing some gestures and postures than others
Skeletal tracking
o creates a digital skeleton of you based on depth data.
o If you move left or right or jump around, the sensor will capture it and put you in the
game.
Facial recognition
o Players have to create a kinect ID (profile)
o This ensure that the kinect will recognize the player in future
Kinect has 2 camera :
A traditional color video camera (light dependent)
o Take pictures and enable video conference
Infrared light sensor (not light dependent)
Digital Skeleton
8/3/2019 Kinect Report
7/26
o Measure depth position and motion
For facial recognition to work, it uses both cameras
Voice recognition
o Have 4 mics within the sensor to recognize and separate your voice from other
noises
o Allow you to control movies and etc with your voice
(e.g. when playing a movie, u can say play kinect or when pausing a movie, pause
kinect)
Kinect for Windows SDK
Kinect is also window compatible
Need to download some additional drivers specially for kinect (OpenNI, Sensor
kinect drivers, NITE middleware) as the official driver havent been released yet
Able to build kinect applications with C++, C#, or Visual Basic by using Microsoft
Visual Studio 2010
Have the following features
1. Raw sensor streams: Access to low-level streams from the depth sensor, color
camera sensor, and four-element microphone array.
2. Skeletal tracking: The capability to track the skeleton image of one or two people
moving within the Kinect field of view for gesture-driven applications.
3. Advanced audio capabilities: Audio processing capabilities include sophisticated
acoustic noise suppression and echo cancellation, beam formation to identify the
current sound source, and integration with the Windows speech recognition API.
4. Sample code and Documentation.
8/3/2019 Kinect Report
8/26
How to set up kinect with xbox 360 console ?
Position the kinect sensor
.
Kinect sensor needs to see your entire body.
Place the sensor near the edge on a flat, stable surface.
Position the sensor between 2 feet (0.6m) and 6 feet (1.8m) from the floor. Ideally,
the sensor should be within 6 inches (15 cm) above or below your TV.
Avoid positioning the sensor in direct sunlight or within 1 foot (.3m) of audio
speakers.
Do not manually tilt the sensor, it adjusts automatically.
Be careful not to drop the sensor
8/3/2019 Kinect Report
9/26
Set up the play space
Kinect needs to be able to see your entire body.
Clear the area between the sensor and the players.
One player: Stand back 6 feet (1.8 m).
Two players: Stand back 8 feet (2.4 m).
Make sure that the play space is at least 6 feet (1.8 m) wide, and not wider or longer
than 12 feet (3.6 m).
Make sure the room has bright, even lighting.
Connect the cables
8/3/2019 Kinect Report
10/26
Original Xbox 360 console
Plug one end of the cable into the USB port on the back of the console and the other
end into an electrical outlet.
If you have a wireless networking adapter, plug it into the USB port on the front of
the console by using the supplied USB extension cable.
Move all other USB accessories to USB ports on the front of the console.
Xbox 360 S console
If the Kinect sensor is attached to a cable with a power plug, disconnect the power
plug portion of the cable. A power cable is not needed with the Xbox 360 S console. Plug the sensor cable into the AUX port on the back of the console.
8/3/2019 Kinect Report
11/26
Update Console Software
To update your console software for Kinect
Insert a Kinect game disc and follow the on-screen instructions to update the console
software.
Don't have a Kinect game disc? Sign into Xbox LIVE. If a console update is available,
you will be prompted to update your console software.
8/3/2019 Kinect Report
12/26
Technology in kinect
Front view
Back view Side view Bottom view
kinect sensor device consisted of 6 components
3D depth sensors
RGB camera
Multi-Array MIC
Motorized Tilt
Kinect sensor
Microsoft software
3D Depth Sensors
RGB Camera
MultiArra MIC
Motorized Tilt
3D Depth Sensor Emitter
8/3/2019 Kinect Report
13/26
3D Depth Sensors
Consists of infrared laser projection combined with CMOS (complementary metal
oxidesemiconductor) sensor
captures video data in 3D under any ambient light conditions
adjustable sensing range
calibrating the sensor based on gameplay and players physical environment
make facial recognition more accurate
RGB Camera
a video camera that deliver 3 basic color components
found in various webcams and mobile phones
standard 640x480 resolution
operating at 30 frames per second
helps to enable facial recognition and more
depend on visible light
Multi-Array MIC
a microphone that able to locate voice and extract ambience noise.
Enable use of voice command
Able to navigate around with voice Enables chatting online (msn, skype)
Multi-Array Mic consisted of many microphones functioning as a single directional input
device.
Using sound propagation principles, the principal sound sources in an environment can be
located and distinguished from each other
Kinect Sensor
Combination of RGB Camera, 3D Depth Sensor and multiarray microphone
Track full body movement and individual voice, turning you into a controller
Microsoft Software
A proprietary software layer
Differentiates kinect from other technology in the market with its ability to enable
human body recognition and extract other visual noise.
Motorized Tilt
8/3/2019 Kinect Report
14/26
Necessary as not every room is the same size and not every television has the same
height
Can tilt up or down 30 degree
Zoom function
Projects on kinect
Project which involve hand manipulation
Object Manipulation
In the screenshot above, the man is manipulating the kettle by controlling the movement of
the kettle with his 2 hands.
More information can be found here :http://www.vimeo.com/17045326
Grabbing and letting go
http://www.vimeo.com/17045326http://www.vimeo.com/17045326http://www.vimeo.com/17045326http://www.vimeo.com/170453268/3/2019 Kinect Report
15/26
Need to enter the contours and convexity defects to estimate when a hand opens or closes
in the screenshots above , the person is controlling the virtual hand grabbing the yellow ball
and replacing it at another location with his own hands.
More information can be found here :http://vimeo.com/17333026
Shadow Puppet
In the screenshot above, a program was created in the hacked kinect to recognize the elbow,
forearm, wrist, fingers and thumb to in the bird shape.
The bird movement is controlled by the movement of the hand , by opening and closing your hand
like a beak causes the bird to do the same .
http://vimeo.com/17333026http://vimeo.com/17333026http://vimeo.com/173330268/3/2019 Kinect Report
16/26
More information can be found here :
http://www.popsci.com/diy/article/2010-11/five-hacks-free-microsofts-kinect-xbox?page=6
Making my room dance
In the screenshot above, the person is dancing and as the person dances, the room react to
the dancing movement of the person by shaking and moving with the flow.
More information can be found here :http://vimeo.com/17176078
Manipulating stuffed monkey
In the screenshot above, the person is controlling the monkey with his body movements
More information can be found here :
http://www.popsci.com/diy/article/2010-11/five-hacks-free-microsofts-kinect-xbox?page=6http://www.popsci.com/diy/article/2010-11/five-hacks-free-microsofts-kinect-xbox?page=6http://vimeo.com/17176078http://vimeo.com/17176078http://vimeo.com/17176078http://www.popsci.com/diy/article/2010-11/five-hacks-free-microsofts-kinect-xbox?page=68/3/2019 Kinect Report
17/26
http://technabob.com/blog/2011/08/11/kinect-hack-stuffed-monkey/
Project which help visually impaired to navigate around
In the screenshot above, the kinect is helping a visually impaired person to navigate around
warning the user about any obstruction or obstacles in his way by a vibrator in the belt, and
a Bluetooth headset which provide verbal feedback.
More information can be found here :http://medgadget.com/2011/03/project_navi_a_kinect_hack_that_helps_visually_impaired_navigat
e_indoors.html
Projects that involve creativity and drawings
Ghost following you
http://technabob.com/blog/2011/08/11/kinect-hack-stuffed-monkey/http://technabob.com/blog/2011/08/11/kinect-hack-stuffed-monkey/http://technabob.com/blog/2011/08/11/kinect-hack-stuffed-monkey/http://medgadget.com/2011/03/project_navi_a_kinect_hack_that_helps_visually_impaired_navigate_indoors.htmlhttp://medgadget.com/2011/03/project_navi_a_kinect_hack_that_helps_visually_impaired_navigate_indoors.htmlhttp://medgadget.com/2011/03/project_navi_a_kinect_hack_that_helps_visually_impaired_navigate_indoors.htmlhttp://medgadget.com/2011/03/project_navi_a_kinect_hack_that_helps_visually_impaired_navigate_indoors.htmlhttp://medgadget.com/2011/03/project_navi_a_kinect_hack_that_helps_visually_impaired_navigate_indoors.htmlhttp://technabob.com/blog/2011/08/11/kinect-hack-stuffed-monkey/8/3/2019 Kinect Report
18/26
In the screenshot above, the ghost is draw in the video frame and its designed in such a way
that the ghost is always behind your blind spot (behind you)
More information can be found here :
http://www.destructoid.com/kinect-hack-makes-the-mario-boo-ghost-really-follow-you-195351.phtml
Projects which involve virtual effects
Dragonball Z-Style Super Saiyan
In the screenshot above, the person grow out fake virtual hair and has a yellow aura around
his body and he can unleash the super saiyan which consisted of light beam and various
lightning effects.
More information can be found here :
http://www.pcworld.com/article/218681/go_super_saiyan_dragonball_zstyle_with_kinect_hack.ht
ml
Comic book sound effect in fighting scene
http://www.destructoid.com/kinect-hack-makes-the-mario-boo-ghost-really-follow-you-195351.phtmlhttp://www.destructoid.com/kinect-hack-makes-the-mario-boo-ghost-really-follow-you-195351.phtmlhttp://www.destructoid.com/kinect-hack-makes-the-mario-boo-ghost-really-follow-you-195351.phtmlhttp://www.pcworld.com/article/218681/go_super_saiyan_dragonball_zstyle_with_kinect_hack.htmlhttp://www.pcworld.com/article/218681/go_super_saiyan_dragonball_zstyle_with_kinect_hack.htmlhttp://www.pcworld.com/article/218681/go_super_saiyan_dragonball_zstyle_with_kinect_hack.htmlhttp://www.pcworld.com/article/218681/go_super_saiyan_dragonball_zstyle_with_kinect_hack.htmlhttp://www.pcworld.com/article/218681/go_super_saiyan_dragonball_zstyle_with_kinect_hack.htmlhttp://www.destructoid.com/kinect-hack-makes-the-mario-boo-ghost-really-follow-you-195351.phtmlhttp://www.destructoid.com/kinect-hack-makes-the-mario-boo-ghost-really-follow-you-195351.phtml8/3/2019 Kinect Report
19/26
in the screenshot above , when a person is being punched/kicked by another person , a
comic book-esque sound effects together with the wordings will be played
more information can be found here :
http://www.pcworld.com/article/221231/comic_kinect_turns_human_interaction_into_a_comic_b
ook_fight_scene.html
cool lighting effects
In the screenshot above, the kinect sensor is reacting to the movement and gesture of the
lady hand and thus triggering the lighting on the sofa
More information can be found here :http://www.youtube.com/watch?v=ho8KVOe_y08
http://www.pcworld.com/article/221231/comic_kinect_turns_human_interaction_into_a_comic_book_fight_scene.htmlhttp://www.pcworld.com/article/221231/comic_kinect_turns_human_interaction_into_a_comic_book_fight_scene.htmlhttp://www.pcworld.com/article/221231/comic_kinect_turns_human_interaction_into_a_comic_book_fight_scene.htmlhttp://www.youtube.com/watch?v=ho8KVOe_y08http://www.youtube.com/watch?v=ho8KVOe_y08http://www.youtube.com/watch?v=ho8KVOe_y08http://www.youtube.com/watch?v=ho8KVOe_y08http://www.pcworld.com/article/221231/comic_kinect_turns_human_interaction_into_a_comic_book_fight_scene.htmlhttp://www.pcworld.com/article/221231/comic_kinect_turns_human_interaction_into_a_comic_book_fight_scene.html8/3/2019 Kinect Report
20/26
Interactive stage lighting with kinect
In the screenshot above, hands will be placed in the hole and the kinect sensor inside the
box will read the hand movement and correspond to them
Some of the beautiful lighting effect projected out
More information can be found here :
http://kinecthacks.net/interactive-stage-lighting-with-kinect/
http://kinecthacks.net/interactive-stage-lighting-with-kinect/http://kinecthacks.net/interactive-stage-lighting-with-kinect/http://kinecthacks.net/interactive-stage-lighting-with-kinect/8/3/2019 Kinect Report
21/26
project involve robots
kinect powered robots that could help earthquake victims
In the screenshot above, the robot have sensors which act as eyes to search for victims in
an earthquake zone
They are the 3rd
team of peoples who revamped this rescue robots but using kinect as
shown below
http://kinecthacks.net/interactive-stage-lighting-with-kinect/http://kinecthacks.net/interactive-stage-lighting-with-kinect/http://kinecthacks.net/interactive-stage-lighting-with-kinect/http://kinecthacks.net/interactive-stage-lighting-with-kinect/http://kinecthacks.net/interactive-stage-lighting-with-kinect/http://kinecthacks.net/interactive-stage-lighting-with-kinect/http://kinecthacks.net/interactive-stage-lighting-with-kinect/http://kinecthacks.net/interactive-stage-lighting-with-kinect/8/3/2019 Kinect Report
22/26
In the screenshot above, a kinect powered robot can help to locate earthquake victims and
rescuing them out of the earthquake zone
More information can be found here :
http://www.pcworld.com/article/221075/kinect_powered_robot_could_help_earthquake_victims.h
tml
Kinect assembly robot
http://kinecthacks.net/interactive-stage-lighting-with-kinect/http://kinecthacks.net/interactive-stage-lighting-with-kinect/http://kinecthacks.net/interactive-stage-lighting-with-kinect/http://kinecthacks.net/interactive-stage-lighting-with-kinect/http://www.pcworld.com/article/221075/kinect_powered_robot_could_help_earthquake_victims.htmlhttp://www.pcworld.com/article/221075/kinect_powered_robot_could_help_earthquake_victims.htmlhttp://www.pcworld.com/article/221075/kinect_powered_robot_could_help_earthquake_victims.htmlhttp://www.pcworld.com/article/221075/kinect_powered_robot_could_help_earthquake_victims.htmlhttp://www.pcworld.com/article/221075/kinect_powered_robot_could_help_earthquake_victims.html8/3/2019 Kinect Report
23/26
step 1 : worker first prepare the materials on the surface
Step 2 : initiate the robot by pressing the start button
Step 3 : robots began to assembly the parts as programmed by the person in charge
More information can be found here :http://www.kinecthacks.com/kinect-assembly-robot/
Surgical robots
Start button
http://www.kinecthacks.com/kinect-assembly-robot/http://www.kinecthacks.com/kinect-assembly-robot/http://www.kinecthacks.com/kinect-assembly-robot/http://www.kinecthacks.com/kinect-assembly-robot/8/3/2019 Kinect Report
24/26
in the screenshot above, a kinect camera is used for hand gesture recognition to give
commands to the robots and to browse through radiological images on a nearby computer
screen
Example of hand gesture recognition : cutting with index and middle finger to indicate scissors.
more information can be found here :
http://medgadget.com/2011/02/kinectpowered_purdue_robot_to_replace_scrub_nurses.html
Kinect powered robot (jedibot) duelling
Kinect Sensor
http://medgadget.com/2011/02/kinectpowered_purdue_robot_to_replace_scrub_nurses.htmlhttp://medgadget.com/2011/02/kinectpowered_purdue_robot_to_replace_scrub_nurses.htmlhttp://medgadget.com/2011/02/kinectpowered_purdue_robot_to_replace_scrub_nurses.html8/3/2019 Kinect Report
25/26
In the screenshots above, the kinect sensor tracks the location of both lightsabers and has the
robot make an attack or defending move based on their positions.
More information can be found here :
http://science.kqed.org/quest/2011/07/26/stanford-students-create-kinect-powered-jedibot/
1 of the most popular games involve kinect
Star Wars
Kinect Star Wars allows fans to physically experience training as a Jedi, using the Force and
battling with a lightsaber.
Gameplay Screenshots
http://science.kqed.org/quest/2011/07/26/stanford-students-create-kinect-powered-jedibot/http://science.kqed.org/quest/2011/07/26/stanford-students-create-kinect-powered-jedibot/8/3/2019 Kinect Report
26/26
Flashing and slashing of lightsabers
Using the force to move and carry objects