Date post: | 13-Apr-2017 |
Category: |
Technology |
Upload: | bhoomit-morkar |
View: | 320 times |
Download: | 2 times |
KINECT SENSOR
Group members
Bhoomit Morkar 14103180Sachin Keniya 14103170Vaishnavi Shah 14103182
Ears and Eyes of your application
Content Introduction
• Basic Construction• Basic Specifications• Kinect’s Potential and applications
Working of Kinect• Classification of Depth Measurement Techniques
Triangulation Principles of TOF Imaging Principle of structured light
• Skeletal Tracking
Applications
References
Introduction
• It was Designed by Microsoft for the Xbox 360 video game console and windows PC’s.
• While making the Kinect sensor , Microsoft’s aim was to enable interactions between users and a game without the need to touch a controller.
• Kinect is an RGB-D sensor providing synchronized color and depth images.
• It’s greatest differentials is depth sensing technology that could be extended far beyond gaming and at a much lower cost than traditional 3-D cameras.
Why Kinect is revolutionary ?
• Kinect has proven to be an incredible piece of technology.
• It is the first practical natural user interface for machines communicating with people.
• It is combining acoustical and visual sense.
• It is recognizing full body movements, even complicated ones.
• It finds application not only in games but also in the field of medical sciences and robotics.
• After selling a total of 8 million units in its first 60 days, the Kinect holds the Guinness World Record of being the "fastest selling consumer electronics device".
Basic Construction
Power Comes from the Sum
• The Kinect Sensor is the outcome of several devices(or sensors) working together.
• It is comprised of a color camera (also known as RGB), a depth sensor, a 4-microphone array, and a motorized tilt.
• The depth sensor, in itself, consists of an infrared (IR) projector and an IR camera.
• Due to that Kinect being a motion sensing input device, able to capture, track, and interpret full-body movements(3D), and recognize gestures and voices.
• The depth sensor is a monochrome complementary metaloxide semiconductor (CMOS) sensor.
• One of Kinect’s greatest differentials is its relatively low cost depth sensor.
• The depth-sensing technology is licensed from the Israeli company PrimeSense. Although the exact technology is not disclosed, it is based on the structured light principle.
• The IR projector is an IR laser that passes through a diffraction grating and turns into a set of IR dots.
• The sensor has an RGB (red-green-blue) camera for color video.
• It has four microphones that enable your application to respond to verbal input, in addition to responding to movement.
Basic Specifications
Hardware
• Depth resolution : 640x480 px • RGB resolution : 1600x1200
px• FrameRate : 60 FP
Software• Depth;
30 FPS: 80x60 , 320x240 , 640x480
• Color;12 FPS : 1280x980 RGB15 FPS : Raw YUV 640x48020 FPS : 640x480
Kinect’s Potential and applications
Depth sensor Skeletal tracking Microphone array
Background removalObject recognition
Multi-user Easy Gesture Recognition
Sound source detection Speech recognition
How it works ?
Audio input
Microphone array
Sound threshold
How does Kinect hear ?
Track me !
Or
Speak louder .
what does it see?
Near mode depth ranges
Default mode depth ranges
Angle of vision(RGB-D)
Area under Observation
Classification of Depth Measurement Techniques
Triangulation
• In trigonometry and geometry, triangulation is the process of determining the location of a point by forming triangles to it from known points.
• Here rather than measuring distances to the point directly as in trilateration; we use of both angles and distance
Basic Mathematical model for Kinect
Principles of TOF Imaging
• Time-of-Flight (TOF) Imaging refers to the process of measuring the depth of a scene by quantifying the changes that an emitted light signal encounters when it bounces back from objects in a scene.
Pulsed ModulationContinuous Wave Modulation
Principle of structured light
• Project a known light pattern into the 3D scene, viewed by cameras.
• Distortion of light pattern allows computing the 3D structure.
• This pattern has to be unique per position in order to recognize each point in the pattern.
• Kinect uses pseudo random pattern.
speckles
Based on distance and orientation
First Region
High accurate depth.
(0.8 – 1.2 m)
Second Region
Medium accurate depth.
(1.2 – 2.0 m)
Third Region
Low accurate depth.
(2.0 – 3.0 m)
Depthobserved patternCalibration
IR camera observes the scene
known pattern
projected
3D map of the
beginning frame
New 3D map of the
scene
Renew 3D map of the
scene
Between projector
and camera&
x-direction speckle shift
Precision
Spatial x/y resolution
3mm @ 2m distance
Depth Z resolution
1cm @ 2m distance 10 cm @ 4m distance
operation range
0.8m ~ 4m | 0.5m ~ 3m
Skeletal Tracking
In the beginning, it was justhands;
• Head and hands,• Added shoulder and elbows.
Today!• It can track 20 joints of our body and
rans fast.
Process
Modes
Applications
• The Kinect fusion project allows a user to use an off-the-shelf Kinect camera as a 3D scanner for producing high-quality 3D scans of small or large objects.
• Researchers in China, in collaboration with Microsoft Research, developed the Kinect
Sign Language Translator, a system that can translate sign language into spoken and written language in near real time.
• The Holoflector is an augmented reality mirror. Behind the mirror is a large LCD monitor and right above the mirror is the Kinect.
• Combining a Kinect, a few hacking tools and a 3D environment, Kinectasploit allows a
user to break into a computer’s security system using body gestures. The system works almost like a video game. In one demo, a hacker ‘shoots’ at Wi-Fi targets to break into them.
• UBI Interactive developed their software UBI which combines your Windows 8 computer to Kinect and projector to create an interactive touchscreen off any surface.
• In another case of copying from the movies, many users have hacked their Kinect to allow users to control robots via body movement.
• We can pick out cloths and try them all out virtually with the help of Fitnect, an interactive dressing room.
References http://mms.ecs.soton.ac.uk/2014/papers/2.pdf
http://lshao.staff.shef.ac.uk/pub/KinectReview_TC2013.pdf
https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/Microsoft20Kinect20Sensor20and20Its20Effect20-20IEEE20MM202012.pdf
http://mms.ecs.soton.ac.uk/2014/papers/2.pdf
http://campar.in.tum.de/twiki/pub/Chair/TeachingSs11Kinect/2011-DSensors_LabCourse_Kinect.pdf
http://www.slideshare.net/MatteoValoriani/introduction-to-kinect-update-v-18?qid=e70a04cc-282d-4c11-a0d0-cff7f58c2435&v=&b=&from_search=9
http://www.slideshare.net/rojizofrio/xbox-one-kinect?qid=e70a04cc-282d-4c11-a0d0-cff7f58c2435&v=&b=&from_search=15
Videos https://youtu.be/uq9SEJxZiUg
http://youtu.be/bRgEdqDiOuQ?t=10m16s
http://www.google.com/patents/US20090
http://campar.in.tum.de/twiki/pub/Cha... (p. 33)
http://en.wikipedia.org/wiki/Range_im... (Stereo triangulation)
https://www.youtube.com/watch?v=-RELJD0LBg8&spfreload=5
https://youtu.be/uq9SEJxZiUg
http://www.hongkiat.com/blog/innovative-uses-kinect/