Date post: | 15-May-2015 |
Category: |
Technology |
Upload: | tom-mens |
View: | 520 times |
Download: | 2 times |
Introduction Architecture of KinectBalls Lessons learned and future work Conclusion
KinectBallsAn Interactive Tool for Ball Throwing Games
Jonathan Schoreels, Romuald Deshayes, and Tom Mens
UMons
4 July 2013
J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 1 / 17
Introduction Architecture of KinectBalls Lessons learned and future work Conclusion
Table of contents
1 Introduction
2 Architecture of KinectBalls
3 Lessons learned and future work
4 Conclusion
J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 2 / 17
Introduction Architecture of KinectBalls Lessons learned and future work Conclusion
Introduction
Context
Recent interest for new games and entertainment applications usingnew HCI techniques like natural interaction :
Nintendo’s Wii console
Microsoft’s Kinect sensor
Leap motion
J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 3 / 17
Introduction Architecture of KinectBalls Lessons learned and future work Conclusion
KinectBalls
Goal
Bridge the gap between the real and virtual world. The aim of thegame is to bring down a pile of virtual boxes by throwing a real balltowards them.
For this, we exploited the raw information of the Kinect’s 3D sensorto track the moving ball.
J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 4 / 17
Introduction Architecture of KinectBalls Lessons learned and future work Conclusion
Realisation
J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 5 / 17
Introduction Architecture of KinectBalls Lessons learned and future work Conclusion
Limitations
The Kinect has a low frame-rate (30 Hz). This could beaddressed by using an other sensor with better frame-rate.
Ball moving too fast can be unseen by the device.
Nothing can be seen at a distance below 50 centimetres.
The shape of the ball can be distorted.
J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 6 / 17
Introduction Architecture of KinectBalls Lessons learned and future work Conclusion
Table of contents
1 Introduction
2 Architecture of KinectBalls
3 Lessons learned and future work
4 Conclusion
J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 7 / 17
Introduction Architecture of KinectBalls Lessons learned and future work Conclusion
Implementation overview
J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 8 / 17
Introduction Architecture of KinectBalls Lessons learned and future work Conclusion
Object detection and tracking
For each frame Fn, we have for each pixel the depth. We cancompare a frame Fn with the previous frame Fn−1.
If that difference exceeds a threshold T (to filter out noise) :Something has moved on that particular pixel. We create the matrixMoveMap.
Creation of MoveMap
MoveMap(i , j) =
{1 if |Fn(i , j)− Fn−1(i , j)| > T0 otherwise
J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 9 / 17
Introduction Architecture of KinectBalls Lessons learned and future work Conclusion
Object detection and tracking
We search the biggest square of 1’s in MoveMap.
J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 10 / 17
Introduction Architecture of KinectBalls Lessons learned and future work Conclusion
Trajectory prediction
At least 3 positions of the ball : Approximation with 3second-degree polynomials.
At each new frame where the ball is detected, a newapproximation is computed.
At the predicted time of impact, a virtual ball is created atthe predicted position.
Knowing the exact 3D position of the wall, we can approximate thecollision between the trajectory approximated and the wall.
J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 11 / 17
Introduction Architecture of KinectBalls Lessons learned and future work Conclusion
Trajectory prediction
J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 12 / 17
Introduction Architecture of KinectBalls Lessons learned and future work Conclusion
Table of contents
1 Introduction
2 Architecture of KinectBalls
3 Lessons learned and future work
4 Conclusion
J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 13 / 17
Introduction Architecture of KinectBalls Lessons learned and future work Conclusion
Lessons Learned
Kids of 5 years and older tested the game for the ”Printemps dessciences 2013” :
They were enthusiasts.
The approximated impact point was good (between 1 to 5centimetres of precision)
Some adults had difficulties to interpret the 3D virtual world.
J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 14 / 17
Introduction Architecture of KinectBalls Lessons learned and future work Conclusion
Future work
Use a sensor with a frame-rate of 60 Hz
Exploit the Kinect’s RGB camera to create a virtual ball of thecolor of the thrown ball.
Automate the calibration between the Kinect and the screen.
Use an extra Kinect to track the head of the player and create avirtual window effect.
Multi-player game.
Performance improvements of algorithms.
Use a 3D beamer.
J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 15 / 17
Introduction Architecture of KinectBalls Lessons learned and future work Conclusion
Table of contents
1 Introduction
2 Architecture of KinectBalls
3 Lessons learned and future work
4 Conclusion
J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 16 / 17
Introduction Architecture of KinectBalls Lessons learned and future work Conclusion
Conclusion
KinectBalls requires only one very affordable 3D sensor.
Developed algorithms are fast enough to run in real time on astandard computer.
The solution worked fine in all tested indoor situations
J. Schoreels, R. Deshayes, and T. Mens (UMons) KinectBalls 4 July 2013 17 / 17