Post on 13-Jan-2016
transcript
Final Honours Presentation
Principal Investigator: João LourençoSupervisor: Dr Hannah Thinyane
Presentation Outline
Introduction Design and Implementation User Study Future Work Conclusion
2
Introduction
Problem Statement Research Motivation and Goals
3
Introduction
Problem Statement Research Motivation and Goals
4
Problem Statement
Migration from 2D output to 3D output3D TVSurround sound speakers
Suggests a need for migration from 2D input to 3D inputMicrosoft Kinect (previously Project Natal)Wii3D
5
Introduction
6
Research Motivation and Goals Intuitive 3D Interaction Techniques are
needed Develop 3D Interaction using Nintendo
Wii Remote Compare Proposed interaction
technique with existing techniques
7
Design and Implementation Hardware Software
OS and FrameworksBluetoothLibrariesPoint TrackerGestures and Gesture RecognitionUser Interface
8
Design and Implementation Hardware Software
OS and FrameworksBluetoothLibrariesPoint TrackerGestures and Gesture RecognitionUser Interface
9
Hardware
2 Nintendo Wii Remotes Personal Computer with Bluetooth
Capabilities 2 WiiGloves
10
Nintendo Wii Remotes
Layout Maximizing Volume
11
Nintendo Wii Remotes
Layout Maximizing Efficiency
12
WiiGlove
Circuit
2 Configurations2 IRs per hand4 IRs on one hand
13
Design and Implementation
14
OS and Frameworks
Windows 7 Microsoft C#.NET 4.0
15
Design and Implementation
16
Bluetooth
Microsoft Windows built-in Bluetooth Stack
17
Design and Implementation
18
Libraries
WiimoteLib 1.7.0.0 – Wiimote Connection
Accord.NET 2.1.1.0 – Statistics Math.NET Iridium 2008.8.16.470 –
Linear algebra 32feet.NET 2.2.0.0 – Bluetooth
Connection
19
Design and Implementation
20
Point Tracker Point tracking and probabilistic estimation
were required to ensure that the system would handle multiple inputs (multitouch) and continue to function even in the event of the loss of points. A loss of points can occur for several reasons:An infra-red source can leave the field of view of
the Wii RemoteAn infra-red source can move behind another
infra-red sourceTwo infra-red sources can come together and
merge
21
Point Tracker
Several ComponentsStereoscopic Triangulation SolverDistance MinimizerPredictive Interpolator
22
Stereoscopic Triangulation Achieved by solving for the point at the
near intersection of two rays Camera intrinsic properties needed
Viewport widthViewport heightHorizontal FOVVertical FOV
23
Stereoscopic Triangulation
24
Stereoscopic Triangulation Using the fields of view, a vector from the
camera position to the detected point was obtained. These vectors can be thought of as rays from the infra-red source to the camera.σ the horizontal field of viewψ be the vertical field of viewx be the horizontal component of the two
dimensional coordinatey be the vertical component of the two
dimensional coordinate
25
Stereoscopic Triangulation
26
Stereoscopic Triangulation With the two rays from the two Wii
Remote Cameras, the intersection of the two rays can be calculated using linear algebra.
27
Stereoscopic Triangulation
28
Stereoscopic Triangulation Physical System – rays would never
perfectly intersect Need to take an estimate
Estimate distance along first vectorEstimate distance along second vectorAverage the two points and make note of
the distance between them
29
30
31
Distance Minimization
The points that are reported by the two Wii Remotes are not necessarily sorted in the same order
32
Distance Minimization
Pairwise permutation of points in matrix form
Minimize distance error per row
33
Predictive Interpolation
Try to predict where a point should be – expected pointsInstantaneous velocityAverage velocity over a period
Minimize the errors between the observed and expected points
34
Predictive Interpolation
35
Predictive Interpolation
The selected point per row is used as the point that the Wii3D System reports
36
Design and Implementation
37
Gestures and Gesture Recognition Supported gestures Recognition methods
38
Click Gesture
A finger going down and forward, and then up and backward
39
Pan/Scroll Gesture
Two fingers that are close together moving in similar directions
40
Zoom Gesture
Two fingers moving in opposite directions with the distance between them increasing
41
Rotate Gesture
Two fingers moving in opposite directions with the distance between them constant
42
Circle Gesture
Hidden Markov Model Gesture
43
Multitouch Gestures
Number of points used for each gesture has been linked to the intensity of the gesture
44
Recognition
Movement – polling Simple gestures – Finite State Automata Complex Gestures – Hidden Markov
Models
45
Design and Implementation
46
User Interface
States:
47
User Interface
Upon start-up, the Wii3D System attempts to connect to the Wii Remotes
48
User Interface
Two possible errors – not 2 Wiimotes connected or no Wiimotes found
49
User Interface
Search for devices
50
User Interface
Found devices
51
User Interface
Allow the devices to install
52
User Interface
Connection success
53
User Interface
Start polling
54
User Interface
Currently polling
55
User Interface
Events received
56
User Study
Goals of the Experiment Design and Methodology
Tasks
Results
57
User Study
Goals of the Experiment Design and Methodology
Tasks
Results
58
Goals of the Experiment
To investigate whether the Wii3D System is a useful system
To investigate whether the Wii3D System could be used as an alternative to the mouse and keyboard
59
User Study
60
Design and Methodology
20 participants 15 minutes per user User consent required No individual data stored, only collated
results Supervised by researcher
61
User Study
62
Tasks
Introductory Questions Tasks Overview Questions
63
Introductory Questions
How long have you been using a mouse and keyboard for?8 years
How often do you use a mouse and keyboard?3 hours/day
64
Task 1 – Pointer Movement
65
Move pointers over targets
Task 2 - Clicking
Move over targets Click gesture
66
Task 3 – Panning/Scrolling Pan/Scroll and Image
67
Task 4 – Zooming
Zoom into and out of an image
68
Task 5 - Rotating
Rotate an image about the Z and X axes
69
Task 6 – Complex Gesture Hidden Markov Models were used Circle gesture
70
Overview Questions Would you use the Wii3D System as an
alternative to the mouse and keyboard? Why or why not?
If no, do you believe that the Wii3D System would be useful to someone else? Why?
Would you add/remove/change anything in the Wii3D System? Why?
You have undoubtedly had more experience using a mouse and keyboard than this type of system. What tasks would you find Wii3D more suitable for than a mouse and keyboard?
71
User Study
72
Results
Preliminary Observations Principal Results Participant Subjective Feedback Other Findings
73
Preliminary Observations
Confusion between rotation and zoom gestures
People are more used to mouse and keyboard
Hidden Markov Models not performing efficiently
Hidden Markov Models only successfully recognizing a circle gesture 50% of the time
74
Principal Results
None of the users would use Wii3D as a replacement for a mouse and Keyboard:Lack of typing supportGestures are not always recognizedAccuracy of the Wii3D System pointers was
not as good as a mouse
75
Principal Results
The majority of the participants stated that they thought that the Wii3D System would be useful
76
Movement
Mouse better for a single pointer
Wii3D better when there are more pointers
77
Clicking
Mouse better for single pointer clicks
Wii3D slightly better for multiple pointer clicks
78
Panning/Scrolling
Panning and scrolling is preferred with a mouse
Average and above ratings for Wii3D
79
Zooming
Zooming is preferred with the mouse Best ratings out of all the gestures for
Wii3D
80
Rotating
Preferred, again, is the mouse
Many users made zoom gestures instead
81
Complex Gesture
Circle gestures were recognized correctly 50% of the time using Hidden Markov Models
This result is reflected in the ten participants who rated this task as “Very Unintuitive”
82
Participant Subjective Feedback Users thought that the Wii3D System
would be useful for gaming and other three dimensional interactions
The users want typing functionality
83
Other Findings
The longer people have used computers, the more time they spend on them daily
84
Future Work
Improvements Additions
85
Future Work
Improvements Additions
86
Improvements
Multiple Wii Remotes Acceleration in predictive interpolation Dynamic Hidden Markov Models Gaussian Mixture Models in HMM
(continuous)
87
Future Work
88
Additions
GesturesDouble ClickFlickScaling an ObjectTyping
Artificial Neural Networks for Recognition
89
Conclusion
There is a use for the Wii3D System The Wii3D System would not work as a
replacement for the traditional mouse and keyboard
90
Questions/Comments
91