Wiimote and Kinect: Gestural User Interfaces add aNatural third dimension to HCI.

Rita FranceseUniversity of Salerno

via Ponte Don Melillo, 1Fisciano (SA), Italy

francese@unisa.it

Ignazio Passero∗

University of Salernovia Ponte Don Melillo, 1

Fisciano (SA), Italyipassero@unisa.it

Genoveffa TortoraUniversity of Salerno

via Ponte Don Melillo, 1Fisciano (SA), Italytortora@unisa.it

ABSTRACTThe recent diffusion of advanced controllers, initially de-signed for the home game console, has been rapidly followedby the release of proprietary or third part PC drivers andSDKs suitable for implementing new forms of 3D user in-terfaces based on gestures. Exploiting the devices currentlyavailable on the game market, it is now possible to enrich,with low cost motion capture, the user interaction with desk-top computers by building new forms of natural interfacesand new action metaphors that add the third dimension aswell as a physical extension to interaction with users. Thispaper presents two systems specifically designed for 3D ges-tural user interaction on 3D geographical maps. The pro-posed applications rely on two consumer technologies bothcapable of motion tracking: the Nintendo Wii and the Mi-crosoft Kinect devices. The work also evaluates, in terms ofsubjective usability and perceived sense of Presence and Im-mersion, the effects on users of the two different controllersand of the 3D navigation metaphors adopted. Results arereally encouraging and reveal that, users feel deeply immersein the 3D dynamic experience, the gestural interfaces quicklybring the interaction from novice to expert style and enrichthe synthetic nature of the explored environment exploitinguser physicality.

Categories and Subject DescriptorsH.5.2 [Information Interfaces and Presentation]: UserInterfaces; B.4.2 [Input/Output and Data Communi-cation]: Input/Output Devices.

General TermsDesign, Experimentation, Human Factors.

Keywords3D Interfaces, Natural User Interfaces, Motion Capture, Ki-

∗Corresponding author.

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.AVI ’12, May 21-25, 2012, Capri Island, ItalyCopyright 2012 ACM 978-1-4503-1287-5/12/05 ...$10.00.

nect, Wiimote, Human Computer Interaction, Empirical Eva-luation.

1. INTRODUCTIONOn the first day of April 2011, Google was announcing

the revolutionary Gmail Motion Beta application. Thanksto standard webcams and Google’s patented spatial track-ing technology, Gmail Motion claimed it would detect usermovements and translate them into meaningful charactersand commands. Despite the April Fool character of this an-nounce (one of the authors need to confess that, forgivingthe particular day, he tried to experiment the announcedservice), on the supporting website [4] there are sentencesreferred to gestural interfaces that sound indubitably inter-esting: “Easy to learn: Simple and intuitive gestures”, “Im-proved productivity: In and out of your email up to 12%faster” as well as “Increased physical activity: Get out ofthat chair and start moving today”.

Because of its main typewriting nature, the mailing activ-ity, is not the best target for benefiting of a gestural inter-face, but the technology is now mature and offers new con-sumer hardware that can easily support applications basedon natural human computer interaction. Traditional GUIsadopt mouse and keyboard building the interaction with theuser on artificial elements like windows, menus or buttons.Natural user interfaces disappear behind the content, anddirect manipulation style (e.g., touch, voice commands andgestures) is the primary interaction method adopted [2, 10].Despite that, too often the window/icons/mouse metaphorscontaminate the gestural interfaces vanishing their efficacy[28], the user is involved in a frustrating experience: themotion capture based interface fails in being effective sinceit’s used only for mimicking the classic mouse interaction.Indeed, differently from artifacts (e.g., documents, pictures,videos, etc. ), the cursor arrow is not a good target fordirect manipulation interfaces. Taking a look at game mar-ket, the game consoles capable of motion capture (almostall now) limit the window/icon interaction only to the basicoperations (e.g., game menus and console administration),and offer to the players gaming experiences based on theanalogies between control gestures and the real ones.

In this paper, we describe two applications that adopt ges-tural interaction for controlling user navigation of Bing maps[11]. The two applications, Wing (Wiimote Bing) and King(Kinect Bing), represent the occasion for experimenting,in the context of 3D environments, two natural interfacesbased on two consumer controllers: the Nintendo Wii Re-mote (also known as Wiimote) [19] and the Microsoft Kinect

Figure 1: The Wiimote and Nunchuk sensors con-figuration adopted for the Wing gestural interface.

[13]. The novelty of the interaction proposed is in the con-trolling metaphors that completely abandon the point andclick interaction of the Bing classic PC navigation for twonatural interfaces with the users based on gestures. Aimingat evaluating how the degree of body involvement affectsthe user perceptions about the experience, the two applica-tions have been empirically evaluated via the Usability Sat-isfaction Questionnaires [9] and via the well know Witmerand Singer’s one [30], specific for assessing perceived senseof Presence and Immersion in a virtual environment. Re-sults confirmed the enthusiastic impressions we previouslyhad observing that the users were quickly feeling comfort-able with the interfaces and were pleasantly interacting withboth systems.

2. BACKGROUNDIn the past years, the game console market has been a re-

ally competitive sector that was exploiting and often drivingthe development of state of the art processing and graphicaltechnologies to compete on a really exigent customer popu-lation. Recently, the market trend has changed and, follow-ing the evolving customer preferences, has been mainly fo-cused on realistic gestural human-computer interfaces thanon computing performance or on graphical capabilities ofthe proposed products [23]. With Wii TM console, Nintendo(2006) has proposed a game platform not particularly ex-citing in terms of performance but it broke several recordsas best sold console [32]. Reasons for this success are therevolutionary characteristics of Wii control system, the Wi-imote [8] (shown in Figure 1), its high expandability withseveral accessories and the possibility of offering to usersexperiential games improved with active gestures and re-ally effective playing metaphors. The success of NintendoWii clearly states the influence of the associated novel ges-tural interfaces on user satisfaction. Following Nintendo,Sony and Microsoft, the other two competitors in the gameconsole sector, were proposing their motion sensing gamecontrollers for answering to the user need of playing in anatural manner. Their answers to the market demand havebeen the PlayStation MoveTM (2008) and, only in November2010, the KinectTM controller. While the Wiimote and PSMove offer two similar controlling experience to the users,limited by the need of holding the controllers with hands,Kinect represents the first consumer full body motion cap-

Figure 2: Wing: the Wiimote and Nunchuk motioncontrollers during navigation.

ture device simply based on an Infra Red emitter and twovideo cameras. However, thanks to motion detection, allthese controllers let the gaming experience to be realisticallybased on gestures analogous to the mimicked ones. WhileWiimote and Kinect are imported from game console worldto the computer one, Asus and PrimeSense were propos-ing Wavi Xtion, their low cost motion capture alternative,specifically designed for PCs and smart TVs [1]. Exploitingthe availability of a simple connection with normal PCs andthe diffusion of official or unofficial SDKs [26, 16, 21] fordeveloping desktop applications controlled by these devices,the Research is exploring new interaction instruments andmodalities as well as new natural interfaces for the more dif-ferent applications in several disciplines, ranging from teach-ing to medicine.

2.1 Wiimote and ApplicationsWii Remote (often shorten as Wiimote) [19] was intro-

duced in 2006 by Nintendo and promoted the success of theWii console. The Wiimote communicates over a wirelessBluetooth connection, offers a set of classic joypad buttonsand senses acceleration along three axis. Wiimote is alsoequipped with an optical sensor that, associated to an InfraRed (IR) source (i.e., the Wii sensor bar), allows to deter-mine where the device is pointing. Wiimote can be comple-mented with Motion Plus that adds a gyroscope to improvethe detection of complex movements [29]. The device isequipped with 5.5 Kilobytes of memory (almost adopted foruser customisations) and adopts as feedback mechanisms aspeaker, a vibration motor and four light emitting leds [8].Nunchuk is an extension that plugs into Wiimote via a con-nection cable and adds 2 buttons, an analog joistick as wellas an independent three axis accelerometer (usually associ-ated with the secondary hand of the user). The adoptionof motion sensing game controllers on desktop computersenables to implement novel interfaces capable of deeply in-volving users in realistic experiences. In [28], the authors,considering touch based interfaces, claim natural user inter-faces to be characterised mainly by a high learnability. Inour case, the users quickly and spontaneously move fromwhat we consider a basic navigation style to an expert one,

but, as shown in the Evaluation Section, we are also inter-ested in the impact of user perceptions and involvement onthe proposed 3D navigation, as in the case of the kinestheticlearning experience proposed in [5]. In this paper, Ho-ShingIp et al. propose a didactic experience exploiting the inter-play between body, mind and emotions for amplifying thelearning value and a model for investigating the effects of im-mersive body movement interaction with virtual charactersand scenarios. In particular, they adopt the Wiimote andthe Nunchuk extension for controlling the flight of a bird ina Hummingbird Flying Scenario. As [5], we also exploit theamplifying effects on usability and user involvement due tonatural interaction style interfaces and their physical naturefor giving physicality to the synthetic environments adopted.De Paolis et al. propose, in [3], a serious game based on thephilological reconstruction of city life in the XIII century.As input peripheral, the authors adopt a Wiimote controllerwith the Balance Board extension, that adds four pressuresensors to the control system and is used for detecting walk-ing gestures. Yang and Lee, in [32], propose the adoptionof Wiimote as a wireless presentation controller and a wire-less mouse. They adopt the IR sensor for tracking the Wi-imote pointing direction of up to four users. Santos et al., in[22], perform a user study aiming at comparing two differ-ent Wiimote configurations with the classic desktop mousein controlling Google Earth navigation. Both the proposedWiimote configurations mimic with two buttons the mouse:one detects user movements via accelerometer, the other viathe IR sensor. The study reveals that Wiimote presents sev-eral advantages over desktop and mouse. Differently from[22], we adopt and evaluate two applications based on Wi-imote and Kinect controllers and propose two natural in-terfaces explicitly designed for 3D navigation and really farfrom the classic desktop metaphors.

2.2 Kinect and ApplicationsWith Kinect, Microsoft distributes, as a controller for

Xbox system, the first motion capture device on the con-sumer market. The device is available from November 2010,the first unofficial SDK is dated December 2010 [26, 27],while the first official SDK for PC users was released byMicrosoft on June 2011 in beta version and is free for noncommercial uses [16]. An estimation of Kinect marketingsuccess can be done if we consider that within the first 25days, Microsoft sold 2.5 millions Kinect devices [23].Kinect sensor embeds a four-element linear microphone

array capable of sophisticated acoustic echo cancellation,noise suppression and direction localisation as well as anIR emitter and two cameras that deliver depth information,colour images and skeleton tracking data. The natural userinterface API, in the Kinect for Windows SDK, enables ap-plications to access and manipulate the data collected bythe sensor [16]. The optimal working distance ranges from0.8 to 4 meters. In this range, the depth and skeleton viewsdetect users only if the entire body fits within the capturedframe but the device pointing direction can be adjusted bya motorised tilting mechanism. For overcoming the workingdistance restrictions still maintaining a good screen readabil-ity, we were visualising the King client via a room projector(we were doing the same for Wing, aiming at avoiding screendifferences to bias the proposed evaluation).In the context of 3D models navigation, Lacolina et al.

adopt natural interfaces based both on multitouch tables

Figure 3: King: the Kinect device controls Bingmaps navigation.

and on gesture recognition [7]. The motion capture is per-formed analysing the raw depth images provided by a Kinectsensor.

Phan adopts the OpenNI toolkit and develops a Kinectclient for controlling Second Life gestures aiming at estab-lishing a direct channel between the user body and his/heravatar [24]. Also in this case, the aim is improving the vir-tual environment experience and the perceived immersion byletting the user interface to disappear behind real gestures.

Boulos et al. base on Kinect their application, Kinoogle[6], and develop a gestural interface for controlling GoogoleEarth navigation. The proposed gestures are mainly basedon hand tracking and resemble the classic multitouch inter-action style. Differently from them, we propose a navigationcontrol that is inspired to natural flight gestures and user ac-tions reflect on the map navigation according to metaphoricsimilarity.

3. WING AND KING APPLICATIONSWing and King are the two controller applications devel-

oped on Wiimote with Brian Peek’s SDK [21] and on Kinectwith the official SDK [16]. Both applications control a Bingmap client [11] and react to user gestures inspired by wellaccepted metaphors.

It is important to point out that Bing maps represent justan instance of 3D navigable environments and provide us thedimensions for experimenting our natural interfaces. Duringthe evaluation phases we noticed, for both Wing and Kingsystems, that the users quickly and spontaneously movedfrom novice use, characterised by single navigating com-mands at a time, to a sort of expert use, when they startedto combine turning with altitude and movement commands,generating a more complicate navigation path. A video ofthe applications is available at [20].

3.1 Wing and the Wiimote ControllerWing is a Bing map navigator controlled by the accelerom-

eters of a Nintendo Wiimote and a Nunchuk [8]. The ap-plication is developed in C# [15] and connects to both con-trollers via bluetooth using the Wiimote lib [21]. The con-trollers and the dimensions adopted for building the Wing

natural interface are shown in Figure 1. In the image, themovements detected as controls are depicted on the Wiimote(right side of the image) and on the Nunchuk (left side) de-vices. Wing proposes to users two interaction metaphorswell diffused and accepted in videogame sector. The maincontroller acts on forward/backwards movements when ro-tated along its longer dimension (i.e., Roll). Its inclination(i.e., Pitch) determines if the navigation turns. Both thegestures are inspired to the motorcycle metaphor: while theWiimote acts with its Roll rotation as a motorcycle throt-tle command connected to navigation forward and backwardmovements, the turning gestures resemble the handlebar ofthe imaginary motorcycle during turning. The forward andbackward movements can be requested at different speedsaccording to the Wiimote rotating angle. The aeroplanecloche metaphor is implemented on the Nunchuk and con-trols altitude: its tilting direction determines vertical varia-tions of the map navigation.Figure 2 shows the Wing application and how users grasp

the controls during the navigation. The Wing user holdsboth controllers with forearms aligned with the elbows. Theturning gestures are detected when the user turns the Wi-imote: the credibility of the handlebar metaphor is deeplyperceived and, during navigation, we observed a big partof users keeping Wiimote and Nunchuk aligned even if thetwo components are independent. The altitude gestures areactivated by Nunchuk when the user rotates the wrist up-wards (increases alt.) and backwards (decreases alt.) orwhen he/she accordingly bends the forearm on the elbow. Inboth cases the gesture well reflects the videogame action ofpitching up/down an aeroplane cloche. Altitude, movementand turning commands can be combined obtaining complexflight/navigation behaviours. All the proposed gestures arebecoming more and more popular among gamers [17, 18, 19]and are ready to be extended to PC users.

3.2 King and the Kinect controllerKing is a Bing map navigator based on Microsoft Kinect

controller. The application has been developed in C# andassociates to the Bing map a simple window showing a paperaeroplane on a sky background. The application controls theKinect sensors via the official SDK [16].Figure 3 shows the King application and the Bing map

client during the navigation. The paper aeroplane reflectsthe gesture performed by the user and is the only feedbackmechanism (useful, if we consider that respect to Wing, theKing interface is completely hand-free). King proposes toits users the bird (or aeroplane) metaphor and customiseson it the gestures associated to the various commands. Fig-ure 3 shows a user performing a left turn: she inclines thealigned arms downward on the left as a bird or an aero-plane would have done and while the flight on the Bing mapturns, accordingly, the paper plane in the feedback windowperforms a similar rotation. The idea is to mimic the bird’swing movements, when possible, with the arm gestures. Atthe moment, the game market still does not offer examplesof similar gestures but generic natural interfaces for sport,fighting or dancing games are already available [14, 25, 12].Figure 4 shows the controlling gestures used for King mapnavigator. The neutral position for the navigation is de-picted in sub-picture (a): when the user stands with openand aligned arms, the navigation halts. The gesture asso-ciated to forward moving is depicted in sub-plot (b) and is

Figure 4: The King Control gestures.

Figure 5: The results of ASQ questionnaires.

detected when the user moves one (slow motion) or bothhands (fast motion) ahead of the elbows. This correspondsto the skeleton depicted in (b) (arms bent) but is also de-tected when the user extends forward his arms.The bird/plane metaphor does not contemplate a backwardmovement and we do not violate this assumption providinga surrogate gesture. Figure 4 (c) and (d) depict the turn ges-tures that were previously described. Altitude of navigationis controlled by gestures (e) and (f). The idea is in exposingto King users two gestures that could be easily associatedto rise or going down effects, avoiding the need of dynam-ically mimicking the bird flight gesture that is quite tiring.For rising or decreasing altitude, King users are required tostart and continue the static gesture ((e) or (f)) until thedesired observation height is achieved. Figure 4 depicts alsothe states of the feedback paper aeroplane accorded to thegesture detected. The movement, turning and altitude ges-tures proposed as King natural interface can be combinedobtaining the desired navigation experience.

4. EVALUATIONThe Wing and King applications have been evaluated in

an laboratory session organised according to the suggestionsprovided by Wohlin et al. in [31] aiming at assessing per-ceived usability and sense of Presence in the Virtual En-vironment [30]. Participants of the study have been 24 (8girls) undergraduate students and employees of our Faculty

who volunteered taking part to the experiment. The stu-dents population we selected was chosen from a programthat does not require or provide particular competences in3D virtual environments, games and natural user interfaces.Their ages ranged between 18 and 41 years old with an av-erage of 24. Before starting the experiment, we assessedparticipant skills in the videogames and natural interfacessectors. In our sample, 8 participants indicated to be playingdigital games at least once a week, three were Nintendo Wiiplayers and just two of them were using Xbox and Kinect.

4.1 Subjective UsabilitySubjective usability has been evaluated via the After-Scena-

rio (ASQ) and the Computer System Usability (CSUQ) ques-tionnaires that, as shown by Lewis, provide strong evidenceof generalizability of results and of wide applicability. Thequestions have been evaluated on the seven-point Likertscale anchored from 1 (strongly disagree) to 7 (stronglyagree).The ASQ is a three-item questionnaire that is used to

assess participant satisfaction after the completion of eachtask and evaluates the time to complete the task, the easeof completion and the adequacy of support information.The CSUQ questionnaire is made by 19 questions assess-

ing user satisfaction with system usability and can be aggre-gated in four factors:

• Overall Evaluation (OVR),

• System Usefulness (USE),

• Information Quality (INFO),

• Interface Quality (INTERF);

More details on the questionnaires and the questions areavailable in [9].

4.2 Presence and ImmersionIn this work we adopt Bing maps as a 3D virtual environ-

ment in which experimenting the Wing and King interfaces.3D environments have a significant advantage over settingsbased on 2D technology since they induce a strong Presencesensation in their users [30]. During Bing map navigation,users move in a virtual space generated by the computer,react to actions and change their point of view on the scenewith movement. Witmer and Singer define Presence as “thesubjective experience of being in one place or environment,even when one is physically situated in another”and“...pres-ence refers to experiencing the computer-generated environ-ment rather than the actual physical locale”. As stated bythem, several factors contribute to increase presence: Con-trol, Realism, Distraction and Sensory input. Presence ismaximised when the user interacts with the environment ina natural manner, controls the events, when he sees the sys-tem behaving as expected and the 3D environment changingaccordingly to his commands. The minimisation of distrac-tions that can occur when a user has problems in controllingthe navigation, as an example, can increase the perceivedimmersion in the experience and the virtual environment.As suggested in [5], we hypothesised that the physical di-

mension of the proposed interfaces may influence user senseof immersion in the proposed navigation experience. Aim-ing at assessing the degree of Presence perceived by usersduring the tasks, we extracted 12 questions appropriate for

Table 1: Witmer and Singer questions

1Were you involved in the experimental taskto the extent that you lost track of time?

INV

2How involved were you in the virtual envi-ronment experience?

INV

3

How well could you concentrate on the as-signed tasks or required activities ratherthan on the mechanisms used to performthose tasks or activities?

DF

4How much did the control devices interferewith the performance of assigned tasks orwith other activities?

DF

5How responsive was the environment to ac-tion that you initiated (or performed)?

CF

6How natural was the mechanism whichcontrolled movement through the environ-ment?

CF

7 How natural did your interactions with theenvironment seem?

CF

8How proficient in moving and interactingwith the virtual environment did you feelat the end of the experience?

CF

9Were you able to anticipate what wouldhappen in response to the actions that youperformed?

CF

10How quickly did you adjust to the virtualenvironment experience?

CF

11 How compelling was your sense of movingaround inside the virtual environment?

CF

12How much did your experiences in thevirtual environment seem consistent withyour real-world experiences?

CF

our empirical evaluation from the Witmer and Singer ques-tionnaire. The questions are reported in Table 1 aggregatedunder three factors:

• Involvement (INV),

• Distracion Factor (DF),

• Control Factor (CF).

Also the answers to this questionnaire have been formulatedon the seven-point Likert scale: from 1 (strongly disagree)to 7 (strongly agree).

4.3 Experiment DesignIn the proposed usability study, participants tried in quick

succession our gestural interfaces engaging in two naviga-tion tasks. After being singularly instructed on the Wingand King systems, the users were required to complete thenavigation of two geographical paths involving well knownItalian cities:

• SEA:Cagliari-Napoli-Palermo

• LAND:Genova-Roma-Venezia

Both tasks are comparable in terms of distances and diffi-culties in localising the target cities. However, aiming atavoiding to bias the evaluation with task or tested appli-cation orders, we adopted for the experiment a balancedpaired design as suggested in [31]: we divided our users intwo groups: each member of the same group was startingthe experiment with the same system. Among each group,half of the participants was starting with the SEA task and

Figure 6: The results of CSUQ questionnaires.

Table 2: CSUQ categories detailsOVR USE INFO INTERF

µ σ µ σ µ σ µ σWing 5.13 1.08 4.85 1.21 5.17 0.94 5.75 1.33King 5.78 0.75 5.89 0.78 5.4 0.82 6.25 0.85

the other with the LAND one. After each task, all partici-pants filled the ASQ, the CSUQ questionnaire and answeredthe questions from Witmer and Singer questionnaire [30] re-ported in Table 1. Let us point out that question 4 resultshas been reversed before aggregating the DF category.

4.4 ResultsThe first good impressions on the usability of the proposed

systems were collected during the experiment by listeningat participant comments and observing their behaviours.These insights were lately confirmed by examining the ques-tionnaires answers. Figure 5 shows the boxplots depictingthe ASQ results that gave a preliminary idea about taskdifficulties, user preferences and perceived usability of theproposed interfaces. The users globally assigned really highscores to both systems but the effects of the King feedbackmechanism (the paper aeroplane) brought higher the INFOscore associated to King respect to its competitor. In thesame direction the EASE boxes confirm the tasks performedvia the Kinect interface to have been perceived as easier re-spect to Wing ones and this categories show the bigger dif-ference among the two systems. Also the TIME categoriesstate that the two tasks have been perceived in the samemanner respect to the time assigned, but the results are al-ways characterised by a little preference for King system.Figure 6 reports the results of CSUQ aggregated in the fourcategories suggested by Lewis. The first observation on datais about dispersion: comparing the two boxplots in Figure6, it is evident that users perceptions are characterised by ahigher variability for Wing scores respect to King ones. Thelatter system was also perceived as better respect to all thefour aggregating factors but exhibits higher differences re-spect to OVR (Overall Usability) and USE (System Useful-ness) categories (as shown in Figure 6 and detailed in Table2): the better performance of King is mainly concentratedin its System Usefulness and influences the overall opinionabout the systems. Table 2 resumes, via the µ and σ values,the results of CSUQ categories. All the considerations de-ducted on Figure 6, on boxes (and consequently on medians)reflect, obviously, on the values reported in Table 2: Kingencounters more user enthusiasm than Wing. A direct ob-

Figure 7: The results of Presence questionnaire.

Table 3: Witmer and Singer categories detailsINV DF CF

µ σ µ σ µ σWing 5.39 0.83 4.41 1.14 5.87 0.27King 5.89 0.69 5.39 0.94 6.14 0.26

servation of users during the study suggests the same conclu-sion: participants have been almost all disappointed at theend of their King task showing that they would prefer con-tinuing the experience based on bird/aeroplane metaphor.Once assessed the degree of Usability and perceived SystemUsefulness for both Wing and King systems, we extendedthe evaluation to user perceptions in terms of Presence andInvolvement in the virtual experience. At this aim, we ex-tracted 12 questions from Witmer and Singer questionnaireto integrate ASQ and CSUQ ones. As shown in Table 1, thePresence questionnaire is aggregated in three factors aimingat assessing how users experience the computer-generatedenvironment rather than their physical locale. The adoptionof a controller based natural interface (Wing) and a hand-free natural one (King) let us understand if, in the contextof 3D map navigation, physical gestures (and the sensor de-vice nature) increase experience likability and involvementdeepness. Figure 7 shows the Presence questionnaire resultsaggregated in boxplots respect to the previously describedfactors. The measures of central tendency and spread for thePresence categories are reported in Table 3. Also in the caseof Presence and Involvement, Wing opinions show a highervariability respect to King ones and, accordingly, user im-pressions are more concordant for King system respect toWing. What is really remarkable is that, despite Wing sys-tem had a good success in user evaluations, higher resultvalues were obtained by the King experience. As an exam-ple, Involvement factor was evaluated µ=5.39 for Wing whileKing was performing better (µ=5.89). Both systems havebeen positively judged in term of Control Factor, as shownin the rightmost boxes of both subplots of Figure 7: Table 3details King to obtain a µ=6.14 while Wing scores µ=5.87.The Wing interface (µ=4.41), respect to the category Dis-traction Factor, has been perceived a little less effective thanthe King one (µ=5.39). This has been probably due to thehand-free interface that was built tanks to the adoption ofKinect sensor: while the Wing user holds the wiimote, theMicrosoft device has proved to be really effective in lettingthe interface to disappear behind natural gestures.

Respect to the sense of Presence and Involvement in theBing Virtual Environment it can be interesting to further de-

Figure 8: The results of Witmer and Singer ques-tions.

tail the answers to some of the questions reported in Table 1,that let better understand the effects of the proposed inter-faces. Figure 8 aggregates in the two subpictures (labelledWing and King) the histograms for all the twelve questionsextracted from the Witmer and Singer questionnaire.Question 2 is directly formulated for assessing the degreeof user involvement in the Virtual Environment experience.As shown in Q2 histograms of Figure 8, Wing received goodscores from almost all participants but King was the betterperforming application and concentrated the great part ofuser votes around 6 and 7, directly stating that the phys-ical interface proposed induces a strong and deep sense ofinvolvement in its users.Question 3 focuses on the intrusiveness of the interface thatideally should disappear respect to user actions. King in-terface appears to be the more transparent to the users andalmost all participants scored it above 4 while the Wingexperience induced in its users more various opinions (dif-ferently from King, also negative ones). This is confirmedby the answers to Question 6 and 7 specifically evaluatinghow natural the proposed gestural interfaces have been per-ceived. In particular, question 6 aims at evaluating the in-terface while the 7th one is focused on the interaction withthe environment. Despite Wing has obtained high values forquestion 6 (Figure 8 shows the result bell centred between 5and 6), King shows all the benefits related to its hand-freegestural interface with 21 users voting it more than 5. Alsoin this case, it is important to point out that, with betteruser opinions, King evaluators also agreed with a minor dis-persion on their preferences. By examining the answers toquestion 7, it is possible to notice that the user preferences

went in the same direction of previous answers but, in thiscase, the differences between the two systems are less evi-dent: King natural interaction with Bing environment washowever better perceived than Wing’s one. Resuming, theevaluation provided really good user impressions: Wing andKing systems were judged usable and the user were satisfiedwith both. The proposed experience has also shown thatthe more the interface is natural (in the sense that it disap-pears behind the gesture) the more the users are involvedin the virtual environment and hosted activities. However,we are conscious that Kinect novelty may have influencedparticipants, but we also trust that the physicality of thenatural interfaces proposed and the immediate learnabilityof the metaphors are the main motivations that positivelyinfluenced the testers and their opinions (see also the highscores obtained by Wing and its interface).

Obviously, results presented in this work are strictly re-lated to the context adopted for the evaluation and are lim-ited to 3D navigation of virtual environments. The sameinterfaces may not be suitable for other applicative domainsand the user reactions and opinions may be different. TheKinect approach, indeed, even if easy to learn, is not appro-priate for a prolonged usage because of the physical effortrequired to users. This effort is however vital for giving aphysical dimension to the navigating experience of 3D en-vironments: the involvement results are very positive andthe physical perception of the experience, amplified by thesurrounding and physical nature of the controlling interface,is very relevant. As a consequence, the Kinect approach(but also the wiimote one) can be very appealing for kideducation. Indeed, when teaching geography, this kind ofexperience helps to deeply involve kids and soliciting theirspatial perception of the explored environment. As a futurework, it will be interesting experimenting Wing and Kingapproaches in a primary didactic setting and understand-ing the educational effects of the proposed, game derived,controlling metaphors.

5. CONCLUSIONThe recent diffusion of consumer game controllers that

offer motion tracking functionality and the release of con-nection drivers and SDKs and represent wonderful oppor-tunities for implementing and experimenting new forms of3D user interfaces based on gestures. In this paper, Wingand King applications, as well as the associated navigationmetaphors, have been presented and evaluated in terms ofsubjective usability and perceived sense of Presence andImmersion. The proposed applications adopt user motiontracking via the Nintendo Wii and the Microsoft Kinect de-vices. The two Bing map navigators proposed just representthe occasion for experimenting 3D gestural interfaces andtheir usability as well as assessing the effects on user senseof Presence and immersion in a synthetic 3D environment.

Results of the evaluation performed via standard ques-tionnaires are really encouraging and, also considering thenatural satisfaction boost related to the novelty of Kinectcontroller, suggest that the more the interface is naturaland involves their body in the action, the more the userare satisfied and involved in the 3D maps navigation ex-perience. Important success usability factors found are theease of use of the King system and the deep involvementhis gestural interface induces in users. Lesson learnt withthis experience suggests to avoid, when possible, the clas-

sic window/icon/mouse interaction for experimenting, ob-viously for appropriate tasks (i.e., not keyboard intensive,etc.), new gestures and new forms of physical commands.As a future work, we intend to experiment the interfaces ina kid geographical didactic context that will probably bene-fit of the effects of the proposed experiences and interfaces.

6. ACKNOWLEDGMENTSWe would like to thank the little Giuseppe for his natu-

ral attitude to game and his children’s insatiable desire toexplore.

7. REFERENCES[1] ASUS. Wavi xtion. retrieved on December 2011 from

http://event.asus.com/wavi.

[2] J. Blake. Natural User Interfaces in .NET (EarlyAccess Edition). Manning Publications Co.,Greenwich, CT, 2010.

[3] L. De Paolis, G. Aloisio, M. Celentano, L. Oliva, andP. Vecchio. Mediaevo project: A serious game for theedutainment. In Computer Research and Development(ICCRD), 2011 3rd International Conference on,volume 4, pages 524 –529, march 2011.

[4] Google. Gmail motion beta. retrieved on December2011 fromhttp://mail.google.com/mail/help/motion.html.

[5] H. Ip, J. Byrne, S. Cheng, and R. Kwok. The samalmodel for affective learning: A multidimensionalmodel incorporating the body, mind and emotion inlearning. In DMS 2011 : The 17th InternationalConference on Distributed Multimedia Systems, pages1–6, august 2011.

[6] M. Kamel Boulos, B. Blanchard, C. Walker,J. Montero, A. Tripathy, and R. Gutierrez-Osuna.Web gis in practice x: a microsoft kinect natural userinterface for google earth navigation. InternationalJournal of Health Geographics, 10(1):45, 2011.

[7] S. A. Lacolina, A. Soro, and R. Scateni. Naturalexploration of 3d models. In Proceedings of the 9thACM SIGCHI Italian Chapter InternationalConference on Computer-Human Interaction: FacingComplexity, CHItaly, pages 118–121, New York, NY,USA, 2011. ACM.

[8] J. Lee. Hacking the nintendo wii remote. PervasiveComputing, IEEE, 7(3):39 –45, july-sept. 2008.

[9] J. Lewis. Ibm computer usability satisfactionquestionnaires: psychometric evaluation andinstructions for use. International Journal ofHuman-Computer Interaction, 7(1):57–78, 1995.

[10] W. Liu. Natural user interface- next mainstreamproduct user interface. In Computer-Aided IndustrialDesign Conceptual Design (CAIDCD), 2010 IEEE11th International Conference on, volume 1, pages 203–205, nov. 2010.

[11] Microsoft. Bing maps beta. retrieved on December2011 from http://www.bing.com/maps/.

[12] Microsoft. Dance central 2. retrieved on December2011 from http://tinyu.me/vO2eN.

[13] Microsoft. Kinect. retrieved on December 2011 fromhttp://www.xbox.com/en-US/Kinect.

[14] Microsoft. Kinect sport season two. retrieved onDecember 2011 from http://tinyu.me/EZFNk.

[15] Microsoft. Visual c#. retrieved on December 2011from http://msdn.microsoft.com/en-us/library/kx37x362.aspx.

[16] Microsoft. Kinect sdk for developers. retrieved onDecember 2011 from http://kinectforwindows.org/,2011.

[17] Nintendo. Donkey kong jet race. retrieved onDecember 2011 from http://tinyu.me/VdTB7.

[18] Nintendo. Mario kart wii. retrieved on December 2011from http://www.mariokart.com/wii/launch/.

[19] Nintendo. Wii remote. retrieved on December 2011from http://www.nintendo.com/wii/what-is-wii/#/controls.

[20] I. Passero. Two bing maps controllers based on kinectand wiimote devices. retrieved on December 2011 fromhttp://youtu.be/ITtd02h5G5w.

[21] B. Peek. Wiimote lib: Managed library for nintendo’swiimote. retrieved on December 2011 fromhttp://wiimotelib.codeplex.com/.

[22] B. Santos, B. Prada, H. Ribeiro, P. Dias, S. Silva, andC. Ferreira. Wiimote as an input device in googleearth visualization and navigation: A user studycomparing two alternatives. In InformationVisualisation (IV), 2010 14th InternationalConference, pages 473 –478, july 2010.

[23] K. Sung. Recent videogame console technologies.Computer, 44(2):91 –93, feb. 2011.

[24] P. Thai. Using kinect and openni to embody an avatarin second life: Gesture & emotion transference.retrieved on December 2011 fromhttp://tinyu.me/o2GzQ.

[25] UBISOFT. Fighters uncaged. retrieved on December2011 from http://fighters-uncaged.uk.ubi.com/.

[26] Various. Openkinect. retrieved on December 2011from http://openkinect.org.

[27] N. Villaroman, D. Rowe, and B. Swan. Teachingnatural user interaction using openni and themicrosoft kinect sensor. In Proceedings of the 2011conference on Information technology education,SIGITE ’11, pages 227–232, New York, NY, USA,2011. ACM.

[28] D. Wigdor and D. Wixon. Brave NUI World:Designing Natural User Interfaces for Touch andGesture. Morgan Kaufmann, 1 edition, Apr. 2011.

[29] C. A. Wingrave, B. Williamson, P. D. Varcholik,J. Rose, A. Miller, E. Charbonneau, J. Bott, and J. J.LaViola Jr. The wiimote and beyond: Spatiallyconvenient devices for 3d user interfaces. ComputerGraphics and Applications, IEEE, 30(2):71 –85,march-april 2010.

[30] B. Witmer and M. Singer. Measuring presence invirtual environments: A presence questionnaire.Presence, 7(3):225–240, 1998.

[31] C. Wohlin, P. Runeson, M. Host, M. C. Ohlsson,B. Regnell, and A. Wesslen. Experimentation insoftware engineering: an introduction. KluwerAcademic Publishers, Norwell, MA, USA, 2000.

[32] Y. Yang and L. Li. Turn a nintendo wiimote into ahandheld computer mouse. Potentials, IEEE, 30(1):12–16, jan.-feb. 2011.