Designing User Interfaces for Games

Zhan YeHuman-Computer Interaction Institute

Carnegie Mellon UniversityDecember, 2000

Introduction

We have seen a lot of articles about artificial intelligence (AI) programming, path-findingalgorithms, and 3D modeling in various books and magazines. Given the fact that thegame market is so competitive, almost every aspect of game design and development hasbeen studied carefully to find better ways to create better games. But it is sad to see thatone important aspect of game design has been long neglected - user interface (UI).Nobody has ever mentioned the role and importance of UI to a game. Nor have we seenany research on how to design a better UI for games. It is interesting to notice that othersoftware industries have all invested a lot of time and effort in finding new methods andprocesses to design and evaluate UI. Just look at those online job sites, you will find a lotof job openings for UI designers, interaction designers, usability specialists and userexperience designers. But in the game industry, UI designers still dont have animportant role; in fact in most teams they dont even exist. Even in big companies thathave realized the problem, such as Microsoft, their attempt to integrate UI design into thewhole game design and development process is still in its early stage and hasnt shownany promising results yet. Game designers still ask fundamental questions like Whydoes user interface matter? In the first part of this article, I will try to answer thisquestion. In the second part, I will briefly describe the differences between the UIs of PCand console games. In the third part, I will give some ideas about why and how to usehuman-computer interaction (HCI) methodologies to improve UI design for games andhow to integrate these methods into the game design and development process. In thefinal part of the article, I will give a detailed example using The Age of Empires II fromMicrosoft to show how HCI methods can be used to help game designers locate designproblems and create innovative designs.

Why User Interface Matters

User Interface: What Gamers Actually See and Interact With

There are thousands of games coming out everyyear, which can be classified into 5 to 7 differentgenres. But no matter what genre games belong to,whether they are action games or RPG games,strategy games or sports games, they all share thesame structure. From a system architects point ofview, a game is a three-layered system. Figure 1shows this layered system. For purpose of thispaper, we call the inner layer the Game Core, and itrepresents the mechanics of the game. It defines therules of a game, for example, how high the charactercan jump, how many enemies he will encounter inthis level and where they are located. The GameCore is designed by game designers who have a

perfect mental model of the game. Gamers, however, can not access this layer directly,and therefore they can never get a complete mental model of the internal mechanics of agame. If they do, playing the game would be no fun at all. To keep them away from theGame Core, game designers wrap it with another layer, which is the UI layer. The onlyway gamers can get knowledge of the Game Core is through the UI layer. It includesboth hardware elements (mouse, keyboard, controller) and software elements (menus,buttons, icons.) The UI layer is a medium between the Game Core and the gamers.Gamers have complete access to the UI layer and they have to be knowledgeable aboutthis layer to successfully play the game. The outermost layer is the Gamers layer. Weinclude gamers in this model because good UI is not a function of the software, but theproperty of the user. Without gamers, the system is not complete and cannot beevaluated.

User Interface Defines Game Play

Most game designers dont realize the fact that user interface can define game play. It isunderstood that game play is the most important property of a game. No matter howbeautiful the 3D images are, or how involving the story is, without good game play agame definitely cannot succeed. But game play is a rather vague concept and hard todescribe. Software engineering principles tell us that we cannot improve a property of asoftware product unless we can define and measure it. So it is very important to find away to define game play. One possible way is to define it is by the way gamers interactwith games is, through the UI and the interactions associated with it. By doing that,we will ablecan be desc

User Inte

Figure1. Game as a layered system.

GameCore

User Interface

Gamers; that

to analyze game play, not as an abstract concept, but something concrete that

ribed, measured, and studied.

rface Makes Games More Challenging

For RPG games, like the Final Fantasy series, you dont want gamers to see the wholeworld map when they start the game. Instead, you want them to explore the world pieceby piece. The fun of playing games largely depends on the exploration and theexcitement and uncertainty it brings. For the same reason, we dont want gamers toknow the internal mechanics of a game when they start playing the game. We want themto gradually discover how the game tricks them and what kind of strategies they shoulduse. Through trial and error, they will build an increasingly complicated mental model ofthe game and feel a sense of accomplishment because their knowledge and skills areimproving over time. So UI doesnt only help gamers to play games successfully, butalso controls the pace at which the internal mechanisms will be revealed to gamers.Eventually the internal mechanics will be revealed, and gamers will be able to finish thegame successfully.

User Interfaces of PC and Console Games

Differences between the User Interfaces of PC and Console Games

There are many differences between the UIs ofPC and console games. The most importantdifference, however, is that the games usedifferent input/output (I/O) devices. PC gamesheavily rely on the combination of keyboardand mouse. Whereas console games havecontrollers, which are more refined, althoughlimited, input devices. The form factors of thehardware, especially the I/O devices, greatlyaffect the design of UI. From a gamers pointof view, the UIs of PC games are moreobvious than those of console games. Mostgamers realize that they are interacting withsome kinds of user interfaces when they areplaying PC games such as The Age of Empiresand C&C. Just look at the familiar icons,menus, and windows. But gamers playingconsole games sometimes dont realize thatthey are interacting with the UIs. Can you aska gamer what kind of UI Super Mario 64 has?UIs of console games are hidden deeply in thecareful calculation of the usage of controllers,which is not obvious from the screen. We cancall the UIs of PC games software-orientedinterfaces, which means they utilize lots ofconventional GUI elements to representactions. The UIs of console games, on theother hand, are hardware-oriented interfaces,which means they are designed around theform factors of controllers and use relativelyfewer GUI elements.

Why Transported Games Rarely Succeed

The reason why most transported games fail is very simple. The original game succeedsbecause of its game play, which largely relies on the UI design. In order to transport agame to another platform, game designers are facing totally new I/O devices. They haveto redesign the UI since it is the form factors of the I/O devices that determine what kindof UI will work. By doing this, the original, successful, proven game play will beaffected. There is no guarantee that the new game play will be as good as the originalone since they are inherently different. Transporting a game to a different platform

Figure 2. Console/Arcade Games like theStreet Fighter series use minimum GUIelements. Their UIs are designed aroundspecific hardware the controllers.

Figure 3. We can find a lot of conventionalGUI elements (icons, buttons, sliders, statusbars, tabs) in the UIs of PC games like theCivilization series. Some games, such as theDOS version of Transport Tycoon, evencreated their own windows systems.

should be considered one of the major challenges thatthe UI designers face. The amount of work requiredwould equal, or even surpass, designing a UI for anew game.

Figure 4. To solve the transportingproblem, The Playstation version ofTransport Tycoon even included amouse with the software. But eventhat didnt help much and thePlaystation version still didnt do aswell as the PC version.

Why Use HCI Methods to Guide UI Design

Cannot Rely Solely on Our Intuition

In the current game industry, most game designs are guided only by designers intuition.Game designers are seasoned gamers; they have a feeling for what works in games andbase their designs on this. There may be a few geniuses in the world who can do thingsright by just using intuition, but for most of us, we cannot rely on our intuition alone.The quality of games can certainly not be guaranteed this way. We need formal methodsand processes to guide, evaluate, and help us improve our designs. Sooner or later, gamedesigners will realize the limitations of their intuition and the ad hoc solutions they areusing now, and admit the necessity of incorporating formal methods and processes intogame design and development.

HCI Is the Closet Research Field to Game Design

Game designers need formal methods and processes in order to create high-qualitygames. But where can they get them? Game designers dont have enough time toresearch and design methods and processes from scratch. What they can do, however, isto look at related research fields and other software industries and borrow ideas fromthem. HCI is the closest research field to game design. It is the study of how people usecomputers, and how to design, implement and evaluate computer systems so that can beused easily, effectively, and enjoyably. UI is one of the major research areas of HCI.Researchers have developed a number of methods and processes over the past 10 years,some of which have been successfully used in other software industries and proven to bevery useful tools for UI designers. Among these methods, heuristic evaluation (HE),cognitive walkthrough (CW), and Fitts law are especially valuable and can be adaptedfor use by game UI designers.

User Experience and Game Play

I am curious about why UI has been long ignored by most game designers. It isinteresting to compare the game industry with other software industries. It wasnt untilseveral years ago that some software companies began to realize the importance of UI.Since then, user experience, as a newly invented "magic word", has been talked a lot.User experience is the high level emotional impact on the users by the visual elements,information architectures, navigation, and interactions of a UI. Delivering an enjoyableexperience to users while they are using software is the ultimate goal of HCI research andapplications. Now many software companies have established their user experiencegroups and started using formal methods and processes to achieve this goal. If we look atthe game industry, we will be surprised to see that it is way ahead of other softwareindustries in realizing the importance of the general user experience. The game industryeven invented a unique name for user experience for games, which is game play, from itsvery first day since the early '80s. The game play in a game is comparable to the userexperience to a Web site or business software. The game industry coined the magic word

almost 10 years before other software industries. But the game industry just stoppedthere, without any conscious attempt to find out formal methods and processes to helpgame designers to create good game play. Game designers still highly rely on theirintuition to do their work, whereas Web designers have a number of well-developedmethods and tools to use.

How to Apply HCI Methods to UI Design

Get Average Gamers Involved

Most UI designers know the HCI mantra, You are not like the user. What it says is thatnobody, especially domain experts or usability experts, can replace the role of users inuser tests. Designers cannot evaluate their own designs effectively since they are notactual end users. For example, in the business software industry, developers usuallydont use the software they develop. Before they start a project, they have no indicationtowards what the software will be used for. It makes sense to hire real users to do userresearch and user tests. But in the game industry, the situation is quite different. Mostgame designers are avid gamers themselves; they know a lot about games. Therefore, itseems as if there is no need to get average gamers involved in the game design process.But this is wrong! We still need average gamers. There are two reasons for this: first,game designers are too skillful. Their knowledge about games is way beyond that ofaverage gamers. Ironically, this will prevent them from finding obvious design problems.Things that they think are too easy may be too tricky for average gamers. We must bearthis in mind expert gamers only represent a small fraction of the whole gamerpopulation and we dont want to design games only for them. Second, game designersknow too much about the design. Their mental model of the game is almost perfectbecause they designed it. Average gamers can never achieve this. They have their ownsimple, incomplete, sometimes incorrect, mental model. The mental model determinesthe user behavior. This means that the user behavior of game designers will be differentfrom that of average gamers. Due to these two reasons, game designers, though they arevery knowledgeable about games, cannot replace average gamers in the game designprocess. We need average gamers to evaluate and test our designs. The results we getfrom these evaluations and tests will prove to be unbiased and invaluable.

Adapting HCI Methods for Game UI Design

Weve mentioned that there are a number of HCI methods that have been proven to beuseful and can be adapted for game UI design. The following three are particularlysuitable for game UI design.

Cognitive Walkthrough (CW): CW is an inspection method that focuses onevaluating a UI design for ease of learning by exploration. Its focus is motivated bythe observation that many users prefer to learn software by exploration. This is oftenthe case for games. When gamers start playing a game, they generally know nothingabout it. They learn how to play and use the interface by trial and error. When gamedesigners design a game, they should be aware of this, and should design a game thatcan be successfully played by exploration. CW can be used to quickly go through thegame design and determine whether or not gamers can do it. Furthermore, thismethod doesnt require a matured prototype since it is a mental process.

Think-Aloud Protocol: This method can be used to observe gamers playing anexisting game. Essentially, it requires that UI designers ask gamers to think aloudas they play games. It can help us to know whats in gamers minds, the strategiesthey use, their expectations, frustrations, and confusions. Some people considerthink-aloud protocol may be the single most valuable usability engineering methodbecause designers now can discover things inside gamers mind.

Fitts Law: In addition to these two methods, another powerful tool that can behelpful is the Fitts law. It is a very important theory that, once applied, can greatlyimprove the efficiency of a UI design. It states that the time required to move thecursor to a target size S which lies a distance D away is given by T=Im Log 2 (D/S +0.5). The longer the distance, the smaller the target size, and thus more time will beneeded to click the target, as the consequence, the user will feel more stress. Thegeneral design goal, then, is to reduce the distance between two controls and enlargethe size of the controls under given constraints. Designer should also avoid deviceswapping and disturbing the focus of attention. Fitts Law is extremely valuable notjust because we can now evaluate two alternative designs quantitatively, but alsobecause it provides a guideline and perspective for us when we think about design.

Problems of Rapid Prototyping and Iterative Design

A widely used design approach in other software industries is the combination use ofrapid prototyping and iterative design. Designers can use HTML, Director, or VisualBasic to build a prototype, only implementing the front-end interface without worryingabout the back-end functionality, then they test the prototype with some users. Aftergetting useful feedback, they make adjustments to the initial design and enter anotherround of iteration. This approach has been proven to be very successful in other softwareindustries. But it may not work that well for games. Games need precise controls andtiming, whereas business software does not. You dont need to worry about responsetime when you build a prototype for business software. But you cannot ignore it whenbuilding a prototype for a game. A simple prototype cannot faithfully represent the realgame play that the final game will have. A complicated prototype may be too expensiveto build. Comparing to business software, it is also much harder to separate UI withback-end functionality of a game. So far there is no easy solution for this problem.Perhaps game designers need some kinds of tools like Director, but with more interactionfeatures, that they can use to build prototypes.

UI Design Guidelines

User Interface Shouldnt Be Distracting

One common mistake that we have seen in a lot of games, even successful ones, is thatthe UI is so beautiful, so overly decorated that it becomes a distraction. UI should haveaesthetic taste, but more importantly, it should be simple, efficient, easy to use, andconsistent with the whole game environment.

User Interface Should Provide Enough Visual Affordance

Many studies show that the majority of end users are very reluctant to read user manuals.They prefer to learn how to use software through exploration. This is also true forgamers. Never expect gamers read manuals, even for complicated strategy games likeAOE! Design a UI with good affordance instead! By affordance we mean form affordscertain actions and makes that apparent. A good UI should use its visual appearance tosuggest its functions. Gamers should be able to understand and act easily by just lookingat the UI.

Figure 5. The UI of Iron Helix is a hyper-realistic and highly literalrecreation of a real-world console. Notice the details such as thedamage and corrosion. But the question is, Do gamers reallywant to look at the UI, or they want to immerse into the beautifulouter space? The UI occupies almost screen space and leaveslittle room to display the virtual world the gamers really interestedin. This is one extreme example of how UI can be distracting andoverly weighted.

User Interface Should Be Balanced

The visual elements (buttons, labels) in a UI should be properly arranged, sized, andaligned to reinforce logical relationships among them and ensure the UI is stable on thescreen. Most game UI designers pay little attention to these details, and they just putcontrols and labels everywhere possible.

Figure 6. AOEs building buttons use the actual images of the buildings. They dont provideenough affordance. They are visually complicated and look similar to each other (especiallythe first three in the first row and the last three in the second row.) They fail to provide acognitive bridge between the buildings themselves (the images of the buildings) and thefunctionality of the buildings (what kind of products they produce.) Gamers, even experiencedones, cannot easily figure out which building is for what purpose and will hesitate when theyneed to choose among them.

Figure 7. In AOE II, the building buttons are greatly improved. They are elegantly simplifiedicons. Although simplified, they represent their functions clearly. Because instead of usingthe actual images of the buildings, UI designers focused on the functionality of the buildings,that is, the products the buildings produce. Gamers will have no problem choosing the onethey want among them. The new design is more efficient since it can lower the cognitive loadof the gamers when they play games.

User Interface Should Be Transparent

A good UI should be transparent. Gamers should be able to forget about the menus,buttons, icons, and windows and immerse into the virtual world that the game creates. Inthat sense, UI designers job is to design UIs that are transparent UIs that can be usedso naturally by the gamers so that nobody would even notice its existence. I stronglybelieve that it is the ultimate goal of UI design.

Figure 8. The display panel in AOE II is unbalanced, although the labels and icons are aligned. Itis overly weighted in the left side and leaves plenty of empty space in the lower right-hand corner.

Case Study:Age of Empires (AOE) II is a strategy game published by Microsoft. It is well receivedand praised by both gamers and game designers because of its innovative graphics andinvolving game play. But even a successful game like AOE II still has its problems, andusing it as an example will enable us to realize that there are still many areas that can beimproved.

In order to evaluate AOE IIs UI, the first step, we need to find some tasks that arefrequently performed by gamers. Each task is an action sequence that is performed stepby step by the gamer to achieve a specific goal.

The Action Sequence of Constructing a Building

One of the most frequently used tasks in AOE II is constructing buildings. This actionsequence includes 5 major steps, selecting a villager, moving to the building panel,selecting a building type, moving to the intended location, and clicking to drop thebuilding.

Figure 9. The action sequence of constructing a building. Red lines with arrows are used to show thepath of the cursor movement.

Before we analyze the actionsequence, we need to makesome assumptions about theinitial positions of thevillager and the cursor.Lets assume that we have avillager standing in thecenter of the screen, and theinitial cursor position is alsoat the center of the screen.

Step 5: Then the gamermoves the cursor back to thecenter of the screen todrop the building.

Step 1: The gamer clicks thevillager. The building panelwill show a number ofavailable actions.

Action sequence is done.

Step 2: The gamer moves thecursor down to the panel.

Step 3: The gamer choosesone building

Step 4: The gamer moves thecursor to the edge of thescreen to scroll the map.

Pop-up Menu Can Improve the Efficiency of the UI

The above action sequence, although it looks pretty complicated, is just a sequence ofmouse movements and clicks. You move the mouse to place A, click, then move it toplace B, then click, and so on. The time the gamer spend on moving the mouse is a lotlonger than that of a simple click. According to Fitts Law, the longer the movingdistance is, the longer the action sequence takes, the more muscle fatigue gamers willfeel. Cursor movement itself doesnt produce any direct results. The most obvious wayto reduce the moving distance is to use a pop-up menu. As Figure 10 shows, when thegamer clicks a villager, a menu will pop up near the villager, the gamer can easily chooseone building from the pop-up menu. Comparing the distances between the two designs,the new one may save less than 0.1 second every time. Sounds pretty trivial; but think

about it: gamers usually have to do the same action sequence hundreds of times whenthey play the game. The new design saves a lot of time, and much more importantly,relieves the muscle fatigue greatly. It is not only an efficient design, but also helps torelieve the pain and stress associated with computer usage to some extent.The new design (b) also saves a lot of screen space. The building panel occupies a huge amount of valuablescreen space, and it is empty most of the time. Changing it into a pop-up menu leaves more screen spacefor the world map. Research shows that the sense of immersion is very important to games and VRsystems. Having a large amount of screen space devoted to UI will reduce the sense of immersion, sinceUI is not an organic part of the virtual world of the game. UIs should occupy the minimum amount ofspace required to display the most important controls and information.

Two-Handed Interaction as a Hardware Solution to Map Scrolling

The new design we proposed early only solves the problem of the long mousemovements required to construct a building. From the analysis of the action sequence,we noticed that map scrolling also requires a lot of mouse movements. A gamer has tomove his mouse to one of the edges of the screen and stay there to scroll the map. Doesthis remind us of Web browsing? You move your mouse to the right edge of the screen

(a) (b)Figure 10. Comparing to the old design (a), the new design (b) greatly reduces the required mousemovement.

to click on the scrollbar to scroll the Web page. They all require long mouse movementto the edge of the screen. To solve this problem, Microsoft provided an innovativedesign, IntelliMouse, which is a mouse with a scroll wheel. It is a hardware solution thattotally eliminates the required movement. The D in Fitts law is reduced to zero, whichmeans the efficiency is greatly improved. Similarly, we can use a hardware solution forthe map scrolling in AOE II.

In the real world, people perform tasks withboth hands, because it is more efficient. Weoften use our non-dominant hand to holdobjects while using our dominant hand toperform precise operations on them. Most oftodays UIs still make use of only one pointingdevice that users manipulate with theirdominant hand. A lot of research, however,has shown that carefully designed two-handedinteraction can improve the efficiency of UIs.The most famous example is Xerox PARCsToolglass. I will not describe the details here,but basically they used two pointing devices,one mouse for the dominant hand, onetrackball for the non-dominant hand. The non-dominant hand controls the position of theToolglass using trackball because it requiresrelatively lower precision. The dominant handcontrols the cursor. We can adapt this designfor AOE II using a trackball to control thescrolling of the map and a mouse to control the

re, first, the required cursor movement is zero. It is problem. Second, by using two pointing devices, wee game. Now gamers can move the cursor and mape action sequence will be changed from beingot of time.

to analyze a UI and game play by studying its actione design guidelines and Fitts law to find problems in two-handed interaction shows that HCI research can

Figure 11. Microsoft IntelliMouse is anexcellent hardware solution to the deviceswapping/mouse movement problem.

Fscursor. The benefits of this design abasically a hardware solution to theintroduce parallel interaction into thconcurrently. The steps 2 to 5 in thserialized to parallel and it saves a l

Conclusion

The above case study explains how sequences. It also shows how to usthe action sequences. The proposed

igure 12. Trackball is ideal for mapcrolling because of its form factor.bring innovative ideas to game designs. Although we need time and real-world testing todetermine if the ideas innovated in the labs can succeed and eventually benefit us, it isalways important to have these innovative ideas first.