User interface design in video games
User interface design in games differs from other UI design because it involves an additional element — fiction. The fiction involves an avatar of the actual user, or player. The player becomes an invisible, but key element to the story, much like a narrator in a novel or film. This fiction can be directly linked to the UI, partly linked, or not at all. Historically games didn’t have any real link to the game’s narrative, most likely because early games rarely had strong story elements.
Erik Fagerholt and Magnus Lorentzon explored theories of game UI design in their thesis for Chalmers University of Technology titled: Beyond the HUD — User Interfaces for Increased Player Immersion in FPS Games. They introduce terms for different types of interfaces depending on how linked to the narrative and game geometry they are:
Diegetic user interface elements exist within the game world (fiction and geometry) so the player and avatar can interact with them through visual, audible or haptic means. Well executed diegetic UI elements enhance the narrative experience for the player, providing a more immersive and integrated experience.
Metro 2033 uses a complete Diegetic UI with no HUD elements to help to support the game’s narrative. It runs the risk of frustrating the player though slow response times but this forms part of the game mechanic.
Many games get away with using Diegetic patterns because their narrative is set in the future, where UI overlays in daily life are commonly accepted. This is the case for the latest version of Syndicate. If the story was set in a different time period the UI elements would be probably be considered Spatial (explained in detail later in this post) instead of Diegetic.
Assassin’s Creed manages to use a lot diegetic patterns even though it’s set in a historical world because the player of the player is using a virtual reality system in the future. So the story is in fact futuristic rather than historical.
There are cases when diegetic UI elements aren’t appropriate, either because they aren’t legible in the geometry of the game world, or there’s a need to break the fiction in order to provide the player with more information than the character should or does know.
Sometimes UI elements don’t fit within the geometry of the game world. They can still maintain the game’s narrative but sit on the 2D hub plane — these are called Meta elements.
A common example of a Meta UI element is blood the splatters on the screen as a form of health bar, as in Call of Duty: Modern Warfare 2.
Interacting with the phone in Grand Theft Auto 4 is an interesting example. It mimics the real world interaction — you hear the phone ringing and there is a delay before the character and player answer it. The actual UI element itself appears on the 2D hub plane though, so it’s actually a Meta element, though the start of the interaction is Diegetic.
Meta UI elements can be difficult to define in games without a strong narrative element, such as sport or racing games. In Need for Speed: Hot Pursuit I still feel that the speedometer in the 2D HUD is meta because the player’s character — the driver, would know what speed the car is is traveling at and therefor it forms part of the narrative, as much as there is one. Other HUD elements such as track position or track location can be more difficult to define, some could be considered Meta because the drive would have that information while the rest are simply Non-Diegetic.
Spatial UI elements are used when there’s a need to break the narrative in order to provide more information to the player than the character should be aware of. They still sit within the geometry of the game’s environment to help immerse the player and prevent them from having to break the experience by jumping to menu screens. The closer these follow the rules of the game’s fiction the more they can help immerse the player.
Splinter Cell Conviction also adopts Spatial elements in the form of projections that illustrate objectives within the game world. Their scale does seem to challenge the fiction slightly more than other examples.
Fable 3 is another example where Spatial elements are used to provide more information to the player and prevent them from jumping to a map screen. The glowing trail almost fits within the fiction given it’s magic aesthetic quality but the character isn’t meant to be aware of it. It guides the player to the next objective.
Spatial elements can be beautiful pieces when they work with the geometry of the world. These Spatial elements from Forza 4 demonstrate a simple style can contrast the rich 3D qualities of the game.
Then there are traditional non-diegetic elements, these elements have the freedom to be completely removed from the game’s fiction and geometry and can adopt their own visual treatment, though often influenced by the game’s art direction. I think these elements are best used when the diegetic, meta and spatial forms provide restrictions that break the seamlessness, consistency or legibly of the UI element.
World of Warcraft uses a mostly Non-diegetic UI, one exception being the Spatial player names. It allows the user to completely customise it, hopefully ensuring a familiar experience.
Finally, Mass Effect 3 uses many Non-diegetic UI elements in order to inform the player of the character’s selected weapon and power — among other things. Given it’s futuristic setting I can’t help to think if some of this information could have been integrated in to the game world, narrative, or even both.
Physical interaction methods and immersive technology such as VR headsets promise to challenge game UI design, allowing for a stronger connection between the avatar and character as both engage in similar actions at the same time. Technology provides an opportunity for deeper levels of interaction with the addition of audio and haptic elements. This will mean less use of non-diegetic UI.
Game UI has a key advantage (or disadvantage from some perspectives) in that players are often engaged with the narrative and/or game mechanic enough for them to learn new interaction patterns, or forgive bad ones. This is likely the reason so many games have bad UI, as testing needs to encompass the core game mechanic while UI is seen as secondary.
Further reading online
- Stefanos Papalampro explores diegesis and interface design in first person shooters.
- Marcus Andrews from DICE examines recent games using these theories from Fagerholt and Lorentzon in this article posted on Gamasutra.
- Michael Grattan, a senior at the University of Southern California writes a response to the article on his blog.
- Dave Russell has also written an article on his blog.
- Video: The UI designer for the game Brutal Legend talks about his work and shares his process.
- 13 principles of display design by Christopher Wickens et al. in their book An Introduction to Human Factors Engineering.
3D icon design by Nicolas Morand.