User interface design in video games

User interface design in games differs from other UI design because it involves an additional element — fiction. The fiction involves an avatar of the actual user, or player. The player becomes an invisible, but key element to the story, much like a narrator in a novel or film. This fiction can be directly linked to the UI, partly linked, or not at all. Historically games didn’t have any real link to the game’s narrative, most likely because early games rarely had strong story elements.

Erik Fagerholt and Magnus Lorentzon explored theories of game UI design in their thesis for Chalmers University of Technology titled: Beyond the HUD — User Interfaces for Increased Player Immersion in FPS Games. They introduce terms for different types of interfaces depending on how linked to the narrative and game geometry they are:

UI Map

Diegetic

Diegetic user interface elements exist within the game world (fiction and geometry) so the player and avatar can interact with them through visual, audible or haptic means. Well executed diegetic UI elements enhance the narrative experience for the player, providing a more immersive and integrated experience.

Metro 2033 uses a complete Diegetic UI with no HUD elements to help to support the game’s narrative. It runs the risk of frustrating the player though slow response times but this forms part of the game mechanic.

The character’s watch is used to measure how long the filter in the gas mask will last and how visible he is.

Many games get away with using Diegetic patterns because their narrative is set in the future, where UI overlays in daily life are commonly accepted. This is the case for the latest version of Syndicate. If the story was set in a different time period the UI elements would be probably be considered Spatial (explained in detail later in this post) instead of Diegetic.

Syndicate’s DART overlay mode highlights enemies and allows the character and player to see through cover

Assassin’s Creed manages to use a lot diegetic patterns even though it’s set in a historical world because the player of the player is using a virtual reality system in the future. So the story is in fact futuristic rather than historical.

Assassin’s Creed uses it’s eagle vision to highlight enemies and their patrol track. The player and the character see the same thing.

There are cases when diegetic UI elements aren’t appropriate, either because they aren’t legible in the geometry of the game world, or there’s a need to break the fiction in order to provide the player with more information than the character should or does know.

Meta

Sometimes UI elements don’t fit within the geometry of the game world. They can still maintain the game’s narrative but sit on the 2D hub plane — these are called Meta elements.

A common example of a Meta UI element is blood the splatters on the screen as a form of health bar, as in Call of Duty: Modern Warfare 2.

Blood splashing on the screen within the 2D HUD plane to tell the player that the character is losing health

Interacting with the phone in Grand Theft Auto 4 is an interesting example. It mimics the real world interaction — you hear the phone ringing and there is a delay before the character and player answer it. The actual UI element itself appears on the 2D hub plane though, so it’s actually a Meta element, though the start of the interaction is Diegetic.

The character is answering the phone but the actual UI element is placed within the 2D HUD plane that only the player sees

Meta UI elements can be difficult to define in games without a strong narrative element, such as sport or racing games. In Need for Speed: Hot Pursuit I still feel that the speedometer in the 2D HUD is meta because the player’s character — the driver, would know what speed the car is is traveling at and therefor it forms part of the narrative, as much as there is one. Other HUD elements such as track position or track location can be more difficult to define, some could be considered Meta because the drive would have that information while the rest are simply Non-Diegetic.

Need for Speed: Hot Pursuit uses HUD elements such as the speedometer to relay information about the car to the player

Spatial

Spatial UI elements are used when there’s a need to break the narrative in order to provide more information to the player than the character should be aware of. They still sit within the geometry of the game’s environment to help immerse the player and prevent them from having to break the experience by jumping to menu screens. The closer these follow the rules of the game’s fiction the more they can help immerse the player.

Splinter Cell Conviction also adopts Spatial elements in the form of projections that illustrate objectives within the game world. Their scale does seem to challenge the fiction slightly more than other examples.

Type is overlaid in to the environment to communicate messages to the player rather than the character

Fable 3 is another example where Spatial elements are used to provide more information to the player and prevent them from jumping to a map screen. The glowing trail almost fits within the fiction given it’s magic aesthetic quality but the character isn’t meant to be aware of it. It guides the player to the next objective.

Fable’s sparkling trail allows the player to guide the character in the right direction

Spatial elements can be beautiful pieces when they work with the geometry of the world. These Spatial elements from Forza 4 demonstrate a simple style can contrast the rich 3D qualities of the game.

Bold iconography combined with strong typographic layouts help establish a beautiful art direction for Forza 4’s UI

Non-diegetic

Then there are traditional non-diegetic elements, these elements have the freedom to be completely removed from the game’s fiction and geometry and can adopt their own visual treatment, though often influenced by the game’s art direction. I think these elements are best used when the diegetic, meta and spatial forms provide restrictions that break the seamlessness, consistency or legibly of the UI element.

World of Warcraft uses a mostly Non-diegetic UI, one exception being the Spatial player names. It allows the user to completely customise it, hopefully ensuring a familiar experience.

Most of the UI elements in World of Warcraft sit on the 2D hub plane, some elements sit within the world’s geometry such as the player names however the character isn’t aware of any of the UI

Finally, Mass Effect 3 uses many Non-diegetic UI elements in order to inform the player of the character’s selected weapon and power — among other things. Given it’s futuristic setting I can’t help to think if some of this information could have been integrated in to the game world, narrative, or even both.

Mass Effect 3’s Non-diegetic elements still inherit the visual style associated with the game world

Conclusion

Physical interaction methods and immersive technology such as VR headsets promise to challenge game UI design, allowing for a stronger connection between the avatar and character as both engage in similar actions at the same time. Technology provides an opportunity for deeper levels of interaction with the addition of audio and haptic elements. This will mean less use of non-diegetic UI.

Game UI has a key advantage (or disadvantage from some perspectives) in that players are often engaged with the narrative and/or game mechanic enough for them to learn new interaction patterns, or forgive bad ones. This is likely the reason so many games have bad UI, as testing needs to encompass the core game mechanic while UI is seen as secondary.

Recommended books



List Price: £72.42 GBP
New From: £59.95 GBP In Stock
Used from: £57.86 GBP In Stock



List Price: £37.99 GBP
New From: £37.79 GBP In Stock
Used from: £25.99 GBP In Stock


Designing Interactions (Hardcover)

By (author): Bill Moggridge

List Price: £32.95 GBP
New From: £18.78 GBP In Stock
Used from: £15.99 GBP In Stock



List Price: £18.99 GBP
New From: £10.79 GBP In Stock
Used from: £10.59 GBP In Stock

Further reading online

  1. Stefanos Papalampro explores diegesis and interface design in first person shooters.
  2. Marcus Andrews from DICE examines recent games using these theories from Fagerholt and Lorentzon in this article posted on Gamasutra.
  3. Michael Grattan, a senior at the University of Southern California writes a response to the article on his blog.
  4. Dave Russell has also written an article on his blog.
  5. Video: The UI designer for the game Brutal Legend talks about his work and shares his process.
  6. 13 principles of display design by Christopher Wickens et al. in their book An Introduction to Human Factors Engineering.

3D icon design by Nicolas Morand.

Share this page

Share on FacebookGoogle+Share on LinkedInShare on RedditTweet about this on Twitter

23 Responses to “User interface design in video games”

  1. Robert Kabwe

    Great roundup and thanks for the links.

    From your posts it sounds like you might be interested in exploring game design a bit further. No reason you can’t work with game designers on their UI or work on your own game while you do web design. Or push the boundaries of web design using javascript / html5 / video / sound etc. — as browsers and JS engines become more powerful the line between game canvas and web canvas will blur.

  2. anthony

    Thanks for the comment Robert, you have some interesting posts on your site I’m currently reading my way through.. Yes, it would be interesting to work on the UI of a game — I’m interested in all forms of interaction design and think as the lines between the mediums blur I may even get a chance at some point in the future! I don’t think I have any skills in actual game (mechanic) design so can’t see myself creating anything on my own though.

  3. Aneesh Karve

    It seems that the original meanings of “diegetic” and “mimetic” have been reversed. “Diesis” narrates, whereas “mimesis” demonstrates. Therefore, user interface elements that do not exist in the game world are diegetic, whereas those that do exist in the game world are mimetic. But diegetic / mimetic is more of a continuum than a dichotomy. Some games have diegetic elements that are rendered in the style of the game world, making those elements partially mimetic. StarCraft is one example. Does the HUD exist in the world of StarCraft or the player’s world? It’s somewhere in between, wherever the battle commander is.

    So the dimensions of the four-quadrant diagram should perhaps be diegetic/mimetic and in-perspective/out-of-perspective. The latter is independent of whether the game is 3D, since 2D “God’s eye” games, like Populous, have mimetic elements.

  4. J MCKellan

    @ Aneesh

    Actually if you read further down that wiki article, you can see how the terms are applied to cinema, which are different from those applied to theatre (as you describe). The approach to these terms in cinema has naturally transferred over to games, due to a lot of the same concepts being used (we have a camera, we can show the player where to look, we can edit the experience). Diegetic = in the game world, non-diegetic = outside the game world.

  5. Chance Bliss

    Let’s nerd on esoteric UI terminology in game design. Graham Ericksen will be the only one interested.

  6. Graham Ericksen

    And let’s just admit that game designers are a billion years ahead of the wack UX community made up of hacks like us.

  7. Chance Bliss

    I feel like a hack when I read stuff about ‘Diegetic’ vs. ‘Non-Diegetic’ interfaces. Web doesn’t even come close. Mobile is at least moving forward.

  8. Graham Ericksen

    That’s because anything you learned in school has absolutely nothing to do with your current job. Remember, when we were in college the interwebs didnt really exist.

  9. Warren Jerzyszek

    @Graham Ericksen it’s pronounced Die-a-get-ic.

    To the author, this is a really informative post, a really helpful insite on the ins and outs of UI design for games, I certainly had no idea that Diegetic, Spatial, Non-Diegetic and Meta were the technical names for the elements that form game UIs.

    Are we likely to see more articles of this nature any time soon?

    Best,
    WJ

  10. Anthony Stonehouse

    Thanks Warren,

    This has been the most popular article on my website since it was published a few years ago. I’ve made improvements to it’s structure and examples over the last few months to keep it fresh. I’ve also written a few more articles on UI and video game design, but nothing specific to diegetic, spacial and meta. If you’ve got a suggested topic please let me know and I can certainly investigate!

  11. Warren

    No problem, It’s a great read.

    It would be awesome to see an article focusing on your process of creating a UI from start to finish, your preferred workflow — it would also be awesome to see how you actually visualise your UI ideas, I find that my sketches are always a bit sloppy that only I can understand, so any advice/articles on that type of thing would also be very helpful. To be honest, there’s not a lot out there on those subjects or the subject you covered in this article, so it would be good to see more of this type of content.

    Best,
    WJ

  12. Anthony Stonehouse

    Thanks for the suggestions Warren, I can certainly explore those topics in more detail in the future as they are of particular interest to me. My publishing schedule is often sporadic and inspired by events or books so never know when the next one will be; I’ve recently had a son so there may be a slightly longer delay! Earlier this year I published a post covering my design process, if you’re interested you can read it here.

  13. Warren

    Congrats!

    Your design process is good, nice breakdown of steps, I assume these apply to other/all areas you design for?

    I really want to get to a point where I am a respected master of Graphics & GUI Design, but as I’m so new to the GUI side of things I feel this will always be a negative in any interview situation. Can you recommend a direction/avenue to explore, further reading or a new way of looking at and breaking down games that will increase my exposure and knowledge of GUI Design. Short of putting together comps UI’s for games include iOS etc I’m not sure what else to explore as the subject feels a little niche and there’s not much out there that I can focus on learning, in a linear black and white fashion. As the subject is so new to me I would massively appreciate your advice and guidance no matter how little or detailed.

    Best,
    WJ

  14. Maxwell Hansen

    The inverse of diegetic is not called “non-diegetic”, but rather “mimetic”.

  15. Anthony Stonehouse

    Thanks for the update Maxwell. You are right, I’m not familiar with that variation for the term and when I wrote this post a few years ago all the university papers just used the word non-diegetic. I’ll leave it as I wrote it for now, but good to know there’s another, perhaps more descriptive word.

  16. Stephen

    The cracks on the screen in metro 2033 shouldn’t be labeled as meta. The cracks only appear while the player has a gas mask equipped, they are meant to show the player that their mask is nearly broken and needs to be replaced. Since the game character is wearing the mask it seems like they too would be aware of its broken state.

  17. Anthony Stonehouse

    Thanks Stephen, that example was a recent addition and was hoping to move away from the ‘blood on screen’ effect. I haven’t played Metro 2033 myself so wasn’t aware that the cracks exist in the game world rather than on the HUD. I have re-created the UI elements map and used it as the feature image. This time using blood on the screen to represent health for the meta example.

  18. shay casey

    This is a great article. Thanks for sharing.

    I designed the UI for the most recent Tomb Raider and my whole mission was to prove that an augmented reality UI could be successful without being in a sci-fi narrative. I started with two assumptions. One, UI is a layer of abstraction outside the game world and two, we could say contextually that what the player sees is what Lara Croft was thinking. It was like a view into her mind and thought process, ie. I see a fire and I could light my torch with it. Or, I should upgrade my weapon what parts do I have available.

    We made it a point to have as little Meta UI as possible because it really does help immerse the player into the narrative and the game world. Also, I was a huge fan of Dead Space and Dirt for their use of 3D UI.

    Advance to the one minute mark
    https://www.youtube.com/watch?v=20SFuGWlEEw

  19. Anthony Stonehouse

    Thanks Shay, that’s a really interesting point and congratulations on designing such a lovely interface. I was also a fan of the original Dead Space and Dirt UI and partly what inspired me to write this post.

    You’re right; most games that use diegetic interfaces rely on a sci-fi element to ensure they fit the narrative, such as Assassin’s Creed, Dead Space and Syndicate. When the elements can’t sit within the narrative they become spatial and tend to break the immersive experience a little, though not as much as non-diegetic UI.

    Adopting diegetic UI in a non sci-fi narrative is an interesting challenge. 3D menus are some-what difficult to categorise, they would normally be meta but if they sit in the 3D world and the character is aware of them they become diegetic. It’s just making the connection between the UI and the character believable through narrative that can prove challenging! The best way to achieve that is through some sort of sensory experience as you’ve created for Tomb Raider’s UI.

×

Comments are closed.