Keep it Real: Best Practices in Virtual Reality Interface Design

Background

Context

Emerging technologies like virtual reality (VR) offer new and exciting challenges to usability professionals. That's one reason I selected VR to serve as the focus of my thesis while completing my Masters of Information Science at the University of North Carolina at Chapel Hill’s School of Information and Library Science. It was a large undertaking, but I will attempt to outline the project here.

Goals

The goal of the paper was not only to conduct a usability study, but also to discuss the practical application of its findings and establish actionable recommendations in the hope that designers and product developers may use the information collected as a guide to creating good, usable VR products.

Research Question: What are the best practices for user-centered design in virtual reality gaming interfaces?

Purpose

So, why focus on games and gaming systems, when there are so many (seemingly more valuable) applications of virtual reality?

  1. VR gaming is the mainstreaming of virtual reality.
  2. VR games are raking in profits.
  3. The global market for VR gaming is growing at an astonishing rate.
  4. Games are a learning tool which can be applied to many different contexts.
  5. Also, yeah, they're super fun.

Study Design

Overview

The project was divided into three parts:

  1. Part I: Literature Review
  2. Part II: Heuristic Evaluations
  3. Part III: Usability Test

For the sake of space, I will be focusing on the usability testing portion of the study hereafter.

The Systems

Three distinct VR head-mounted display systems were evaluated to encompass a range of products on the market and their potentially varied usability issues. The study used a between-groups design, splitting participants into three groups, one for each VR system:

Game Selection

To immerse the users within virtual reality and engage them with the systems, it was necessary to select a game for them to interact with. I chose Space Pirate Trainer, an award-winning arcade-style VR action game that is compatible with all three systems.

Space Pirate Trainer Screenshot

Task Order

Participants were asked to complete game-related tasks within their assigned VR system. The tasks were presented in a static order. Often in usability studies, tasks are ordered randomly to reduce bias, with breaks in-between for questions. Randomization was not ideal for this study, however, due to the natural flow from task to task inherent in the gaming environment and the awkwardness that would be caused conducting "blind" interviews, pausing the game, and/or removing the headset between each task.

The Usability Test

Setting

The tests were conducted in the Manning Hall VR lab at UNC’s School of Information and Library Science. The lab is a private 10’ x 10’ room outfitted with two gaming computers and VR equipment including HTC Vive and an Oculus Rift. A Samsung Odyssey Windows Mixed Reality Headset was also brought in for the duration of the study.

Participants

Seventeen undergraduate and graduate students at the University of North Carolina at Chapel Hill participated in the study (two for the pilot test and five for each testing group). Their ages ranged from 19 to 32 years old. One-third reported previous experience with VR. All reported experience with video games, averaging in the mid- to low- range.

Data Collection

The test incorporated methods to collect qualitative and quantitative data:

This study was not meant as a product comparison, but rather as a means to highlight the features from each system that contribute to the best user experience. For this reason, data was analyzed for each discrete experimental group but findings will be discussed wholistically.

Findings

Pain-points:

Positive Findings:

Participants were most impressed with the systems’ immersive abilities. Two-thirds of participants reported thinking of the virtual environment as “somewhere they had visited” rather than images they had seen. Sensory feedback played a key role. “It makes you feel like your really there,” commented one participant, “You actually feel like you’re in that space.”

Limitations

  1. System Variance

  2. Design differences (especially among system controllers) may have biased the results, but I felt this a worthwhile exchange for the opportunity to uncover a greater range of usability issues.

  3. Single Game Selection

  4. Limiting the study to a single virtual experience also limits the generalizability of its findings.

  5. User Diversity

  6. Ideally, the study would include a larger and more diverse pool of participants (in terms of age, gender, experience, and ability).

  7. Artificial Testing Environment

  8. The perfect researcher would be gifted with telepathy and invisibility. Then again, that study would never be approved by the IRB.

  9. A Usability Team of One

  10. I served as facilitator, moderator, note-taker, recorder, observer, technician, troubleshooter, and data analyst for this project. If I learned one thing from the experience, it is the value of working on a team.

Conclusion

Best Practices

Drawing insights from the literature review, heuristic evaluations, and usability test, I derived Fifteen Best Practices for VR Interface Design:

  1. First impressions matter. Wow them.
  2. Controllers are hard to learn. Make them as simple and effective as possible.
  3. Focus on learnability. Allow time for users to adjust.
  4. Give instructions and guidance (in a format that users understand).
  5. Basic functions should be readily available.
  6. Always provide feedback. Engage all the senses.
  7. Use reality as a guide (but don’t take things too literally).
  8. Use text, but make sure it’s readable. Minimize lingo.
  9. Keep it simple… but avoid flat design.
  10. Give users complete control.
  11. Be wary of cybersickness and emotional distress.
  12. Pay attention to comfort and accessibility.
  13. Design for reality blindness. Always keep safety in mind.
  14. Spend time getting to know VR for yourself.
  15. Conduct (iterative) usability testing.

Read more in the full paper:

Tabitha Frahm. A Usability Study of Virtual Reality Systems: On Best Practices for User-Centered Design in Virtual Reality Gaming Interfaces. A Master's paper for the M.S. in I.S. degree. April 2018. 106 pages. Advisor: Chad M. Haefele.

Special thanks to my advisor, Chad Haefele, for his wisdom and support.