Tuesday, 22 January 2013
We are thrilled that our paper on exploring user testing and Biometric Storyboards in game development has been accepted at CHI 2013. This post provides a brief overview of the paper.
Few methods are available in Games User Research (GUR) to formally test how game designs work for players. In particular, the usefulness of user tests and physiological measurements for providing formative feedback for game designers on their titles under development has not been fully studied.
In this paper, we investigate the differences in game design between games developed using different user evaluation approach. In our study, we evaluated three versions of a game developed under three conditions: first, we observed players performing gameplay actions, then conducted interviews and questionnaires on what they liked and didn’t like – a classic user test. Second, we used a physiological-based evaluation approach called Biometric Storyboards that combines designer intentions, user experience evaluations and physiological player reactions in a single user evaluation report. Finally, in the third condition (serving as control) we developed a game without user evaluation, only based on designers’ expertise.
Our results support the games user research mindset that user tests improve a game considerably. We show that using either classic user tests or physiological-based user tests leads to games that are better designed and rate more positively than games designed without any form of user testing. In addition, our results support the notion that using physiological-based user tests affords more nuanced changes, scoring highest on the evaluation factors fun, visuals, and gameplay quality.
The purpose of GUR, is to gain insights from players to help game developers optimise their designs. Since GUR is an emerging field, there is a lack of data on how well the use of HCI methods and other techniques assist in this iterative design process.The quality of game design ultimately depends on the developer’s creative vision. It is difficult to evaluate the contribution of GUR to the final quality of a video game. Our paper is the first at CHI to show the impact of this contribution from start to finish. Our paper provides a relevant contribution to HCI by demonstrating how different GUR techniques affect the understanding of users as well as the design process itself. We intend to move this discussion forward by addressing how the different techniques inform designers and consequently each make an impact on the design process.
Email me for a sneak preview of the paper 😉
We thank our partners, participants, external designers (Joel, Mina, Brodie, James, Chris and Kei) for their study contributions. We also thank our colleagues at Sussex University’s HCT group (Ben, Gareth, Ellie, Kate, Jim, Ewen, Eric, Lesley and Judith) and UOIT’s GAMER Lab (Andrew, Bill, Mirza, Saad, Matthias and Dan) for their feedback on early drafts of this paper. We are also thankful to our colleagues in game industry (Veronica, Graham, Seb, Mirweis and Steve) for the valuable discussion.
This research was supported by University of Sussex, UOIT, NSERC, GRAND NCE, and SSHRC IMMERSe.
Now that you have made it to the end of this post, we have a bonus video for you. Special thanks to Mirza Beig for making our videos.