Sunday, 30 December 2012
This post is written by Ellie as a write up summary of our HCT weekly meeting where I talked about my work at UOIT over the summer 2012, mostly the on-going work with Biometric Storyboards for my thesis. The Sussex HCT blog is available HERE.
This week the lab meeting featured Pejman talking about his work at UOIT over the summer, and his on-going work with biometric storyboards for his thesis. Sadly we had apologies from Leslie, Eric and Judith today, but we were joined by Nick Collins.
Pejman started by introducing us to some of the work that he did over the summer. The group he was working with are part of the GRAND research network in Canada, so he was immediately involved in submitting to and presenting at the GRAND 2012 conference in Montreal. From the he went to CHI2012 in Austin, where he helped to run the 2-day workshop on Games User Research (GUR), [we are also involve in organising the GUR workshop in CHI2013] which was apparently one of the largest run this year.
Along with this he was also involved with some of the projects at UOIT Game Science Group . Apparently they have a system where postgrads have to pitch exciting projects to the undergrads and get them to work on them. He showed us two extremely polished videos produced by the undergraduates he was working with, detailing a couple of games produced. They looked seriously impressive, with one group integrating a USB bicycle into a first-person shooter, and the other group using brainwave sensors to control their gaming environment. Pejman says these sorts of innovative controllers and the way that they change the experience for the players and models of joint play is something he would like to return to, and that maybe we should look to bring something like this kind of event to Sussex.
(He also says that he did get to do a little travelling, and that Canada is beautiful. I think I need to book flights.)
Of course, he was also working on his PhD thesis at the same time as all of this. His great motivation when he started was to help to understand users, with the overriding aim to make better games. To further this noble goal, he started out by using biometrics to get feedback on games in development. He uses the biometrics because they allow for a more continuous gameplay experience than surveys, and capture more data than an interview at the end of play (when players have already forgotten many incidents). Rather than try to interpret the emotion of the player from the feedback (which is tricky) he uses it to guide his post-play interview, incorporating video of the game to help jog the memory of the participant.
Using this method he has demonstrated that in conjunction with other, more conventional HCI methods it enables a greater range of problems to be diagnosed. However, his main problem has been how best to report this extra data to the developers without overwhelming them with the masses of data he gathers, and help to highlight the actual problem areas. His ‘biometric storyboards’ are the result – a kind of composite feedback graph of the game broken down by game areas/events called ‘game beats’, showing positive and negative emotion.
One of his main challenges over the summer was to design and implement an experiment that showed the effectiveness of this kind of feedback. He used a platform game, and ran a user study using it. He then prepared two reports: a biometric storyboard, and a classic usability report. He then asked 6 games designers to recommend game changes, based on the usability report, the biometric storyboard and with no user feedback at all. The game was then developed into three branches using the recommendations, and these were re-tested. He has some interesting results, not just from the testing of the new versions of the game but from observing the way the designers interacted with the data they were given. (I don’t want to give away his findings – he has papers in the works!) [The paper is accepted at CHI2013 🙂 ]
Ellie asked (first, to let Ben off for once!) whether this focus on how to report back to the developers was new. Pejman could name a book chapter on how to feedback on websites and other user experience, but not games specifically.
Ben had a concern about some of the sensors that Pejman’s using, but also that he seemed to be highlighting negative moments that may actually be part of the overall high at a later point. Pejman’s answer to this was actually extremely interesting. He says he isn’t putting any kind of valence on the feedback he gives the designers, he is just trying to show them the user experience in a way that allows them to determine if it’s as they expect. So frustration may be fine, if that is what the designer was trying to achieve at that point.
Edgar asked if some game-types are better represented using the biometric storyboards than others? Pejman felt they might not be ideal for puzzle games, but had worked for a wide range of other kinds of games.
Katy wanted to know if Pejman was happy with the final iteration of the storyboard, given that it was much more complex than the early versions. Pejman has tests with designers on the first three iterations, and has shown that they do prefer the last of the three. He has yet to test the very final version (which is generated using a tool rather than hand-produced), but overall he is happy.
Next week we continue on with the game theme, with Ellie talking about the workshop she went to as part of Fun and Games on player experience in Video games. As always, Tuesday, 11-12, Interact Lab. We’ll have to see if she can match Pejman’s cake…