Turning Bad Feedback Into Good Feedback

26 November 2010

One of the biggest challenges designers face is interpreting feedback from users. I’m thinking of the rabid online communities around video games especially, but this applies to any product subjected to users’ brutal gaze before the product launches.

This article on addictive social games included a perfect example — even more telling that it was provided to show how “useless” feedback is:

McMillen said Super Meat Boy was designed without the aid of metrics. And while there was one focus testing session for the game, most of the feedback was thrown out. (One tester suggested that having a static loading screen would be preferable to a cutscene that couldn’t be skipped for the first few seconds because the level was loading in the background.)

The developer apparently disagreed with the tester’s solution and said “What a stupid idea, focus testing is worthless”. This is a very common type of feedback in game forums – bad solutions. Stuff like “Make my character overpowered” or “Throw out this level entirely”.  But solutions aren’t the goal of focus testing to begin with! What you’re looking for are problems.  Then you can have your experienced designers find the best solution to those problems. The trick here is converting the user’s bad solution into a problem, then working back to a better solution.

In the Super Meat Boy example, the user’s solution translates to a meaningful problem that most likely made it into the released game. I’d guess the user’s problem description should have looked like this:

“The cutscenes get old and take too long. I was unable to skip the cutscenes.”

Adding cutscenes to pretty up loading screens is a fine idea. But it sounds like this user may not have realized the cutscenes were skippable at all. Notice the developer says “Couldn’t be skipped for the first few seconds”? Is there any indication to the user that those cutscenes can be skipped after the first few seconds? Or do users mash the spacebar, see that it can’t be skipped, give up, and go get a sandwich? A silent delay in skipping cutscenes could easily cause extinction of the user’s attempt-to-skip behavior.

Maybe text could show up that says “Press spacebar to skip” once the background loading has completed. Or a progress bar could be overlaid at the bottom of the cutscene. Or the cutscene could switch to a blank loading screen after a keypress to help speed up loading (however slightly).

These are some possible solutions that should have been explored as a result of the user’s feedback. From the sound of it, the developer treated the focus testing feedback as change requests, rejected them, and tossed the whole exercise out the window, bathwater and baby. If feedback had been translated into stories about the user’s problems then focus testing would have have been a valuable tool for improving the game.

This entry was posted on 26 November 2010 and is filed under Games.

Comments are closed.