Turning Bad Feedback Into Good Feedback

26 November 2010

One of the biggest challenges designers face is interpreting feedback from users. I’m thinking of the rabid online communities around video games especially, but this applies to any product subjected to users’ brutal gaze before the product launches.

This article on addictive social games included a perfect example — even more telling that it was provided to show how “useless” feedback is:

McMillen said Super Meat Boy was designed without the aid of metrics. And while there was one focus testing session for the game, most of the feedback was thrown out. (One tester suggested that having a static loading screen would be preferable to a cutscene that couldn’t be skipped for the first few seconds because the level was loading in the background.)

The developer apparently disagreed with the tester’s solution and said “What a stupid idea, focus testing is worthless”. This is a very common type of feedback in game forums – bad solutions. Stuff like “Make my character overpowered” or “Throw out this level entirely”.  But solutions aren’t the goal of focus testing to begin with! What you’re looking for are problems.  Then you can have your experienced designers find the best solution to those problems. The trick here is converting the user’s bad solution into a problem, then working back to a better solution.

In the Super Meat Boy example, the user’s solution translates to a meaningful problem that most likely made it into the released game. I’d guess the user’s problem description should have looked like this:

“The cutscenes get old and take too long. I was unable to skip the cutscenes.”

Adding cutscenes to pretty up loading screens is a fine idea. But it sounds like this user may not have realized the cutscenes were skippable at all. Notice the developer says “Couldn’t be skipped for the first few seconds”? Is there any indication to the user that those cutscenes can be skipped after the first few seconds? Or do users mash the spacebar, see that it can’t be skipped, give up, and go get a sandwich? A silent delay in skipping cutscenes could easily cause extinction of the user’s attempt-to-skip behavior.

Maybe text could show up that says “Press spacebar to skip” once the background loading has completed. Or a progress bar could be overlaid at the bottom of the cutscene. Or the cutscene could switch to a blank loading screen after a keypress to help speed up loading (however slightly).

These are some possible solutions that should have been explored as a result of the user’s feedback. From the sound of it, the developer treated the focus testing feedback as change requests, rejected them, and tossed the whole exercise out the window, bathwater and baby. If feedback had been translated into stories about the user’s problems then focus testing would have have been a valuable tool for improving the game.

Comcast Usage Meter Pilot

02 May 2010

I was surprised to find an e-mail in my inbox from Comcast pitching their new Usage Meter. It’s just what you would expect from the name: a website to see how much bandwidth you’ve used so far this month. The bulk of the text follows:

We are pleased to announce the pilot launch of the Comcast Usage Meter in your area. This new feature is available to Comcast High-Speed Internet customers and provides an easy way to check total monthly household high-speed Internet data usage at any time. Monthly data usage is the amount of data, such as images, movies, photos, videos, and other files that customers send, receive, download or upload each month. Comcast measures total data usage and does not monitor specific customer activities to determine data usage.

The current data usage allowance for the Comcast High-Speed Internet service is 250GB per month. This means that the vast majority of our customers – around 99% currently – will not come close to using 250GB of data in a month, and do not need to check the usage meter.

Heavy bittorrent users may rightfully complain about Comcast advertising “unlimited” when there is actually a 250GB cap on traffic. But I think this is a step in the right direction for Comcast, towards completely transparent usage plans. There will be no more fears of finding out your usage is beyond Comcast’s arbitrary limit by receiving a nastygram canceling your service.

Soon you’ll be able to check your usage and compare to their monthly cap. If you go over, perhaps Comcast will offer a higher bandwidth plans at extra cost. How do you advertise “more than unlimited”? I’ll leave that to Comcast’s marketing team to figure out.

Why wasn’t this available long before they started canceling users’ service or slowing downloads? My guess is this was their escape plan for continuing to restrict heavy users despite the FCC’s move towards Net Neutrality. What does that mean in light of the appeals court ruling overturning the FCC’s authority over Net Neutrality?

According to the e-mail this is only a pilot restricted to a few areas – with the FCC out of the picture, will Comcast have any incentive to be this transparent with their users? Or will they return to their old tricks of secret caps and traffic throttling?

Torchlight: Extended

24 January 2010

I wrote about how flawed the Torchlight demo was in a previous blog post.  I decided not to buy the game after it rehashed Diablo’s formulate to a fault, except with bugs that culminated in a game crash losing my progress after a very long boss fight.

If you know me and how many times I’ve pulled Diablo 2 and even Diablo 1 back out of the closet, though, you won’t be surprised to hear that I ended up buying Torchlight anyway.  After playing the demo with the other two classes and farming the demo’s boss a bit, I just had to plow ahead and see the rest of the levels and spells.  For a $20 bill you can’t be but so picky, right?

My experience with the rest of the game stayed very true to what I said of the demo.  The game mimicked Diablo to a fault, had numerous glitches, but also remained fun throughout as I gathered loot and blasted baddies as I upgraded my spells.  In fact, the game stayed true to what I said of the demo a little too well.

The final boss is a very big monster with a very big health bar.  I would guess the back-and-forth fight took me a full 10 or 20 minutes to whittle his health bar down to zero.  He exploded in a hail of colorful loot which I carefully picked over when – cue shocked gasp – the game locked up, losing all of my progress.  I suppose I can’t say I didn’t know what I was in for after playing the demo.

Update: Removing Mana From DPS Classes in WOW

11 November 2009

The precursor to my previous blog post Removing Mana From DPS Classes in WOW was a post I made to the WOW forums. It turns out Ghostcrawler read it and liked it enough to quote it!


This is a prettty good summary. The answer is that we are changing mana-based nukers but not by removing their mana. Instead we are just going to make them less dependent on mana regen, specifically Spirit. You are correct that we really only want healers to run OOM, and we can’t really get rid of it for reasons Squirrelbot mentions. DPS casters still have to manage mana to some degree but they should have tools (e.g. Evocate) to handle that. They shouldn’t “run dry” the way a Holy priest or Resto shaman needs to run dry (ideally — I know this isn’t happening now) when the encounter isn’t going well for you.

Sounds like no plans for a new resource system for mages or warlocks, but that is confirmation that evocation and similar mechanics are intended to give all dps caster specs near-infinite mana.

Removing Mana From DPS Classes in WOW

10 November 2009

After several hints leading up to this year’s Blizzcon, Blizzard announced they would be changing the Hunter class to use a Rogue-like resource called focus. A Blizzard designer answered some questions about it recently on the World Of Warcraft forums. Quoting for emphasis:


you can’t balance mana-using spells around cost since casters have nearly unlimited mana at any point in time and certainly early on in a fight. Energy (and focus, and rage to a much lesser extent to where it’s actually a problem) are limited at any given moment but come back pretty quickly. Mana-spells have to be balanced around cast times and cooldowns instead.

Abilities and damage can’t be balanced based on mana alone. If you gave mages an ability that takes 50% of their mana and does huge damage, mages would cast that ability twice (gibbing 2 players) and then complain that they go OOM too fast.

Mana is a fight-timer mechanic, not a DPS or rotation balancing mechanic. This used to apply to all ranged classes as well as healers. For healers the fight-timer mechanic is obvious: once your healer is out of mana, the group is going to die. If you can’t beat the boss by then you need a new strategy or better gear.

Applying a fight timer to ranged DPS is less clear-cut. I believe it was intended as a trade-off for the fact that ranged classes took less AOE damage.

Increasingly mana has been made near-infinite for damage classes and specs. Likewise, AOE in the LK expansion has spread out to the ranged dps. The primary purpose of mana in the game is now only to limit the longevity of healers.

This is ultimately a good thing. It guarantees you won’t brute-force encounters by, say, 2-manning a single raid boss for 3 hours with the paladin spamming his biggest heals while you slowly chip away at the boss’ life. More practically speaking, if your group’s tank or dps undergear an encounter, the healer’s mana will prevent you from completing it.

The fact that ret paladins and enhancement shamans also have healing specs prevents them from being converted to a new mechanic. Instead they will continue to receive near-infinite mana in their DPS specs, requiring their abilities to be balanced around cooldowns and buffs/debuffs.

Without a healing spec, it will be much easier to convert hunters to a new mechanic that can influence their dps and rotation decisions. It also “feels” right because mana is traditionally a spell-casting mechanic.

What about the other two non-healing ranged DPS classes that currently use mana? Mages and warlocks could also be improved by switching to a new mechanic. Why should those classes continue to be limited at 5 minutes of damage, while hunters and melee can dps indefinitely?

It might feel strange having mages not use mana, but perhaps they’ll come up with another mechanic that feels equally magical. I don’t think we’ll see this in Cataclysm’s launch, but I’ll still cross my fingers and hope for it!

Red Robot Jack-o-Lantern

09 November 2009

This year for Halloween I carved a Red Robot pumpkin.

Red Robot Pumpkin

Diablo Expansion Pack: Torchlight

08 November 2009

I just finished playing the Torchlight demo. From the moment the game begins, it is made abundantly clear that this is not just a game “in the grand tradition of Diablo” – the game is Diablo. The town, the vendors, the music, the random dungeons, the potions and scrolls: they’re all here.

If you’re looking for a Diablo expansion pack to keep you busy until Diablo 3‘s eventual belated release, Torchlight fits the bill. There are enough new monsters and abilities to keep any fan of Diablo interested, and the price is right at $20.

However, Torchlight is also contending with many other fine games, most of which have more unique and interesting mechanics. Torchlight copies Diablo literally to a fault. Only a handful of new mechanics appear, such as the Shared Stash (no more running 2 copies to trade loot with your mule) and the omnipresent Dog or Cat by your side. Most of the time the same problems and annoyances from Diablo lurk in Torchlight with no efforts at improvement. The gameplay shows signs of poor polish and QA – making it feel even more like a hurried 3rd-party expansion pack milking a quick buck out of the franchise.

The last boss of the demo drove these problems home hard enough to scare me away from the full game. I picked the mage-like Alchemist and started my way down the Lore skill tree. The first tier of abilities have you raising minions from the corpses of dead foes. When I got to the boss though, there were no corpses in the level to raise my minions from. This left me going toe-to-toe with the level’s boss using only my wand.

Ironically, the wand was my most powerful weapon – doing 30-50 damage, where my only magic spell did 6 damage. I had to kite the boss around the room, only stopping to attack when range and line-of-sight allowed. Being a mage without any useful spells made me wonder if they spent any time tuning to ensure all 3 characters and 9 skill trees played sensibly through the early levels of the game.

After this long and clunky ring-around-the-rosy fight, I finally defeated the boss. An NPC appeared for a cut scene, said one line, and then proceeded to wander around the dungeon aimlessly. I can only guess her pathing broke. With the keyboard locked out entirely for this never-ending cut scene, I had to End Task the game process. I didn’t bother starting it back up.

The Tochlight demo left a bad taste in my mouth culminating in that glaring bug. I won’t be buying Torchlight myself, but I can certainly see the appeal of playing Diablo for a few more levels.

Portal and WOW Pumpkin Carving

22 October 2009

It’s about that time to start thinking about what nerdy emblems can be carved into a pumpkin. Here are the past couple years of geek-o-lanterns my wife and I made:

Portal’s GLaDOS survives as a Pumpkin.

For The Horde!
For The Horde!

Squirrelbot before pre-decompisition into a forsaken.

Trick-or-treat!  Go ahead and reach in for some candy…if you don’t mind losing a hand to the murlocs. You can’t see them, but I also toothpicked some pumpkin slivers on their backs for spines.

Enterprise Over-Caching

22 October 2009

Caching your database tables within an app server for a website (eg. Memcached or C#’s Cache) is a very bad idea.

Every time you add caching to a set of data, you have to add extra code – which means added complexity and potential bugs. You’re adding consistency issues as well – especially if any other apps (or DBAs) can modify the data without invalidating the cache. These problems can be solved of course – with more code, which means even more complexity and bugs.

Most importantly, let’s consider speed. If you’re caching data, it’s probably because of speed concerns in the first place. One way or another, you’re going to need to access this data. Even worse, you probably need to sort the data too. Why the heck would you tiptoe around the database like you’re scared to pull data from it? That is exactly what the database is tuned for! Accessing the cache for these types of requests is significantly slower than pulling from an indexed database table.

Why not handle data access speed in your data layer? Rather than using each app server as a horribly inefficient slave database – use a slave database! This is going to be faster, more scalable, and require zero extra code.

Caching isn’t all bad, of course. Caching the static content of the 10 most popular articles that account for 70% of your website’s requests is probably a good idea. Caching your entire Users table? Not such a good idea. For some reason premature caching of entire database tables seems to be the norm in “enterprise” C# and Java web apps, even ones with a handful of users.

If you get put on a project like this, suggest scaling back the scope until it’s needed. Or at least until after the application is ready to be optimized based on performance profiling. Don’t add that code until there is justification that a performance problem exists, and that adding code is really the best way to solve it.

Installing a Linux Left 4 Dead Dedicated Server

01 December 2008

Here’s what I did to install a dedicated linux server for Left 4 Dead. Ideally you want to do this on a non-root account.

First, prep a directory for your steam client. Your game installs will reside here, each in its own subdirectory.

mkdir steam
cd steam

Download the linux steam client and unzip it. Type yes when prompted. Afterwards you’ll have an executable named steam which is used to download and update games.

wget http://storefront.steampowered.com/download/hldsupdatetool.bin
chmod +x hldsupdatetool.bin

Run ./steam and you’ll see a list of options for the steam client. I used the command below to download, but I’m not sure yet what l4d_full is. I figure I can switch to that easy enough if I run into a roadblock with left4dead.

./steam -command update -game left4dead -dir .

I like to run servers inside screen so that I can logout and log back in from any computer with ssh. This will also keep the server running in the background if your desktop has to reboot.

cd l4d

You’ll see a bunch of text fly by, and now your server is running! Press ctrl+a then d to detach from your screen, getting you back to the command prompt. To bring the left4dead server back up make sure you’re logged into the right user and run screen -xD. The screen program is quite elaborate but these are the basics to keep a process running in the background.

This just gets a very basic server running. There are plenty of things you can customize in your steam/l4d/left4dead/ directory. For instance:

motd.txt: This is the message displayed when players join your server.
missioncycle.txt: Pick which missions are played and in what order.
maplist.txt: This is the list of what maps are available on the server. Essentially these maps can be picked by an admin or possibly by player vote, even if they aren’t part of the normal rotation.
host.txt: This appears to be a website displayed when players join the server. This may be displayed alongside the motd.txt?

In the steam/l4d/left4dead/cfg/ directory there are some options for configuring gameplay. A lot of these appear to be defaults for other types of servers and even client configurations which are irrelevant to the server.

server.cfg: This is the one that you typically put all your custom config options in. Fortunately the defaults are very good.
game.cfg: I believe this is the client-side version of server.cfg. So on your machine you play on, you can put custom options here that aren’t part of the default options interface.
infected.cfg: Looks like changing this 0 to a 1 lets humans play as zombies on your server. infected_off.cfg is the same file with the 0 set

I don’t have a good listing of the options that you can put in game.cfg, but if you check through some of the default configs in that directory like two_players.cfg and infected_360.cfg you’ll see a few of the options you can change.