A Day in the Life of Quality Assurance + Thoughts on Game Development QA

requirements

 

 

While I’m not a tester per se, I do end up doing a lot of exploratory testing as I go through the process of doing documentation.

I’ll admit, too, that my experience has not been in games development but in accounting software development.  Testing is hard and tedious work, and can be especially bad when requirements are vague and shifting.  Given my background in QA, maybe I should be more sympathetic to the game publishers who release broken, buggy or sometimes even unplayable games on the assurance that a day 1 patch will fix things out of the box, but I’m really not.  I feel like by the time you get to actual playtesting, where many of the worst gamebreaking bugs should be obvious, the requirements ought to be solidified enough (requirements: be able to play game, complete game objectives, and perform regular tasks during the course of gameplay without defect) to get the dev tasks reissued and have a smoothly working game.

While this model may work for PC gaming, it’s a really lousy way to handle console or cross platform releases, because it works on the assumption that everyone with a console has access to whatever online network is available to download patches. The expectation for console games used to be, and in this, it was one of the few regards where console games were truly superior to PC games, was that the game would work on your hardware without requiring updates, configuration tweaks, patches to the system and patches to the game. That’s not to say that there weren’t broken and unplayable games for console, but those were inexcusable; the attitude toward PC games was more forgiving, and that forgiving attitude has allowed for some really garbage stuff making its way into console games.

Luckily, I haven’t experienced too many game-breaking bugs in console games, but that’s largely because of how restricted I’ve become in my consumption of console games. A part of it, I think, is the “fear climate” created by the stories of AAA titles being released with tons of bugs; I end up not buying games at all because I’m worried that I’ll plop down all this money for a title and then it won’t work and I will have to go through all of the headache of retrieving my Microsoft login info that i have’t used in 6 or 7 years, stringing ethernet cable across the house (I don’t have wifi) and hoping that I can figure out how to patch a console game. I’d rather just wait for a game of the year edition with vital fixes (I’m looking at you, Clavicus Vile!) and some of the DLC added in at a substantial discount.

I know it’s unfair to companies who put out quality products, I know, but I really feel like the companies who put out inexcusably broken content have ruined it for everyone. Because some AAA title that I’d have no interest in anyway comes out horribly buggy and broken, I’m more hesitant to buy console titles I’m actually interested in.

The games industry as a whole would benefit greatly just by investing a little more time, money and manpower into QA than almost any other aspect of their project. All of the assets, gameplay, and storytelling in the world isn’t going to offset the bad PR of a disastrous launch caused by a failure to devote appropriate resources to your testing. Regardless of what industry you’re in, bugs in your software are going to cost you money and trust.

There’s more at work, though, than just QA.  Video games are often marketing driven software development; the sales team has already sold the idea, deadlines are in place, the marketing is already prepared and the development now has to keep up with the goalposts of the deadline and the promises made by sales & marketing.  Anyone in software will tell you that this is a terrible way to drive product development, but it’s really just how it goes regardless of what part of the industry you’re in.  I also want to say that I’m not blaming the Devs; things get added and dropped in software projects at the whims of clients, BAs and team leads.  What needs to change is the attitude toward what is a finished product that can go out the door.  The point of QA is to make sure that broken software does not get approve and released.  The current state of things is that the window of time during which physical software product is manufactured, distributed and sold is additional time bought for developers to finish their coding.  The day the product hits the shelf cannot be treated as the deadline for completing the product, especially for console releases.

Advertisements

2 responses to “A Day in the Life of Quality Assurance + Thoughts on Game Development QA

  1. I don’t often buy new games…

    …But when I do, I often wonder why everyone reaches for such an epic scope in storytelling when something smaller, simpler, and shorter would not only be cheaper, easier to develop, and be easier to market.

    Like, I’m playing Dragon’s Dogma and I’m thinking, “they’re reaching so high for that epic storytelling thing which is already there, but because they made the game LONGER they had to pad it with a bunch of Standard Fantasy RPG Grind.”

    And really, part of me enjoys killing bats, rats, snakes, goblins, and bandits — but I’ve sunk several hours into the game now and I’m still killing bats, rats, snakes, goblins, and bandits. Most of them without names or backstories…

    …And I’m like… why didn’t they just make a “Good Parts Version” like the first God of War game? Then they could have made scaling the monsters in the boss fights less cumbersome and more cinematic or whatever. :/

    On the one hand, the game gets going a LOT faster than say, Monster Hunter Tri (Tri Again, lol), but it only makes it so far before you’re back to grinding. I suppose I could just keep chasing the plot — but I’ve had some BAAAD experiences in the past (I’m looking at you, SquareEnix) where skipping side missions meant I was woefully under-prepared for story missions.

    –Dither

    • In a way, it’s counterproductive to create massive experiences. I mean, I love the hell out of Elder Scrolls games and have devoted an embarrassing about of time to them, well into the triple digits, while Bethesda has probably gotten maybe a few bucks from me at most, considering that I got Morrowind and the Oblivion expansions used.

      On the other hand, you have problems like with the major backlash against the “next gen” (now last gen) titles that had maybe 4-6 hours of single-player game content. I don’t know if the game industry found a happy medium, because there’s backlash against DLC.

      Things are not looking up for physical media plebs like myself, though I was pretty stoked that GOTY Arkham City was all “Here’s a disc with lots of DLC”. One of the things that astounded me watching the credits for Arkham City was how the number of QA people working on it was probably greater than my company’s entire staff (and we’re no small player in the market we’re in).

      Probably one of the worst examples of padding I’ve dealt with in a major console title was Halo 2; MAN there was a lot of places where it was all “wait on this elevator for several minutes while you’re attacked by things”! That’s kind of a different thing, though, than padding in RPGs. The problem with RPGs is coming up with any sort of balancing mechanic, and I feel like padding is the Jerry-rigged solution to dealing with it.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s