» Tue Jun 12, 2012 10:25 pm
As a software engineer, I know it's virtually impossible to exhaustively test any non-trivial program and these games definitely qualify. In our world, safety critical systems are tested as much as they can be and there are restrictions on the process to ensure (as much as possible) that there are no bugs that can actually kill someone. Even so you can never be sure -- it's just too complex. In a game, bugs are annoying, but not actually dangerous to life or even property. They can go on testing forever and still not get the last bug out. The only real way to do this is to have a good process and good people and make a good judgement on when to release. Skyrim seems to be about right to me. Fallout 3 had a few too many lockups for my taste.
It might be worth considering how a user having a problem could send their bugs (along with a game save) to Bethesda through the internet (even using PS3 and XB360s) so that they can find their bugs easier. The trouble is to really get the right data, you'd have to detect the problem and dump the file somewhere the user could send it (or send it automatically if the user gave permission). Or if it is a quest problem, you could send a game save any time. But Bethesda isn't on the hook to solve those problems in the same way as we are on the hook to the government. Ever wonder why government contracts are so expensive? Partly it's all the process and checks and testing and accountability that go with them.