QA Testing Bugs [merged similar topics]

Post » Mon Jun 11, 2012 10:29 am

Unless you interview the QA team directly or they come here to answer your questions you'll likely not get any legitmate answers, only speculation.
User avatar
xx_Jess_xx
 
Posts: 3371
Joined: Thu Nov 30, 2006 12:01 pm

Post » Mon Jun 11, 2012 10:09 pm

Judging by how awful the UI was, the worst shadows i have ever seen in any video game period and the tiny FOV which is more suited to a console game or a CRT computer screen from ten years ago.........I would say that there was very little if any testing on the PC side. I tested this game out on my four year old back up computer which would be considered mid-low range by todays standards and i was easily able to max out the game and I adjusted the ini further and the game still ran just fine so they didnt even have the proper INI tweaks for even a decent current computer much less a monster computer like I'm using now.
User avatar
Aman Bhattal
 
Posts: 3424
Joined: Sun Dec 17, 2006 12:01 am

Post » Mon Jun 11, 2012 6:12 pm

I think I remember hearing that the QC testing was contracted out to a couple drunk monkeys playing poker.
User avatar
Angus Poole
 
Posts: 3594
Joined: Fri Aug 03, 2007 9:04 pm

Post » Mon Jun 11, 2012 10:21 pm

"Yay, it works! No crashes at startup! Ok guys let's sell this thing"
User avatar
Rhysa Hughes
 
Posts: 3438
Joined: Thu Nov 23, 2006 3:00 pm

Post » Mon Jun 11, 2012 10:32 am

Seeing as only somebody involved in the testing could answer with "more than speculation", I expect you won;t get the answers you are looking for.

However, this question struck me as odd, and indicative that maybe you don't have a basic understanding of testing.

"Do the people doing the game development do any thorough in game testing?"

Um, I hope not. Its very ppor testing procedure to have the people who do development also do the testing, because they don't really bring anything new to the table. After all, if it didn't work for them, they wouldn't deliver. But the fact that something works for the developer says very little about what the average user's experience will be!
User avatar
Anna S
 
Posts: 3408
Joined: Thu Apr 19, 2007 2:13 am

Post » Tue Jun 12, 2012 12:18 am

Lol...do you really think Bethesda has any intention of releasing this information?
User avatar
Ashley Hill
 
Posts: 3516
Joined: Tue Jul 04, 2006 5:27 am

Post » Mon Jun 11, 2012 9:04 pm

I read that Bethesda had computers do some of the testing.
User avatar
Joey Bel
 
Posts: 3487
Joined: Sun Jan 07, 2007 9:44 am

Post » Mon Jun 11, 2012 8:59 am

Seeing as only somebody involved in the testing could answer with "more than speculation", I expect you won;t get the answers you are looking for.

However, this question struck me as odd, and indicative that maybe you don't have a basic understanding of testing.

"Do the people doing the game development do any thorough in game testing?"

Um, I hope not. Its very ppor testing procedure to have the people who do development also do the testing, because they don't really bring anything new to the table. After all, if it didn't work for them, they wouldn't deliver. But the fact that something works for the developer says very little about what the average user's experience will be!

Some of that makes sense under certain circumstances. But generally, I expect the developers to test their own work. For example, why all dual wielding weapons...if you can't actually play that way? They put this feature in, but failed to see what would happen if you tried to 'favorite' them and actually attempt to switch to spells and back to weapons. it simply does not work. This.... should not have even made it to the beta kids without being caught.
User avatar
sally coker
 
Posts: 3349
Joined: Wed Jul 26, 2006 7:51 pm

Post » Mon Jun 11, 2012 2:42 pm

I am going to take a wild stab in the dark here, and say: "No, they ain't gonna tell us."

Looking at it though, I am going to guess that whatever testers they had just did quests, and that's it. No free range exploring, otherwise, I should think they would have discovered that going someplace before you have whatever quest, breaks the game.......

In all reality, this IS the beta test, and WE are the testers. :D
User avatar
~Amy~
 
Posts: 3478
Joined: Sat Aug 12, 2006 5:38 am

Post » Mon Jun 11, 2012 9:23 am

Bethesda's QA has always been a bit dicey but you know what? I forgive 'em because I know they'll be working hard on patches.

I have a certain EA game that basically I can no longer progress in due to a bug - there's also several other nasty bugs that are pretty game breaking too. In the 8 months or so it's been out not one single patch. There is never going to be a patch. Oh, and no console commands (I'm on PC) to even counter the bugs.

tbh that made me respect Bethesda even more 'cos they at least support their products long after release.
User avatar
Josephine Gowing
 
Posts: 3545
Joined: Fri Jun 30, 2006 12:41 pm

Post » Mon Jun 11, 2012 3:37 pm

I've yet to encounter any showstopping bugs playing the game, aside from rare CTD, and occasional alt+tab related CTDs. However, I got the game for Christmas, haven't sunk nearly the time into it as most other people here, and tend to play very carefully -- knowing that Bethesda's games tend to be exceedingly buggy at launch, I've read up on bugs in the game and how to avoid them.

But I have sunk some time into the CK since it's been released, and I'm really seeing a pattern to the nature of the bugs, both the ones I've personally experienced, and the ones I've read about. A great many of the bugs they didn't catch (i.e., the ones we, the players, are encountering) don't occur if you do exactly what you're supposed to do, when you're supposed to do it. But in a game as open and free as Elder Scrolls games tend to be, you really can't expect the player to know, or even desire, to do things a specific way, or in a specific order.

I think the testers, whether human or automated, knew too much about what you're supposed to do, and didn't deliberately try to break the game. I also think that the way the developers use the CK, demonstrated in their official videos, and also the only way a bunch of things work in the CK (extensive copy+paste, find&replace, duplicate existing assets and renaming them, rather than creating new entries) is pretty counter-intuitive to anyone who wasn't the developers, and doesn't have an extensive knowledge of the game's assets.

The way the CK was designed, the way the game, and the quests, etc. were all designed, was intuitive to the developers, and so when they played the game to test it, they played differently than an outsider would. They may have tested everything every possible way they could think of, and fixed all the bugs they found, but their own knowledge would frustrate their ability to approach and test as an outsider. The majority of the playerbase could encounter a bug that the developers did not, simply because of that difference in knowledge affecting their approach to playing the game. The players all come to the game with their own idea of what they want out of it, or how it should be played, and each developer or tester would too, but they also have the knowledge sitting in their brain of how it was designed to be played.

In addition, we certainly aren't using the build the developers and testers used, for Skyrim or the Creation Kit. At least some of the bugs we're encountering are likely to be a result of a different build process, of code that was stripped of dev/debugging/testing code, but never itself actually tested to see if those things were removed without damaging the final product.

But the real kicker is that very few programmers in the industry really know what they're doing. Most of them can hack something together, make a product that works most of the time, but there are very few who really understand and can write good, clean code. The rest of the computer industry moves so fast, but software development, and education haven't kept up. They're largely held back by C, and the multitude of variations spawned from it. Most serious software is written using some C variant, because of performance, and because it's the industry standard. But the language itself doesn't encourage good code -- in many ways it actively encourages writing bad code.

It's part of the culture too -- who cares how buggy it is, as long as it does what it's supposed to do, at least some of the time? The problem is that the people who care about good code aren't nearly as "productive" as the people who write bad code. Features impress far more easily than the stuff behind the scenes, because the features can be seen. And delivering features performs better in the market, because people are easily dazzled by features, but it takes time to see the flaws.

So, in a way, it's ultimately the fault of the market. When customers demand that they be given things now, rather than a better product later, they shouldn't be surprised at what they get. Hopefully Bethesda will follow through on their promise to keep patching Skyrim, and the CK as well. They didn't provide very good support for Oblivion.
User avatar
Krystal Wilson
 
Posts: 3450
Joined: Wed Jan 17, 2007 9:40 am

Post » Mon Jun 11, 2012 10:20 pm

i would be interested in going to the uespwiki and seeing how many quests DONT have a bug section taking up the bottom quarter of the page. i cant remember having seen a quest that didnt have at least a couple 'dont do this or you'll break it' notes...

this is not "a game of this scope" or "show it done better". this is they gave up and wanted money now. i think (and hope) the next game they kick out the door in a rush to meet a deadline sells 10% of what this did, because they arent going to learn a [censored]in thing otherwise.

fully patched (by modders) morrowind was great, i bought oblivion years after release but it was fine in its end state as well. having no first hand experience with fresh releases from bethesda i didnt know how deep this pool of [censored] was i was jumping into. now i do...
User avatar
Jose ordaz
 
Posts: 3552
Joined: Mon Aug 27, 2007 10:14 pm

Post » Mon Jun 11, 2012 6:38 pm

What's wrong with weapons and armor headbomb?
User avatar
kelly thomson
 
Posts: 3380
Joined: Thu Jun 22, 2006 12:18 pm

Previous

Return to V - Skyrim