QA Testing Bugs [merged similar topics]

Post » Mon Jun 11, 2012 3:12 pm

Lvl. 80+, 557.+ Hrs., First Play through, 360.

After reading how many QA's " worked" on Skyrim, or any other Bethesda RPG; I haved developed a theory on their business ethic... Sell Now, Fix Later. Either the QA's are blowing off, what is instructed, or not instructed as "minor" issuses, to be fixed after the games release, OR, the QA's are reporting all the defects they find to their managers, and it is they who allow the game to be released to the public.

Obviously I've put some hours into this, and let me tell ya, I'm tripping over all sorts of bugs, glitches, and defects, and not even going out of my way to do it!!!!! I haven't even started on the PC yet.

I really like Bethesdas game lines, but come on now!!! Maybe some of their staff don't appretiate the seriousness of their jobs, or the economy as it is; and it's those workers that are bringing down the honor of the harder working staff! I'm surprised they're putting up with it.

I wonder when their last Federal Audit was? Hmmmm.
User avatar
Jamie Moysey
 
Posts: 3452
Joined: Sun May 13, 2007 6:31 am

Post » Mon Jun 11, 2012 11:15 am


I wonder when their last Federal Audit was? Hmmmm.
What does a Federal audit have to do with testing a game? :confused:
User avatar
Gen Daley
 
Posts: 3315
Joined: Sat Jul 08, 2006 3:36 pm

Post » Mon Jun 11, 2012 3:24 pm

ah a world where deadlines for games are often there downfall....this was not the case for skyrim but id have imagined that the granduer of 11.11.11 put alot of pressure on a staff that spends most of their time in story boarding and prep......more than likely most of what was actually developed for skyrim was crammed in due to the the deadline......that said they sure pulled of alot of awesomeness under pressure.........its just left us with alot of bugs,glitches,etc....could be much worse.......game could svck on top of that
User avatar
Neil
 
Posts: 3357
Joined: Sat Jul 14, 2007 5:08 am

Post » Mon Jun 11, 2012 1:27 pm

i have to admit some of it is absolutely ridiculous especially that some of the skills are not useful at all endgame thats what bugs me the most I spend all my time leveling these skills and putting perks In them and then they all of the sudden there useless
User avatar
kyle pinchen
 
Posts: 3475
Joined: Thu May 17, 2007 9:01 pm

Post » Mon Jun 11, 2012 10:23 pm

It's one thing to QA test a 4-5 hour COD game thousands of times.

Quite another matter to do the same with a massive open world RPG that provides several hundred hours of content. Bethesda could have hired a QA team of hundreds and had them testing nonstop for years and still never found every single bug. Within the context of huge open world sandbox RPGs, this is hands down one of the least buggy launches IMO.

Keep in mind, I'm not a Bethesda employee and this is my personal opinion.
User avatar
Monika
 
Posts: 3469
Joined: Wed Jan 10, 2007 7:50 pm

Post » Mon Jun 11, 2012 9:09 am

It's one thing to QA test a 4-5 hour COD game thousands of times.

Quite another matter to do the same with a massive open world RPG that provides several hundred hours of content. Bethesda could have hired a QA team of hundreds and had them testing nonstop for years and still never found every single bug. Within the context of huge open world sandbox RPGs, this is hands down one of the least buggy launches IMO.

Keep in mind, I'm not a Bethesda employee and this is my personal opinion.
i thought you were a zombie wrangler
User avatar
MISS KEEP UR
 
Posts: 3384
Joined: Sat Aug 26, 2006 6:26 am

Post » Mon Jun 11, 2012 2:21 pm

Quite another matter to do the same with a massive open world RPG that provides several hundred hours of content. Bethesda could have hired a QA team of hundreds and had them testing nonstop for years and still never found every single bug. Within the context of huge open world sandbox RPGs, this is hands down one of the least buggy launches IMO.

I agree this game is no small scale affair like CoD and creating, let alone QA testing a game like this has to be monumental. But honestly, people will playtest games for free. If they had released this in beta and let a hundred or so testers from each platform play with it early, we could have avoided catastrophes like this, But... it comes down to money and a complex QA ecosystem takes money and time that Bethesda wouldn't or couldn't spare. That, friends, is a simple truth.
User avatar
carley moss
 
Posts: 3331
Joined: Tue Jun 20, 2006 5:05 pm

Post » Mon Jun 11, 2012 2:45 pm

It's one thing to QA test a 4-5 hour COD game thousands of times.

Quite another matter to do the same with a massive open world RPG that provides several hundred hours of content. Bethesda could have hired a QA team of hundreds and had them testing nonstop for years and still never found every single bug. Within the context of huge open world sandbox RPGs, this is hands down one of the least buggy launches IMO.

Keep in mind, I'm not a Bethesda employee and this is my personal opinion.
It's called experience. How many "massive open world RPG that provides several hundred hours of content" games has Bethesda created? And yet we're still running into a ridiculous amount of problems, some of which were present in Fallout 3 and Oblivion.

How many times do you have to stub your toe on something before realizing that you need to go another way or simply remove it? Or wear slippers with a titanium shell? They should know what works by now, so instead of scrapping gameplay mechanics from previous games, and adding new, untested ones, they should be improving them.

Even releasing the game so close to the holiday season wasn't a very bright idea. Morrowind had problems upon release. Oblivion had problems upon release. Fallout 3 had problems upon release. Fallout: New Vegas (Bethesda provided the engine) had problems upon release. They should have waited until they knew they had ample time after release to actually fix a vast majority of the game breaking bugs and glitches.
User avatar
Dark Mogul
 
Posts: 3438
Joined: Tue Feb 20, 2007 11:51 am

Post » Mon Jun 11, 2012 2:10 pm

Zero. There wasn't a single, actual, living human being who played Skyrim before it was released who were not already involved in the process of making it. I don't need to look at the platform-specific problems, I don't need an audit of Bethesda HQ, I don't need a crystal ball or divine word to tell me. The quest "Blood on the Ice" is 100% broken for everybody, except (and this is the kicker), for an AI subroutine that would clear the quest with all boxes checked and not even notice the fact that the house was still covered in blood and gore.

That's just one example (for the record, over 4 different playthroughs I've never gotten past the 'accuse the court-wizard' stage of the quest), and I can offer more if anybody would care to hear them. There are so many errors in this game that would only go unnoticed and considered to not even be a mistake by a program that was created to run through the game in a specific way, in a specific order. Skyrim was never playtested by non-partial (people not already invested into the project) humans at all. If there were dedicated testers, they were all either braindead or ignored.
User avatar
Christine
 
Posts: 3442
Joined: Thu Dec 14, 2006 12:52 am

Post » Mon Jun 11, 2012 9:24 pm

The open world bit would matter in this particular Discussion if it weren't for the fact that aside from the player there really isn't anything going on. the NPC's work on scripted paths that they don't deviate from unless they are scripted to do so lol. they already went so far as to remove levitation as to "preserve the conherency of their quests" so you'd think how they modeled the quests would be air tight.

The fucntionality of the Quests don't really have much to do with how the Linear one way Scripted quests play out to be honest. and those "thousands of things that could go wrong" don't even initiate until your in the vicinity. so I do not believe thats much of an excuse.

So massive so grand but it is still static and unmoving until you are thier to witness it. :biggrin:
User avatar
Hazel Sian ogden
 
Posts: 3425
Joined: Tue Jul 04, 2006 7:10 am

Post » Mon Jun 11, 2012 10:59 am

The open world bit would matter in this particular Discussion if it weren't for the fact that aside from the player there really isn't anything going on. the NPC's work on scripted paths that they don't deviate from unless they are scripted to do so lol. they already went so far as to remove levitation as to "preserve the conherency of their quests" so you'd think how they modeled the quests would be air tight.

I don't quite follow.

To QA test any game, you need to play it. To thoroughly QA test Skyrim, you need to play it for hundreds of hours. There are 410 different (non-radiant) quests and 550 unique locations. How can you test all those areas and quests if you don't play them out? (Not to mention the fact that many of these quests intersect in various ways or can involve some of the same NPCs and locations.)
User avatar
Emily Shackleton
 
Posts: 3535
Joined: Sun Feb 11, 2007 12:36 am

Post » Mon Jun 11, 2012 1:33 pm

To QA test any game, you need to play it. To thoroughly QA test Skyrim, you need to play it for hundreds of hours. There are 410 different quests and 550 unique locations. How can you test all those areas and quests if you don't play them out?
I'm certain there are people who have already completed every single quest in Skyrim, and it's only been 2 months since release. Bethesda knew there were going to be problems, as there are with every single game they've ever released, so why didn't they hire more QA staff and have them test the game over 2 months?

'cause they didn't care. The community is going to fix a majority of the bugs for them while they're free to develop their next cash cow.
User avatar
Catharine Krupinski
 
Posts: 3377
Joined: Sun Aug 12, 2007 3:39 pm

Post » Mon Jun 11, 2012 1:42 pm

I don't quite follow.

To QA test any game, you need to play it. To thoroughly QA test Skyrim, you need to play it for hundreds of hours. There are 410 different quests and 550 unique locations. How can you test all those areas and quests if you don't play them out?

I tailored my original post to clarify before I saw your response. and effectively thats the point, you can't -not- play them out over developement. and since like I stated prior the quests are "air tight" in that you can't really complete them outside their intend scripting, it would be telling that they are broken. I don't see the value in the numbers as others may, Greymoor and greenwall are both forts and title Unique locations, as are all the stones/grottos/caves/cairns.

didn't they do a speed test and found that you can complete it it under an Hour/ two hours? these are folks who -know- the ups and downs of the game, so why are there broken quests? and no I am not talking from experience, mine has been quite novel and without serious issue. doesn't mean I don't like partaking in threads that do discuss issues.
User avatar
Isaac Saetern
 
Posts: 3432
Joined: Mon Jun 25, 2007 6:46 pm

Post » Tue Jun 12, 2012 12:20 am

Since almost no one else makes games of this scope, it's hard to hold them to any one objective standard about QA...most of the other open world RPG's of the past simply were nowhere near the same size as Bethesda's. The few games I think of from the past that were this open and non-Bethesda had their share of bugs, even though they were smaller.

There are definitely some annoying bugs, I don't remember them being quite this bad in Oblivion, but i've been lucky enough not to get the brunt of anything game breaking. I have run into a couple bugs that seem like they should have been handles, stuff like quests that don't go away after completion, the Riften Thane bug was pretty annoying, and seemed like they something they would catch in testing as well.

I like how the OP thinks he can threaten Beth with the IRS to fix his game, that's the height of maturity and intelligence right there.
User avatar
BrEezy Baby
 
Posts: 3478
Joined: Sun Mar 11, 2007 4:22 am

Post » Mon Jun 11, 2012 2:56 pm

Other than for the PS3 I think Bethesda did a pretty good job on this great game

:D
User avatar
Natasha Callaghan
 
Posts: 3523
Joined: Sat Dec 09, 2006 7:44 pm

Post » Mon Jun 11, 2012 6:31 pm

Lvl. 80+, 557.+ Hrs., First Play through, 360.
After reading how many QA's " worked" on Skyrim, or any other Bethesda RPG; I haved developed a theory on their business ethic... Sell Now, Fix Later.
Good job. You really opened America and invented a wheel with that idea.
User avatar
Robyn Lena
 
Posts: 3338
Joined: Mon Jan 01, 2007 6:17 am

Post » Mon Jun 11, 2012 7:14 pm


I'm certain there are people who have already completed every single quest in Skyrim, and it's only been 2 months since release. Bethesda knew there were going to be problems, as there are with every single game they've ever released, so why didn't they hire more QA staff and have them test the game over 2 months?


Yes, there are people who have been playing the game for hundreds of hours since launch and completed many or even all of the non-radiant quests. If it takes hundreds of hours to thoroughly QA test this game, and it takes 4-5 hours to thoroughly test COD or Portal 2, where is Bethesda going to come up with 100 times larger QA budget to test Skyrim? It's certainly not going to sell 100 times more copies.

Can you list a few other massive open world sandbox RPGs that provided hundreds of hours of content and had less buggy launches?

I know of several people on my Steam friends list who have already played more than 700-800 hours and still have not completed every major questline.

In my case I've already played 350 hours and I have only done the Companions, about 60% of the Thieves Guild and 70-80% of the Stormcloaks quests. Very little of the main quest, none of the mage's college, none of the dark brotherhood, only 4 of the daedric quests.
User avatar
Lily Something
 
Posts: 3327
Joined: Thu Jun 15, 2006 12:21 pm

Post » Mon Jun 11, 2012 11:01 am

You don't think they have enough money in their budget? They do, they just don't wish to waste it on testing a game they have no intention of ever actually fixing. Morrowind, Oblivion, and Fallout 3 still have bugs, and they've been out for years. What's your excuse for them?
User avatar
cosmo valerga
 
Posts: 3477
Joined: Sat Oct 13, 2007 10:21 am

Post » Mon Jun 11, 2012 5:37 pm

You don't think they have enough money in their budget? They do, they just don't wish to waste it on testing a game they have no intention of ever actually fixing. Morrowind, Oblivion, and Fallout 3 still have bugs, and they've been out for years. What's your excuse for them?

So, can you list for us a few of these huge open world sandbox RPGs with hundreds of hours of content that were less buggy?
User avatar
Portions
 
Posts: 3499
Joined: Thu Jun 14, 2007 1:47 am

Post » Mon Jun 11, 2012 7:16 pm

More power to you guys then, I found when I did the College I was done in reletively no time, and when I attempted the MQ I was already half way through it in a handful of quests. so i stopped. "to preserve the purity" so to speak, I've only beaten Morrowinds MQ once, never finished Bloodmoon or Tribunal, I beat Shiv ilse once and never finished Oblivions MQ, and yet here I stand nearly beating the MQ for skyrim without an ounce of "rushing" yeah I'll hold off. point an case? yeah they haven't touched it, but they'd be finished with it in a heartbeat.



To be more on topic, does the OP care to list -glaring- bugs as related to the Questlines? and as a side note I recall a poster having link where the devs were laughing at bugs, in addition to the lulz fest that was made out of backwards flying dragons (as titled on Steam amazing backwards flying Dragons) these things had jokes made out of em. does no one feel that Bethesda, now ever BIGGER dogs than they were with FO3/Oblivion should no longer recieve the benefit of the doubt in that regard? "because its Beth" ?


there are bug that could have never been Fortold, like Letroush for example, he defies all reasoning, there is no concievable way Beth in all their mightyness would have fortold the demi-god status that is letroush.
User avatar
darnell waddington
 
Posts: 3448
Joined: Wed Oct 17, 2007 10:43 pm

Post » Mon Jun 11, 2012 10:46 am

You don't think they have enough money in their budget? They do, they just don't wish to waste it on testing a game they have no intention of ever actually fixing. Morrowind, Oblivion, and Fallout 3 still have bugs, and they've been out for years. What's your excuse for them?

A game this scope will always have some bugs, you can't expect them to reasonably eliminate all of the possibilities, only that they work on the really game-breaking ones and minimize them as much as possible.

Again, all of the truly open RPG's I can think of..even from 20 years ago (Origin's Ultima games, Gothic series, few others) had awful bugs, and those were much smaller scope than Skyrim.

In fact, when the first two fallout games came out they were buggy as hell...and again much smaller scale.

I even remember alot of game breaking bugs in BG2..and that couldn't even be called 'open world' really.

I don't think Beth's QA is spectacular, but it doesn't seem worse than anyone elses to me.
User avatar
Sweet Blighty
 
Posts: 3423
Joined: Wed Jun 21, 2006 6:39 am

Post » Mon Jun 11, 2012 10:15 am

Some bugs, sure, but Skyrim doesn't just have some bugs. It has a lot of bugs, many of which were present in previous games. You don't even need QA to tell you about them, yet they're still there. Why is that? And why do the previous games still have plenty of bugs? I guarantee that Skyrim will still have quite a few bugs even after the next TES game is released, and that's years down the road.

So, can you list for us a few of these huge open world sandbox RPGs with hundreds of hours of content that were less buggy?
Totally irrelevant. Even if Call of Duty had thousands of bugs, it wouldn't justify what we see in Skyrim.
User avatar
StunnaLiike FiiFii
 
Posts: 3373
Joined: Tue Oct 31, 2006 2:30 am

Post » Mon Jun 11, 2012 10:02 pm


Totally irrelevant. Even if Call of Duty had thousands of bugs, it wouldn't justify what we see in Skyrim.

You guys make me shake my head IRL sometimes.

Of course it's relevant, because there are no other games like this, there is no industry standard for you to compare to, and the few that do exist seem equally or more plagued with bugs.
User avatar
LittleMiss
 
Posts: 3412
Joined: Wed Nov 29, 2006 6:22 am

Post » Mon Jun 11, 2012 2:15 pm

It's okay.

Companies like EA salivate at this kind of Market. Bethesda is completely dependent on this particular niche market for profit. EA acquired Pandemic Studios (which had an open world engine that worked decently) and Bioware (Mass Effect) not too long ago. It was about the same time they were working on The White Council, which was also supposed to be an open world RPG.

They may still be doing so, despite reports that they cancelled that project.

EA is a predatory company. They already have an open world engine. Several of them, actually.

EA's corporate model is built on exactly this kind of circumstance: A company has a niche market or idea, so EA competes with that market or bullies the competition out, takes over, and they then have a death grip on that market.

This is exactly the kind of situation EA looks for, and no I'm not simply speculating.



My point is that if Bethesda doesn't learn to put some more of that massive revenue percentage into taking care of its users better, I wouldn't be shocked if they were gone from the world entirely within a couple of years.

There are companies out there right now that are looking at Bethesda, Skyrim, and this niche market that's being served up to them on a silver platter; a gift on par with a voluntary sacrificial meal...

And they're salivating.
User avatar
JD bernal
 
Posts: 3450
Joined: Sun Sep 02, 2007 8:10 am

Post » Mon Jun 11, 2012 8:44 pm

People that have never done software development do not understand that a game of this scope is bound to have hundred and more probably thousands of bugs. From cosmetic and totally uninmportant to game breaking. The real only possible test at some point is to release to the public. You could hold the game for years for infinite testing with a limited team, yet not catch all issues.
Even your current web browser has hundreds of bugs. Is it unusable ? No.
And yes sometimes critical bugs are not found during the testing phase.
As a side note my experience on PC has been mostly bug free (but I'm only 30h in the game and have done very few quests). As a software developer myself I'm even impressed it doesn't crash more often.
User avatar
Solène We
 
Posts: 3470
Joined: Tue Mar 27, 2007 7:04 am

Next

Return to V - Skyrim