QA Testing Bugs [merged similar topics]

Post » Mon Jun 11, 2012 11:11 am

Eww, Beth isn't the way of EA...yet...but i picking a Beth on the way/with a chance to not be.

vs

an EA that is already there? I'd go with Beth.

Op: :wave: um....can you list examples of Bugs in skyrim as to make the thread relevant to the section its in? :D
User avatar
Sabrina Schwarz
 
Posts: 3538
Joined: Fri Jul 14, 2006 10:02 am

Post » Mon Jun 11, 2012 11:28 am

You guys make me shake my head IRL sometimes.

Of course it's relevant, because there are no other games like this, there is no industry standard for you to compare to, and the few that do exist seem equally or more plagued with bugs.
Why does there need to be a standard?

Bethesda should be striving to release games with absolutely 0 bugs, not games with a max of 250 or so.
User avatar
Isaac Saetern
 
Posts: 3432
Joined: Mon Jun 25, 2007 6:46 pm

Post » Mon Jun 11, 2012 10:29 am

Totally irrelevant. Even if Call of Duty had thousands of bugs, it wouldn't justify what we see in Skyrim.

It's completely relevant, because to my knowledge Skyrim is the least buggy or certainly one of the least buggy launches for this type of game.

If you enjoy playing massive open world sandbox RPGs, bugs come with the territory. It's not devs being "lazy" or "irresponsible." It is simply not feasible to QA test a 500-600 hour open world RPG on the same scale as you can QA test a short 4-5 hour linear shooter on rails.

User avatar
daniel royle
 
Posts: 3439
Joined: Thu May 17, 2007 8:44 am

Post » Mon Jun 11, 2012 9:53 pm

Why does there need to be a standard?

Bethesda should be striving to release games with absolutely 0 bugs, not games with a max of 250 or so.

That is ridiculous, seriously one of the most ridiculous things i've read, much less about a project the size of Skyrim.

There is literally probably NOTHING in existence on PC minus minesweeper that has 0 bugs...man, you just made me shake my head even more.
User avatar
sw1ss
 
Posts: 3461
Joined: Wed Nov 28, 2007 8:02 pm

Post » Mon Jun 11, 2012 7:51 pm

uh....Booty...I don't think the OP is coming back...olol
User avatar
W E I R D
 
Posts: 3496
Joined: Tue Mar 20, 2007 10:08 am

Post » Mon Jun 11, 2012 8:39 pm

It's completely relevant, because to my knowledge Skyrim is the least buggy or certainly one of the least buggy launches for this type of game.

If you enjoy playing massive open world sandbox RPGs, bugs come with the territory. It's not devs being "lazy" or "irresponsible." It is simply not feasible to QA test a 500-600 hour open world RPG on the same scale as you can QA test a short 4-5 hour linear shooter on rails.
No, it's not, because the bugs in Skyrim have also existed in Fallout 3 and Oblivion. The problems with saves on the PS3 have existed in both Fallout 3 and Fallout: New Vegas.

And seriously? They come with the territory? Of course Skyrim is going to have bugs, but the fact that it's a big game doesn't justify it having as many as it did at launch. Bethesda might be the only creator of games this big, but they're also the most experienced when it comes to it, so we should be seeing significantly less bugs with each game.. and I can't say I've been seeing that at all.

There is literally probably NOTHING in existence on PC minus minesweeper that has 0 bugs...man, you just made me shake my head even more.
I don't remember anyone saying Skyrim should have absolutely no bugs. What we're saying is that it has too many, that there's no excuse for a lot of them, and that it wouldn't be nearly as bad if Bethesda had actually invested more in QA.
User avatar
R.I.P
 
Posts: 3370
Joined: Sat Dec 01, 2007 8:11 pm

Post » Mon Jun 11, 2012 9:23 am

People that have never done software development do not understand that a game of this scope is bound to have hundred and more probably thousands of bugs. From cosmetic and totally uninmportant to game breaking. The real only possible test at some point is to release to the public. You could hold the game for years for infinite testing with a limited team, yet not catch all issues.

Not true.

There is no way on Earth that even one person played this game and even made it through the main quest alone without encountering game-breaking bugs.

There is an acceptable level of "variance" in making a game. Bad news: This ain't it.

Yes, some bugs will be present and will have to be fixed later. Then there's "We just don't give a crap. We'll sell them a broken product, maximize our profits to shareholders, and then do what we can to run more ads and cover our PR butts later at bargain basemant prices - and we'll never have to produce a product that actually functions as it should in the process."

Make no mistake. This company, and this product, fit every facet of the second category, and none of the first.

This is absolutely and undeniably the path that was chosen so that the most possible profit could be delivered to the shareholders, regardless of future consequences.

Companies - especially share held companies do this, to maximize profit for today all the time. They do it because they don't care what happens to the company tomorrow. They want to maximize their profit on investment now.

This is exactly what happened to Elevation Partners - Pandemic Studios and Bioware. It's exactly what happened to Westwood. It's exactly what happened to Maxis. The list goes on and on.


We're not talking about "A game that's hard to debug and some things just happen to slip through."

That's not what happened here. What happened here was a complete disregard for the end user, and absolute disregard for the quality of the finished product upon delivery.
User avatar
Kayla Oatney
 
Posts: 3472
Joined: Sat Jan 20, 2007 9:02 pm

Post » Mon Jun 11, 2012 3:35 pm


didn't they do a speed test and found that you can complete it it under an Hour/ two hours? these are folks who -know- the ups and downs of the game, so why are there broken quests?

Yes, the fact that the QA testers have become so familiar with this game that they learned all kinds of quirky details and ways to complete the main quest in 2 hours is a testament to how hard these guys actually work.

I'm sure they did manage to catch and fix a lot of bugs before launch. However, they probably could have had a team of hundreds of folks testing the game, iterating the build, retesting, over and over delaying the launch for years and still never found every single bug.
User avatar
Esther Fernandez
 
Posts: 3415
Joined: Wed Sep 27, 2006 11:52 am

Post » Mon Jun 11, 2012 12:39 pm

Yes, the fact that the QA testers have become so familiar with this game that they learned all kinds of quirky details and ways to complete the main quest in 2 hours is a testament to how hard these guys actually work.

I'm sure they did manage to catch and fix a lot of bugs before launch. However, they probably could have had a team of hundreds of folks testing the game, iterating the build, retesting, over and over delaying the launch for years and still never found every single bug.
But they would have found more. We are not asking for NO bugs. We are asking for LESS. Just because they couldn't find every bug using that method doesn't mean it shouldn't be used, because the entire point behind QA is to eliminate as many bugs as possible.

Bethesda should be throwing as much into QA as possible instead of thinking, "Oh, we'll never find every bug, so lets just get 5 people to play-test for us and call it a day."
User avatar
candice keenan
 
Posts: 3510
Joined: Tue Dec 05, 2006 10:43 pm

Post » Mon Jun 11, 2012 9:59 pm

Yes, the fact that the QA testers have become so familiar with this game that they learned all kinds of quirky details and ways to complete the main quest in 2 hours is a testament to how hard these guys actually work.

I'm sure they did manage to catch and fix a lot of bugs before launch. However, they probably could have had a team of hundreds of folks testing the game, iterating the build, retesting, over and over delaying the launch for years and still never found every single bug.


You do see an issue with "shortcuts" no? like if you're going to do shortcuts why is it being tallied as Fastest completion of the MQ if they are using shortcuts? the years and years and perfection motif isn't neccesary, I'm well aware you can spend forever and a day on something and still missing things, yes I realize this but thats not my point. I'm talking about the glaring issues that some peeps have had where they aren't doing anything special and cannot continue through their game. infact one bug I've come across despite my statement about not having issues was at saarthal where the game crashes consistently after completing a certain task, and the only way to get past it and not break the entire quest chain was to "look away"
User avatar
meghan lock
 
Posts: 3451
Joined: Thu Jan 11, 2007 10:26 pm

Post » Mon Jun 11, 2012 11:58 pm

And seriously? They come with the territory? Of course Skyrim is going to have bugs, but the fact that it's a big game doesn't justify it having as many as it did at launch.

We're not talking about "A game that's hard to debug and some things just happen to slip through."

That's not what happened here. What happened here was a complete disregard for the end user, and absolute disregard for the quality of the finished product upon delivery.

Can you guys list any other massive open world sandbox RPGs that provide hundreds of hours of content and did a better job QA testing?

@Alaston I haven't completed the main quest, but so far I have not completed any game breaking bugs over the course of 350 hours. I'll have to get back to you after I do the main quest.
User avatar
Jonathan Windmon
 
Posts: 3410
Joined: Wed Oct 10, 2007 12:23 pm

Post » Mon Jun 11, 2012 7:05 pm

Can you guys list any other massive open world sandbox RPGs that provide hundreds of hours of content and did a better job QA testing?
That is not a justification for anything. You should not be content with a game just because it's slightly good at not being as bad as others, especially when it's still actually bad.
User avatar
Aaron Clark
 
Posts: 3439
Joined: Fri Oct 26, 2007 2:23 pm

Post » Mon Jun 11, 2012 1:00 pm


But they would have found more.

Yes, they would have found more of them.

And in order to test the game on that same scale, financing a team of hundreds of QA testers to work on this for years as well as the devs needed to iterate the build, it would have likely increased the budget from $100 million to $1 billion or more. The game would be coming out in 2015 and probably bankrupt the company.

But they would have found more of those pesky bugs.
User avatar
Prisca Lacour
 
Posts: 3375
Joined: Thu Mar 15, 2007 9:25 am

Post » Mon Jun 11, 2012 11:48 am

Hindsight is always 20/20.
You guys think they didn't test enough, when I'm sure this game had thousands of bugs when testing first started, now its to dozens.
Its common sense that there's always a deadline and budget in business.
Of course, they could've spent more time and made things better, but what invention or technology can you NOT say that with.
Everything can be improved, everything. Bashing Beth for all of technology's, and any forms in life, woes is ignorant and childish.
User avatar
Naughty not Nice
 
Posts: 3527
Joined: Sat Nov 04, 2006 6:14 am

Post » Mon Jun 11, 2012 8:56 pm

uh....Booty...I don't think the OP is coming back...olol
You're asking somebody to cite examples of something that would number in the thousands. See the 'hardware and software issues' forum at the top of the page? Take a perusal through there and have a look at the bug/crash threads if you're really interested, and don't be so obtuse.

Looking for a massive, open-world game that isn't inherently broken like Skyrim? Two Worlds 2.

Even still, the argument that 'nobody else is doing this, so they can do it as half-assed as they want' is pretty flimsy at best.

You proud folks who actually have working games, feel good about yourselves. For whatever reason, you got the luck of the draw and your hardware configuration actually run the game in a semblance resembling how it was intended. Through no fault of our own, some of us are not so lucky, and to presume that our issues are imaginary, non-critical (to you, maybe), or over-exagerrated is arrogant. Extremely arrogant.
User avatar
WTW
 
Posts: 3313
Joined: Wed May 30, 2007 7:48 pm

Post » Mon Jun 11, 2012 7:48 pm

There is no way on Earth that even one person played this game and even made it through the main quest alone without encountering game-breaking bugs.

It is very possible that your pet game breaking bug is not easily reproductible so give the testing guys a break...



Yes, some bugs will be present and will have to be fixed later. Then there's "We just don't give a crap. We'll sell them a broken product, maximize our profits to shareholders, and then do what we can to run more ads and cover our PR butts later at bargain basemant prices - and we'll never have to produce a product that actually functions as it should in the process."

Make no mistake. This company, and this product, fit every facet of the second category, and none of the first.

This is absolutely and undeniably the path that was chosen so that the most possible profit could be delivered to the shareholders, regardless of future consequences.


Companies - especially share held companies do this, to maximize profit for today all the time. They do it because they don't care what happens to the company tomorrow. They want to maximize their profit on investment now.

This is exactly what happened to Elevation Partners - Pandemic Studios and Bioware. It's exactly what happened to Westwood. It's exactly what happened to Maxis. The list goes on and on.


We're not talking about "A game that's hard to debug and some things just happen to slip through."

That's not what happened here. What happened here was a complete disregard for the end user, and absolute disregard for the quality of the finished product upon delivery.

Sure one of the goal is to maximize profit , but I do not buy at all your theory to deliberately ship a broken product.
No sofware especially of the scope of Skyrim is ever finished. They could have spent 1 year more polishing/testing it endlessly but at some point software has to get out.
And believe it or not, it is a quality product.
User avatar
Cool Man Sam
 
Posts: 3392
Joined: Thu May 10, 2007 1:19 pm

Post » Mon Jun 11, 2012 10:42 am

Yes, they would have found more of them.

And in order to finance a team of hundreds of QA testers to work on this for years as well as the devs needed to iterate the build it would have likely increased the budget from $100 million to $1 billion or more and the game would be coming out in 2015 and probably bankrupt the company.
You're pretty ignorant if you think they work within their budget and make only enough profit to actually cover the costs, or that they spent as much as they could on QA.

Hindsight is always 20/20.
You guys think they didn't test enough, when I'm sure this game had thousands of bugs when testing first started, now its to dozens.
Its common sense that there's always a deadline and budget in business.
Of course, they could've spent more time and made things better, but what invention or technology can you NOT say that with.
Everything can be improved, everything. Bashing Beth for all of technology's, and any forms in life, woes is ignorant and childish.
It should not have thousands of bugs. They are creating the same type of game every few years, constantly developing them for years, with some of the most elite in the gaming industry within their ranks.

The more you do something, the better you get at it. Especially when it's the same exact thing, using similar tools and resources. They have no excuse for continually creating bug-laden games and releasing them at launch with a significant amount of bugs still. They have no excuse for the same bugs being in multiple games within different franchises. They have no excuse for multiple bugs continuing to exist in their games, even after years of patches and fixes.

You'd have to be an idiot to create the same thing, over and over, with the same problems each time you do it. Just look at the problems with saves on the PS3 for Fallout 3, Fallout: New Vegas, and Skyrim. How hard would it have been for them to hire someone who develops exclusively for the PS3? Or to even contact Sony for help with the issue?
User avatar
Danny Blight
 
Posts: 3400
Joined: Wed Jun 27, 2007 11:30 am

Post » Mon Jun 11, 2012 3:11 pm

I it is a quality product.
In what world is 'not-functional' synonomous with 'quality'?
User avatar
FLYBOYLEAK
 
Posts: 3440
Joined: Tue Oct 30, 2007 6:41 am

Post » Mon Jun 11, 2012 9:35 pm

Skyrim is nowhere near "non functional" for the majority of people playing it, I feel relatively secure in saying that.

I've found that most people who post about bugs are usually people who have already played a while, and are justifiably upset that a bug has messed up the game they spent alot of time on, the thing is that this situation is nowhere near something being "non functional"..it just means you have to redo hours of play or something.

I had this with the Riften thane bug..annoying, and I lost like ten hours becuase I only save on sleep, but it's hardly "non functional"..and I don't think most of the bugs people are mentioning make the game unplayable by any means, though sure there are always some with very severe issues.
User avatar
Jimmie Allen
 
Posts: 3358
Joined: Sun Oct 14, 2007 6:39 am

Post » Mon Jun 11, 2012 12:13 pm

In what world is 'not-functional' synonomous with 'quality'?

I'm talking about the PC version for which I had very few issues since day one.
User avatar
NAtIVe GOddess
 
Posts: 3348
Joined: Tue Aug 15, 2006 6:46 am

Post » Mon Jun 11, 2012 10:01 am

I stopped caring when I realized the http://www.escapistmagazine.com/articles/view/issues/issue_83/471-Bethesda-The-Right-Direction founded it circa 1984.

The Bethesda Game Studios division has always been about experimental stuff. That's in its spirit. The story goes back since the physics engine for John Madden Football and the rise of EA.
I bet the ammount of success the elder scrolls role play game caught on was unexpected.

The series is all about testing new stuff that a dev wants and each one does its thing.
What I don't understand is how, after so long and so much notoriety, the team does not care to improve the quality of their software. Publishing highly experimental stuff might be cool, it is the bleeding edge. Yet, I don't want to be hacking the game with each new release.

I'm not impressed enough to buy the next expansions or even the next deal in the series.
So, what can this publisher do to gain my confidence again?

Discuss.
User avatar
Star Dunkels Macmillan
 
Posts: 3421
Joined: Thu Aug 31, 2006 4:00 pm

Post » Tue Jun 12, 2012 12:49 am

I doubt Bethesda had actual human testers to test the game. Remember when Todd Howard said they were using automated games, running 24 hours a day to find bugs? I bet that is all the testing they did. We're the real beta testers when it comes down to it, and Bethesda is definitely one of those "sell now, fix later" kind of company.
User avatar
tiffany Royal
 
Posts: 3340
Joined: Mon Dec 25, 2006 1:48 pm

Post » Mon Jun 11, 2012 5:08 pm

It has my confidence.

End of discussion.
User avatar
Wane Peters
 
Posts: 3359
Joined: Tue Jul 31, 2007 9:34 pm

Post » Mon Jun 11, 2012 10:40 pm

Since almost no one else makes games of this scope, it's hard to hold them to any one objective standard about QA...most of the other open world RPG's of the past simply were nowhere near the same size as Bethesda's. The few games I think of from the past that were this open and non-Bethesda had their share of bugs, even though they were smaller.

There are definitely some annoying bugs, I don't remember them being quite this bad in Oblivion, but i've been lucky enough not to get the brunt of anything game breaking. I have run into a couple bugs that seem like they should have been handles, stuff like quests that don't go away after completion, the Riften Thane bug was pretty annoying, and seemed like they something they would catch in testing as well.

"I like how the OP thinks he can threaten Beth with the IRS to fix his game, that's the height of maturity and intelligence right there."
Broken, busted, even been dead once, however, I haven't been released from the oath I took. Threaten...no. Actual curiosity...yes. Any one of us can get access to tax records, and should we all unite for better business ethics, you would be surprised how we can get our government to work for us....even against game developers. Thank you for your insight though.
User avatar
Symone Velez
 
Posts: 3434
Joined: Thu Sep 07, 2006 12:39 am

Post » Mon Jun 11, 2012 5:00 pm

You don't have to hack the game? I mean, hey, if they can create a game this great with old software, good for them! Why spend money to get newer software when you can do the exact same thing with old software you already have?
User avatar
Brandi Norton
 
Posts: 3334
Joined: Fri Feb 09, 2007 9:24 pm

PreviousNext

Return to V - Skyrim