[BUG] Ash piles bloat savegames

Post » Sat May 26, 2012 10:35 pm

This bug is a holdover from Fallout 3: ash piles never get removed from the game at any point in time, not even when the game cleans up a cell you have not visited for a while. Wherever they are created, they remain forever, and each and every one of them that gets created and not cleared is junk data in your save.

For those wondering, savegame bloat is unwanted data that is permanently stuck in your savegames. The more bloat, the higher the risk of your saves becoming corrupted. Ash piles are a source of bloat, ergo they contribute to heightened risk of save corruption, and possibly help to exacerbate the PS3 performance problems which seem to be tied to the size of savegames.

But the real kicker is that this is yet another unfixed bug from a previous game. Is it too hard to get the game to clean these things up correctly?
User avatar
Katie Samuel
 
Posts: 3384
Joined: Tue Oct 10, 2006 5:20 am

Post » Sun May 27, 2012 2:11 am

The game has to track so many different objects that the addition of a few ash piles here and there isn't going to cause a hit to performance, or otherwise lead to save game bloat. It could be an issue if the game is trying to remove them but fails to, and that has follow on consequences.

But when I first saw them in-game I was wondering if they had the same bug as FO3/NV.
User avatar
Ellie English
 
Posts: 3457
Joined: Tue Jul 11, 2006 4:47 pm

Post » Sun May 27, 2012 12:28 am

The glow effect being reapplied on every discovered (but not collected) Nirnroot every 24hours(ingame) is sure to be having a far greater detrimental effect upon save game sizes.
User avatar
Lucie H
 
Posts: 3276
Joined: Tue Mar 13, 2007 11:46 pm

Post » Sat May 26, 2012 1:59 pm

This bug is a holdover from Fallout 3: ash piles never get removed from the game at any point in time, not even when the game cleans up a cell you have not visited for a while. Wherever they are created, they remain forever, and each and every one of them that gets created and not cleared is junk data in your save.
Not sure of the effect on savegame bloat but I wrote a fix for this in MMM for F3 which Quarn included in the Unofficial Fallout Patch. It's actually pretty simple, I haven't been following the TESnip threads on using it with Skyrim (until we get the CK), but anyone wanting to fix this simply add an object script attached to Ash piles (and in Fallout 3 there were also goo piles) containing:

Begin OnReset	disable	markfordeleteEnd
At the next cell reset the engine will remove them from the world and any data from a subsequent save.
User avatar
Albert Wesker
 
Posts: 3499
Joined: Fri May 11, 2007 11:17 pm

Post » Sat May 26, 2012 4:36 pm

Not sure of the effect on savegame bloat but I wrote a fix for this in MMM for F3 which Quarn included in the Unofficial Fallout Patch. It's actually pretty simple, I haven't been following the TESnip threads on using it with Skyrim (until we get the CK), but anyone wanting to fix this simply add an object script attached to Ash piles (and in Fallout 3 there were also goo piles) containing:

Begin OnReset	disable	markfordeleteEnd
At the next cell reset the engine will remove them from the world and any data from a subsequent save.
That... :facepalm:
It is really THAT easy? :facepalm:
Just what are the programmers at Bethesda Game Studios doing? :facepalm:
User avatar
Sam Parker
 
Posts: 3358
Joined: Sat May 12, 2007 3:10 am

Post » Sun May 27, 2012 12:27 am

That... :facepalm:
It is really THAT easy? :facepalm:
Just what are the programmers at Bethesda Game Studios doing? :facepalm:

Involves cheese, arrows, knees and uhh sweetrolls.
User avatar
Sxc-Mary
 
Posts: 3536
Joined: Wed Aug 23, 2006 12:53 pm

Post » Sat May 26, 2012 11:16 pm

I remember reading in a post that the same respawnable NPC can't be raised two times in the same gameplay. This way, once you have raised each bandit model, you won't be able to raise a bandit again. The same happens with conjurers, vampires, imperial soldiers... well, any generic NPC. I suppose this applies also to NPCs raised by other NPCs.

It could have something to do with ash piles not dissappearing.
User avatar
Jerry Cox
 
Posts: 3409
Joined: Wed Oct 10, 2007 1:21 pm

Post » Sat May 26, 2012 10:45 pm

I've been using Eola as a follower, who casts raise dead constantly and each time leaves an ash pile. I've also been using conjured bows occasionally. In 45 hours my savefile is already 10MB. It seems like any object that goes through a 'falling' motion, whether it's ashed bodies, ice wraith and other 'ethereal' enemy corpses, dropped weapons, fired arrows and such are retained permanently by the game. I've saved by an ice wraith corpse, rebooted, reloaded, and had the corpse still there but unable to be interacted with at all. There are most certainly some issues going on that are affecting game performance the longer the game is played. I haven't even been to that many locations or major cities yet.

I'm not even sure how much a problem something like this is. Repeating or looping scripts would be much worse. Why is it that sometimes my follower has this constant 'magic effect' sound going off near them? How come I hear the sounds of active spells after a quick-load despite the spells being cast nowhere near the quick-save point? I don't know how to write scripts so I'm not really sure if I'm going to even attempt to fix these retention problems myself.
User avatar
Bryanna Vacchiano
 
Posts: 3425
Joined: Wed Jan 31, 2007 9:54 pm

Post » Sat May 26, 2012 5:13 pm

Wow... triple facepalm! This is severe. ;)

Excellent find and definitely saving that scriplet for when the CK comes out. This should still be passed to Bethesda though.
User avatar
Lucie H
 
Posts: 3276
Joined: Tue Mar 13, 2007 11:46 pm

Post » Sat May 26, 2012 8:47 pm

The game has to track so many different objects that the addition of a few ash piles here and there isn't going to cause a hit to performance, or otherwise lead to save game bloat. It could be an issue if the game is trying to remove them but fails to, and that has follow on consequences.

[Citation needed]
User avatar
Emma Parkinson
 
Posts: 3401
Joined: Wed Jul 26, 2006 5:53 pm

Post » Sat May 26, 2012 9:27 pm

Why am I not surprised that this would carry over? :facepalm:
User avatar
LuBiE LoU
 
Posts: 3391
Joined: Sun Jun 18, 2006 4:43 pm

Post » Sat May 26, 2012 11:21 pm

I remember reading in a post that the same respawnable NPC can't be raised two times in the same gameplay. This way, once you have raised each bandit model, you won't be able to raise a bandit again. The same happens with conjurers, vampires, imperial soldiers... well, any generic NPC. I suppose this applies also to NPCs raised by other NPCs.

It could have something to do with ash piles not dissappearing.


Yea i have actually seen something close to this and having to do with ash piles, i raised a necro npc and it followed me till it died outside whiterun and left an ash pile, well a few days later when the cell was respawned, the ash was still there and a necro npc had spawned next to it, and just stands there not attacking, just talks almost like he is still a raised follower, but he isnt a zombie .
User avatar
abi
 
Posts: 3405
Joined: Sat Nov 11, 2006 7:17 am

Post » Sat May 26, 2012 3:38 pm

Didn't someone at beth state they were using a 'new' game engine??? If that's the case, why do we see so many of the same issues we saw all the way back to morrowind?
User avatar
Kelly James
 
Posts: 3266
Joined: Wed Oct 04, 2006 7:33 pm

Post » Sun May 27, 2012 2:03 am

Didn't someone at beth state they were using a 'new' game engine??? If that's the case, why do we see so many of the same issues we saw all the way back to morrowind?


No its just a very very modified version of the same one Oblivion/FO3/FO:NV ran on, they claim its been so heavily modified it might as well be called "new"
User avatar
Eve(G)
 
Posts: 3546
Joined: Tue Oct 23, 2007 11:45 am

Post » Sat May 26, 2012 7:50 pm

No its just a very very modified version of the same one Oblivion/FO3/FO:NV ran on, they claim its been so heavily modified it might as well be called "new"

Unfortunately, such logic does not apply to game engines: they are black and white. Either you have a new engine, or you don't. You can't just modify an already existing engine and call it new, no matter how huge the modifications are. You want a new game engine, you have to start from the very bottom; scrap everything you used previously and do it all over again.

Todd Howard snake-oiled us with the new engine talk. He didn't outright lie, but he was stretching the truth a great deal. Considering how many old bugs I've seen (precipitation ignores objects, ash piles are permanently stuck in the game, lights occasionally don't light the landscape, NPCs don't react to sneak attacks if the victim dies, numerous floating/disconnected objects) I've since dismissed those words completely, and have simply satisfied myself with saying the Creation Engine has always existed since Morrowind. Gamebryo is not a full engine anyways, it's just a renderer. Bethesda plugged Gamebryo into their engine in Morrowind (then it was called NetImmerse) and wrote most of the rest themselves, and this Creation Engine was born. Oblivion was Creation Engine 2, Fallout 3/New Vegas was Creation Engine 2.5, and now Skyrim is Creation Engine 3.

But what this series needs to continue forward is not a new Creation Engine version, we need a new engine, full stop (or at least a new renderer). Skyrim isn't even a month old and it already looks dated. This is as far as the current engine will ever go, its capabilities have effectively bottomed out here. Gamebryo needs to go if this series is to continue to move forward with the times.

But that's enough about that. Our issue here is simple: Bethesda failed to fix the sticky ash piles, causing junk data to accumulate in the saves. At best, this is an optimization flaw, and at worst, it can destroy someone's saves.
User avatar
YO MAma
 
Posts: 3321
Joined: Thu Dec 21, 2006 8:24 am

Post » Sat May 26, 2012 11:17 pm

Didn't someone at beth state they were using a 'new' game engine??? If that's the case, why do we see so many of the same issues we saw all the way back to morrowind?

They're not using a new game engine, end of story. "Creation Engine" is just a new name for marketing.
User avatar
Matthew Aaron Evans
 
Posts: 3361
Joined: Wed Jul 25, 2007 2:59 am

Post » Sat May 26, 2012 10:14 pm

No its just a very very modified version of the same one Oblivion/FO3/FO:NV ran on, they claim its been so heavily modified it might as well be called "new"
I think it makes it even worse, that it's the same engine. I mean, I have no doubts about, that it's capable of very wonderful things and it does seems to be capable even more thant we have seen in the games, but somehow Bethesda treats it like the ugly stepchild, who doesn't receive tender loving care and instead just gets a new set of clothes, but never a bath.

If they have done it my way....
I would have hammered out all the bugs and glitches I could find, to have a rock solid base engine, and then apply to all the new enhancements. Not just keep dressing up a severely mistreated and ignored pile of codes.
User avatar
Steve Smith
 
Posts: 3540
Joined: Sat Jun 30, 2007 10:47 am

Post » Sat May 26, 2012 1:29 pm

Why is it so hard to just impose realistic limits to the game?

Remember only the last 256 arrows, 256 NPC positions, 256 dead-things, 256 dropped items, 256 knocked-over items... A rolling list that just drops the old items once a new one pushes it out of the list. That is a lot easier than trying to prioritize and sort things based on what is needed to remember longer, or forget faster. Poof, instant speed-boost. No thinking involved.

The worst thing is talking to people... it remembers every dialogue option you selected... even those with no more options left... Why? No more options = no more dialogue. Turn them into a generic NPC and remove the "critical character" status. Let them die peacefully.

Things to remember forever... anything placed in our homes. (Not just kicked around, actually placed somewhere. Picked-up and dropped.)

Things to forget about ASAP... any EMPTY dead things.

Things to remember longer... unsearched dead things, that we have actually seen. (Not things killed by AI scripts, no-where near any location we have looked or walked. I keep seeing dead animals, bandits, random encounter event junk... Lingers forever. While things I did encounter, seem to disappear faster than normal.)
User avatar
Gen Daley
 
Posts: 3315
Joined: Sat Jul 08, 2006 3:36 pm

Post » Sat May 26, 2012 10:27 pm

But what this series needs to continue forward is not a new Creation Engine version, we need a new engine, full stop (or at least a new renderer). Skyrim isn't even a month old and it already looks dated. This is as far as the current engine will ever go, its capabilities have effectively bottomed out here. Gamebryo needs to go if this series is to continue to move forward with the times.

As much as I disagree with - and dislike - a lot of the spiteful (and often false) comments going around on these forums about Bethesda, I agree with this. Skyrim's environments are really well designed and I generally love the atmosphere (and the entire game actually), the engine is really showing it's age. It basically seems to be roughly the same rendering engine as Fallout 3 with low quality dynamic shadows added and slightly better LOD rendering.

I really hope Bethesda invests in this for the next title. With the financial succes of Skyrim, they should be able to. I'm sure it'll be beneficial to them, even from a business point of view. It will lead to more pleased customers and more sales in the long run. The thing is, I don't think there's any engine on the market that suits their needs. Most engines provide better rendering but just aren't made for an open world of this scale with the enormous amount of dynamic objects in them. This is why they develop them in-house. I guess they should get a few more software engineers and crank out a brand new house-made engine with none of the previous glitches in it.

I'd gladly be proven wrong if anyone knows of any game that has a scale of TES games with so much objects in them that can be manipulated by the player and tracked and saved for future reference. All open world games I know are way more limited in what you can do in them.
User avatar
Hayley Bristow
 
Posts: 3467
Joined: Tue Oct 31, 2006 12:24 am

Post » Sun May 27, 2012 12:53 am

I've a feeling that Beth figured 'Why bother taking the time and money to fix things that the modders are going to do for free anyway?". In which case they should have gotten the CK out a lot quicker than they are! I'm just afraid that the delay in releasing the CK is because they're busy removing functions they don't want in the hands of modders. Hope I'm wrong about that.

At any rate I've been keeping myself busy using the console to remove a whole bunch of stuff (especially booby trap stuff, loose trash and bones and other useless moveable objects) from the game world and getting my saves back below 10mb. Why in the world did they mark so much stuff 'persistent' (like any dropped object)?
User avatar
brandon frier
 
Posts: 3422
Joined: Wed Oct 17, 2007 8:47 pm

Post » Sun May 27, 2012 1:26 am

I'm just afraid that the delay in releasing the CK is because they're busy removing functions they don't want in the hands of modders. Hope I'm wrong about that.

It's possible. There are dozens of very useful console commands for modders that have been deprecated in the EXE (ie SetDebugText, ToggleDebugText.) They still are returned by the "help" function, but they don't do anything, so they've been removed.

One of the delays is going to be due to adding "Steam WorkShop" integration into the CK. I don't know why this couldn't have just been added later. The CK would have made it far easier for us to identify quest and scripting bugs, for example, so that they could get fixed officially for all platforms. For Morrowind and Oblivion, it was out within a few days of the game. Now... two months?
User avatar
Jeff Turner
 
Posts: 3458
Joined: Tue Sep 04, 2007 5:35 pm

Post » Sat May 26, 2012 2:29 pm

After killing all enemies within both Ustengrav and Morvarth's Lair, it still does not show "Clear" on the Game Map. Does this have something to do with the ash piles?
User avatar
Eileen Collinson
 
Posts: 3208
Joined: Thu Dec 28, 2006 2:42 am

Post » Sat May 26, 2012 7:04 pm

It is gratifying to see you on the bug hunt Kivan :D
I gotta thank you personally for all the work you put into the unofficial patches. Hmm, Canada....

I guess tools like Wrye programs that could remove bloat will surface if these occurrences are not removed.

Cheers!
User avatar
yermom
 
Posts: 3323
Joined: Mon Oct 15, 2007 12:56 pm

Post » Sat May 26, 2012 6:19 pm

It's possible. There are dozens of very useful console commands for modders that have been deprecated in the EXE (ie SetDebugText, ToggleDebugText.) They still are returned by the "help" function, but they don't do anything, so they've been removed.

One of the delays is going to be due to adding "Steam WorkShop" integration into the CK. I don't know why this couldn't have just been added later. The CK would have made it far easier for us to identify quest and scripting bugs, for example, so that they could get fixed officially for all platforms. For Morrowind and Oblivion, it was out within a few days of the game. Now... two months?

Theyre also supposed to be building an entirely new wiki website full of tutorials on how to use the CK, which isnt a light undertaking.
User avatar
helliehexx
 
Posts: 3477
Joined: Fri Jun 30, 2006 7:45 pm

Post » Sun May 27, 2012 1:09 am

I've been using Eola as a follower, who casts raise dead constantly and each time leaves an ash pile. I've also been using conjured bows occasionally. In 45 hours my savefile is already 10MB. It seems like any object that goes through a 'falling' motion, whether it's ashed bodies, ice wraith and other 'ethereal' enemy corpses, dropped weapons, fired arrows and such are retained permanently by the game. I've saved by an ice wraith corpse, rebooted, reloaded, and had the corpse still there but unable to be interacted with at all. There are most certainly some issues going on that are affecting game performance the longer the game is played. I haven't even been to that many locations or major cities yet.
Keep in mind that those elements the engine does clean up, it only does so when that cell is revisited after the cell respawn timer. In this case the cell loads, old elements are cleared and -- if you then save your game -- these are removed from your save (or more precisely, no longer stored in your save).

This means though that elements from cells you have only visited once will remain in your save indefinitely. Similarly, visiting a cell that's reset and saving your game, while clearing out old data, is likely adding new data too -- if new enemies spawn for example, their status now becomes saved. So on the whole, while save game files can sometimes shrink a little the more more cells you visit the larger your savegame becomes.


Todd Howard snake-oiled us with the new engine talk. He didn't outright lie, but he was stretching the truth a great deal. Considering how many old bugs I've seen (precipitation ignores objects, ash piles are permanently stuck in the game, lights occasionally don't light the landscape, NPCs don't react to sneak attacks if the victim dies, numerous floating/disconnected objects) I've since dismissed those words completely, and have simply satisfied myself with saying the Creation Engine has always existed since Morrowind. Gamebryo is not a full engine anyways, it's just a renderer. Bethesda plugged Gamebryo into their engine in Morrowind (then it was called NetImmerse) and wrote most of the rest themselves, and this Creation Engine was born. Oblivion was Creation Engine 2, Fallout 3/New Vegas was Creation Engine 2.5, and now Skyrim is Creation Engine 3.

But what this series needs to continue forward is not a new Creation Engine version, we need a new engine, full stop (or at least a new renderer). Skyrim isn't even a month old and it already looks dated. This is as far as the current engine will ever go, its capabilities have effectively bottomed out here. Gamebryo needs to go if this series is to continue to move forward with the times.
While I agree consider this -- a whole new engine means a whole new set of tools for Bethesda's staff to learn, in addition to engine licensing costs. There's a huge cost overhead for Bethesda to move to a new engine, which is why we see the 'Creation Engine' for Skyrim: it's easier to tack on band-aid 'upgrades' than use a new engine from scratch. Ultimately, though, the Gamebryo base is showing its age and at some time Bethesda will need to switch and foot the cost for future profitability.

However, don't forget -- as modders we've been able to do so much modding and port work from previous games precisely because the engine changes little with each new release. If Bethesda switches to an entirely new engine, depending on its license requirements, modding may be much more limited with the new engine or not be possible at all. Two-edged sword eh?


Why is it so hard to just impose realistic limits to the game?

...[snip]
And ironically, the 4GB LAA patch possibly helps with larger saves (untested, just a theory). Therefore Bethesda could probably fix the large save-game issue by adding a single flag to the .exe that takes all of one second to do.
User avatar
Add Meeh
 
Posts: 3326
Joined: Sat Jan 06, 2007 8:09 am

Next

Return to V - Skyrim