In PS3's defense, ESS files are ridiculously inefficient

Post » Wed May 30, 2012 11:59 am

In the land of bits on machines that don't have much memory, its important to have efficient memory usage.
Looking through the save files of Skyrim, they certainly are not efficient. They store strings such as "WindhelmIntroDone_var" as a string, which is ridiculously inefficient.

Reading the above variable above as a sentence is similar to "Is Windhelm intro done?" which means this variable is probably a flag/boolean value, which requires precisely 1 bit to express.
Assuming that the variable that is saved to memory is expressed in ascii format, then "WindhelmIntroDone_var" uses 147 bits of memory.
That is not to say that the in-memory value that the game uses is in ascii, that is probably just a bit or at most a byte, but if they keep the save file in memory for fast saving, then this is pretty ridiculous.

I have to assume that they are doing something extremely inefficient like this in game memory because you don't use as many Mbs for a save file otherwise.

First, why would they store a boolean value this way? Its possible that they only wanted to store values that aren't default. For instance, if the Windhelm intro is not done, then the value wouldn't be stored within the save file at all, because its default is false.

But then, in this way of saving data, you have to ask, what is the worst possible scenario? When all boolean values in the game need to be saved in the save file, how much data is that eating up? Much, much more than it would if you were to store ALL possible boolean values in an efficient way. 1/147th the size of the current worst case scenario.


But thats just booleans and they happen to be capable of being stored extremely efficiently. What about an NPC? An NPC has a lot more data and there are lots of NPCs. Where are they? Are they alive? What do they have in their inventory? What conversations have you had with them?
There are efficient ways to store such data, but its a little more complicated than a boolean. For instance you don't store a name, you'd store an id.
Lets say that there are fewer than 8192 unique npcs in the game, you can express that with 13 bits, a large savings over identifying them with a full name.

This thread was about to be a lot longer. Things like containers there are many many things to consider like number of items in the container, for each item whats its id, how many in the stack, is it named by the player, if so what is it's name, item type, how many enchantments, for each enchantment which enchantment id, the potency of the charge, the maximum charge, the current charge.
Despite these complexities of the kind of data that must be persisted there are still massive memory savings to be had similar to the ways above.

But the point is, there is a lot of data to be saved. For this reason Beth should have invested more time in making their memory management as efficient as possible for the sake of the consoles.
Save files don't get to be multiple megabytes neither in memory or on disk (not counting images) without being massively inefficient, not even a Bethesda game should eat that much memory.

I think they're too used to having boatloads of memory in the past when they used to make :tes: games for PC almost exclusively. Hell, I have 8gb of system ram and over a gig of dedicated video memory and I still run out of memory in skyrim after hours of gaming.


So, while I def think Skyrim is game of the year, I thing Beth needs to take a lot of this year's success money and invest it in creating world-class memory management. They'll need it in the future if they ever want to go mobile.
User avatar
Ross Zombie
 
Posts: 3328
Joined: Wed Jul 11, 2007 5:40 pm

Return to V - Skyrim