» Tue Jun 19, 2012 9:36 pm First off, make sure you've read http://www.gamesas.com/index.php?showtopic=1353517, by wrinklyninja.
With the latest Beta patch for Skyrim officially getting sent out to all clients, there have been some changes to BSA Load Ordering that you might want to know about. Basically, the issues brought up in the previous thread have been fixed. So this post is only here for educational purposes, in case you're curious as to the order of resources loading.
To summarize, the order resources are loaded is:
- Registered BSAs -> Plugin BSAs -> Loose Files
Disclaimer: The game doesn't actually work exactly like this. This is how it logically works. As in, if you had to describe the decision process the game goes through, this is it. Realistically, working this way would be a waste of resources and processing time (why load up every file, when you already know a different one is going to "win"?). It's just much easier to understand if you think of it like this. A good description of how it would actually work is http://www.gamesas.com/topic/1345724-important-bsas-and-you/page__view__findpost__p__20282031.
Contents:
- Resource Load Order
- Registered BSAs
- Plugin BSAs
- Loose Files
- Registered BSAs
- What This Means For Mod Users
- Other Questions
- Replacing Scripts
- Directory Thrashing
- What are all these terms you use?
- What about Nitpick?
- Replacing Scripts
1. Resource Load Order
Skyrim loads its resources from the Data folder. Makes sense. These resources can also be packed up into a nice neat little archive, called a BSA. These are gamesas's custom archive files, and internally they mirror the layout of the Data folder. They're nice, because they keep the Data folder uncluttered, and it makes it really easy to remove every file from one mod.
Ok, but what happens when the same resource exists in more than one place? For example http://img401.imageshack.us/img401/9967/dummyn.jpg, which is the texture for the map you see all the time in castles, with the little flags and stuff sticking out of them. This file exists in Skyrim - Textures.bsa. That's the original file. If you download a texture replacer, you'll get it as a loose file probably, and if you download HD Skyrimmap from Steam Workshop, you'll get that file in a BSA. So which file actually gets used? Well, which ever one is loaded last. The game loads resources in three stages, with the last files loaded being what you see in game:
Needs Investigation: With pre-1.4.26 Skyrim, only Registered BSAs and Loose Files were loaded when the game first loads. That means anything in a Plugin BSA won't take effect yet - but you'll only notice that if you're trying to replace a Main Menu asset, like http://skyrim.nexusmods.com/downloads/file.php?id=7801. Once you start a game, either by loading a saved game or starting a new one, then the Plugin BSAs are loaded. So in the example of Brumbek's Main Menu Logo replacer, if it's loaded via a Plugin BSA (like it is when installed via Steam Workshop), you won't see the spinning logo until you load a game, then exit back to the Main Menu.
So the investigation needed: Is this still the case? I'll get back to you on that once I get the time to test it.
Registered BSAs
First the game reads your Skyrim.ini file. There's a section that tells the game which BSAs to load. It looks like this for most people:
[Archive]sResourceArchiveList=Skyrim - Misc.bsa, Skyrim - Shaders.bsa, Skyrim - Textures.bsa, Skyrim - Interface.bsa, Skyrim - Animations.bsa, Skyrim - Meshes.bsa, Skyrim - Sounds.bsasResourceArchiveList2=Skyrim - Voices.bsa, Skyrim - Voicesixtra.bsaThe game will load the BSAs in order as listed, BSAs in sResourceArchiveList first, and sResourceArchiveList2 second. So if you manually register a texture replacer BSA (like the HD Texture packs), but put it at the beginning of the list - they won't load! Because the Skyrim - Textures.bsa will override them of course. If you put it after, or at the end of the list, or in the second entry, then they will load.
Easy enough, but there is one drawback: There's a 255 character limit on each of these entries. This is because the game internally uses a 256 byte character buffer to receive the value when reading it. One byte is reserved for the NULL terminator (a byte with a value of 0, indicating the end of the data), so that leaves 255 characters for file names. If you want to get around this limitation, you'll need to use http://skyrim.nexusmods.com/downloads/file.php?id=9591.
Plugin ESPs
Next the game loads any BSAs associated with a plugin. When the game loads an active ESP, it also looks for a BSA of the same name (for the most part, some additional variation allows one plugin to load multiple BSAs, but that's not important for this discussion). So dummy.esp will make the game load dummy.bsa. This is true even if the ESP does absolutely nothing (for example, the official HD Texture packs do this, as well as Oblivion's Shivering Isles expansion). Cool! We found an easy way to make our replacement textures only load when we want! Just deactivate the plugin and they don't load, activate it and they do!
Ok, but what happens if two Plugin-loaded BSAs have the same file in it? Well, it's easy now: whichever one loads last wins (as usual). But now there's a reliable way to control which order they're loaded in: they're loaded in the same order as their associated plugins. This means if dummy.esp loads before another.esp, then dummy.bsa will also load before another.bsa, meaning of course that another.bsa will win, because it loaded last.
Sounds good, but I seem to remember there being other issues with using BSAs...
Yes, nothing major really, unless you're trying to mod your game with a lot of plugins. Due to the way the game indexes plugins and stores the data in your save game, you can only have 256 active mod files. That's because objects are referenced by what's called a FormID, which is a 32-bit (4-byte) unique number, usually written as a 8-digit hexadecimal number. The first two hex digits are used to identify which mod the data came from. Two hex digits can represent a maximum of 256 unique numbers (0-255). Now, Skyrim.esm and Update.esm will always fill up two of those. Another one that's not obvious is your saved game itself - this one is always assigned the ID of hexadecimal FF, or 255. That leaves room for 253 other plugins. And here we get to our point: dummy plugins that don't do anything at all, but exist solely to make the game load a BSA? Yep, those take up one of our plugin slots. This might not seem like a big deal, but once you start trying to play with 300+ mods, you'll be faced with the hard decision of deactivating some mods. Luckily, dummy plugins you can just unpack the BSA and use the resources as Loose Files, then you don't need the plugin anymore. Other plugins can be merged via Wrye Bash into the Bashed Patch, further cutting down your mod count. And hopefully an equivalent of TES4Gecko will come out eventually, which will be able to merge mods into a single plugin. This last one however has an impact on your saved games, so it's for the more advanced user only.
Needs Investigation:Another small strangeness associated with Plugin BSAs, at least with pre-1.4.26 - they aren't loaded until you load either a saved game, or start a new game. That means if you're still at the Main Menu for the first time, any replacers loaded this way haven't taken effect yet! Woah! Not a big deal, unless you're trying to replace the Main Menu Logo...Oh wait. That mod works fine when installed as Loose Files, or if the BSA is registered in your Skyrim.ini, but it only works as a Plugin BSA after loading a saved game. This behavior still need to be verified now that 1.4.27 Skyrim is out.
Loose Files
This one's simple. If the file exists in the Data folder, it will always win. It will always be used, even if the same file exists in a Registered BSA or Plugin BSA. This is how most texture replacers work, and this is what mod managers like Wrye Bash or NMM are made for. Managing the Loose Files so you don't have to track them yourself. It's easy to keep track of which file came from which installer when you only have 3 mods. But when you get up to 150 mods, each with their own resources, a lot of them overlapping, a mod manager just takes the headache out of it.
2. What This Means For Mod Users
Not much really. There is one minor disadvantage to all this: Steam Workshoop uses BSAs by default, even for texture replacers that don't need a plugin. That means you end up with an ESP that is only there to load a BSA, taking up one of your 253 mod slots doing it. If you start getting close to the 253 mod limit, you can cut back by either:
- (Recommended) Unpacking those BSAs to Loose Files. A useful tool for unpacking the BSA is http://obge.paradice-insight.us/wiki/DDSopt. Once the BSA is unpacked, just pack up those Loose Files along with any documentation into an archive, and install with your favorite mod manager. Now you can deactivate the plugin, saving you a mod slot.
- (Not Recommended) Register the BSAs. This method you'll use your Skyrim.ini to load the BSAs instead of using a plugin. First you'll need to deactivate or delete the plugin. Next, you'll need to add the name of the BSA to your Skyrim.ini, in the following section:
[Archive]sResourceArchiveList=Skyrim - Misc.bsa, Skyrim - Shaders.bsa, Skyrim - Textures.bsa, Skyrim - Interface.bsa, Skyrim - Animations.bsa, Skyrim - Meshes.bsa, Skyrim - Sounds.bsasResourceArchiveList2=Skyrim - Voices.bsa, Skyrim - Voicesixtra.bsa
The reason I don't recommend this method is that it's harder to change the order. This method you need to edit your Skyrim.ini each time you want to change the order, whereas with Method #1, you just either change the Installer Order of the package in BAIN, or in NMM. Also, you'll run up on the 255 character limit pretty quick, which means that you'll need to make use of http://skyrim.nexusmods.com/downloads/file.php?id=9591, which in turn requires SKSE. There's nothing wrong with requiring SKSE, but you'll have to wait for a SKSE update after each game update.
Again, each of these tricks only works if the plugin in question is an empty, dummy plugin, solely there for the purpose of loading the BSA. You can check this using TESSnip or another similar utility, or right click on the mod in Wrye Bash and select 'Mark Mergeable...' to find out if the plugin is empty. Wrye Bash will tell you if the plugin is empty, but it won't say anything at all relating to its "emptyness" if it's not empty.
3. Other Questions
Replacing Scripts
Pre-1.4.26 for Skyrim, you couln't replace a Vanilla Script file with one in a Plugin BSA. This is no longer true, it has been fixed, and script replacers should work as both Loose Files and Plugin BSAs now.
Directory Thrashing
What is it? http://wiki.tesnexus.com/index.php/Oblivion.exe_file_and_directory_thrashing. Basically, simply having too many ESP, ESM, and BSA files (even inactive) in the data directory would cause Oblivion to go crazy, bringing your game to a crawl as disk I/O went through the roof. It's why Wrye Bash has the Auto-Ghost feature.
Does it affect Skyrim? I don't know, I didn't really feel like making 400+ plugins to test this out. Not to mention that there was never a reliable number anyone could pin down that triggered it. But, if it does affect Skryim, there's another good reason to not use BSAs at all. Loose Files didn't trigger it for Oblivion, which is why a lot of modders didn't use them (BSAs).
What are all these terms you use?
I used a bunch of terminology here, most of it old, but there's a couple new terms I made up for this explanation.
- Plugin - any .esp or .esm file.
- ESM - Elder Scrolls Master. Usually saved as a .esm file, which has the same format as a .esp file, only one bit (as in, one bit of a byte) is different.
- ESP - Elder Scrolls Plugin. Most mods are ESPs, but some are ESMs. Their format is the same, but the game handles them slightly differently in some specific cases.
- ESS - Elder Scrolls Save. Your saved games. This is loaded after all the ESPs and ESMs are loaded, so that its changes (like what items are in which containers) can override what the plugins say.
- BSA - Bethesda Softworks Archive. Sort of like a .zip file, but specific to gamesas's games, and optimized (sort of) for their specific usage.
- Load Order - The order that your mods (ESPs and ESMs) are loaded. When conflicts occur, the last loaded conflict "wins".
- Install Order - The order that files are installed into your Data folder. When multiple packages install the same file, the last installed package "wins".
- BSA Load Order - The order that the game engine loads BSAs. When conflicts in data files occur, the last loaded BSA "wins".
- Plugin BSA - A BSA that the engine loads because an ESP/ESM with the same name is active.
- Registered BSA - A BSA that the engine loads because it's listed in Skyrim.ini.
- Loose File(s) - A resource file in the Data folder that's "loose", as in not packaged into a BSA.
- FormID - A unique ID assigned to each "record" of a plugin. Each object in the game is defined by a record, and each record has a FormID. A FormID is 32-bits, with the first 8 bits defining which ESM or ESP the record belongs to.
- SKSE - Skyrim Script Extender. A nice utility that everyone should be using. It aims to extend the functions available to scripts written for Skyrim. It's also useful as DLL loader, if one wants to inject some code into Skyrim's game engine.
- SD - Script Dragon. Another DLL loader, but with a different goal in mind than SKSE. Both can work along side each other.
- Vanilla - A term referring to Skyrim as distribute by Bethesda and Steam. This means no modifications have been done. Alludes to the "plainness" of the vanilla ice-cream flavor.
What about Nitpick?
I'm still maintaining it. shadeMe may also release updates as well, he has access to the http://skyrim.nexusmods.com/downloads/file.php?id=9591 page, so either he or I can update it. The Nexus page also has links to the source code, but here's https://github.com/shadeMe/Nitpick and https://github.com/lojack5/Nitpick, links in case the Nexus is down at the moment.
