Yeah, check out:
Edit:

Couldn't tell you for sure off the top of my head. If you give me links to the two, I can probably take a look, but no promises.

For 1., see this post: http://www.gamesas.com/topic/1345724-important-bsas-and-you/page__view__findpost__p__20290392

EDIT: Goddammit.

My filename will now consist of 255 Z's with .esp on the end. What do I get for my flawless victory?

I hate that practice so, so much. Even when it's constrained to just "aa" or whatever. It is not hard to just type the first two letters or so of the mod name and go right to the entry.

Lojack this was an excellent read, but I have a question: If your mod only adds new content to the game, like new creatures, new weapons, new armor, new nifs and new textures, and affects nothing vanilla (you have your own custom folders and everything), can you still use the BSA format without consequence?

Nothing, unless you can come up with a 255 Z's acronym

For the most part yes (see the What This Means For You: Modders section for more detail). Basically, you run into the problem that your new resources can't be overridden, either by an official patch by you, or if someone decides, say to provide an HD texture pack for your mod.

A good example is the... *Looks left... then right...* SpaaAAAAAAAaaaace Core! It is all new content and an *.esp plugin adding it to the world, should always work but probably can't make proper texture replacers for it except only by using another plugin activated *.bsa file which ends later alpha-numeric-wise.

This is one thing and that is another thing. They are not the same thing, you should not use the same terms. It is ambiguous and confusing. Ignoring this sort of "lazy language" is exactly how you wind up in scenarios where the same term refers to two separate things, which leads to confusion and redundancy in language.

This is a technical hobby. You are responsible for having a technical understanding of what is going on, and using terms correctly.

Archive Invalidation was an unreliable method of creating a list of what should be overridden in a BSA.

ArchiveInvalidationInvalidated was an improvement on this method by having a dummy like BSA that redirects to the loose files.

Now we have it that BSA files win over loose files on one hand and on the other we say that Archive Invalidation is no longer an issue with Skyrim (I've been directly told this). That, to me, seems like contradictory information that also will lead to more confusion.

If then it is that we should not refer to the problem of BSA files winning over loose files as an issue of Archive Invalidation (admittedly used loosely to mean any invalidating of what is in a BSA) then what term should we use? I was simply pointing out that this is how it is understood by many people and I doubt that they are confusing the terms as much as technicians are by describing the same mechanism of a problem then stating that we should not use the same term.

Your posts often bare the edge of an axe that constantly needs grinding. I made that post to attempt to show that I don't think the terms are as confusing as some make them out to be. While Archive Invalidation was a specific method for dealing with the same problem - I think people are referring to the underlying problem and not the specific method for dealing with it.

What then is the name for the underlying problem? Constant Archive Supremacy? Archive Validation?

Or:

Registered BSAs in registration order
Everything else in date order. BSA files that match a plugin's name use the plugin's timestamp.

But note that this is an issue because of how steam subscriptions drop files directly in the data directory. If we could tell steam to drop our subscriptions in some other folder, then mod managers could repack things at extraction time (which has been how they have always worked).

Note also that this is also only an issue for mods with conflicting files. Another approach would to have an alert system that tells you about mod conflicts, and then either refuse to use two different mods that supply the same files in a plugin+.bsa pair or live with one of them randomly (depending on last update time on your subscription) overriding the other. This also has some similarity with how mod managers have been used. And, currently, it looks like Steam Workshop would work well enough with people downloading mods, repackaging them and re-uploading them. I do not know if the situation will remain like this, but currently that's a plausible option.

Note also that the symptoms of this problem will eventually get so bad that Valve will have to address it. Worst case, we might need to wait until some high profile problems arise that hit a lot of people. (Once a problem hits a lot of people it will be hard to ignore, and while some problems can be managed by working around the symptoms, a high profile problem is harder to treat in that fashion.)

Note also that if this can be resolved as a Valve problem we will probably see a fix much sooner than if it's treated as a Bethesda problem:

If it's a Bethesda problem then Bethesda is would have to change their file loading code, and the problem there is that this is a real time system which blows up if queues get too deep. So whatever they do, they have to invest significant effort into making sure that none (or almost none) of the slowest systems choke. In other words, lots of testing pre-release and/or lots of unhappy customers after release.

If it's a Valve problem, then all Valve needs to do is add an option for subscriptions (on a per game target basis) to land in an alternate directory. It's probably just a checkbox (with some delay after you check it for files to get moved). Simple to implement, optional to use, so low impact from any problem cases.

How about LFLTLBF
Loose files lose to loose bsa files

Registered bsa files - so any standard skyrim one - is properly overwritten by loose files (so no Archive Invalidation is necesarry)
bsa that belong to esp overwrite loose files, so here we would have to make the in bsa version unusable (which kinda is what OMMs archive invalidation did)

@r.d.m. - took a little more though into an idea you brought up before.

I think some simple filesystem hacks might make a very viable solution to the whole thing.

This solution would work like so:

First, parse Skyrim.ini to determine which BSAs are registered.
Now, intercept the filesystem calls from Skyrim. Anytime it tries to access a BSA, allow it if the BSA is registered, but tell the game the BSA doesn't exist otherwise (hide Plugin BSAs from the game).
Finally, parse the active Plugin BSAs for the contained files. When Skryim parses the Data directory for file listing, return the interlaced resultes of the actual files, and the files in the Plugin BSAs. And when Skyrim tries to load said files, instead redirect it to the BSA containing it.

Plus we can design this utility to be able to control BSA Load Order in a simple way. No Modification Time nonsense or anything. It could be configurable via INI or other similar config file.

I'm liking this approach best so far, as it won't require any modification to the game's code, so future patches won't break it either.

Except if someone releases a patch for a 3 GB BSA, I'd not want to have to redownload a 3GB file to fix a 20KB texture or mesh. It's not feasable. I think Loose files should trump BSAs in all cases, and BSAs that aren't registered need a better way of loading.

Ah, OK, I see the issue. You are not merely being "lazy" in your speech, you are actually misunderstanding the issues. If your understanding were correct, I'd agree that there are enough similarities to warrant using the same term.

OK, so, for an overview of why this is not ArchiveInvalidation, or an ArchiveInvalidation-related issue:

First, ArchiveInvalidation referred to a specific line in Oblivion.ini that no longer exists in Skyrim.ini. This may seem like pedantry, but it's not, as I'll get to.

Archive Invalidation, or the problem that ArchiveInvalidation attempted to fix, was the inability in certain circumstances to have loose files override, primarily, native BSA files that came with Oblivion. And, more importantly, when you say "the problem with ArchiveInvalidation" was not so much that what was intended by AI wasn't what we wanted, it was that the implementation of ArchiveInvalidation was buggy and did not actually take effect. Setting ArchiveInvalidation=1 in Oblivion.ini was supposed to fix issues relating to how BSAs were loaded, but did not do so, hence the need for redirection, AII, and so on.

Importantly, some sort of ArchiveInvalidation fix was necessary to run any sort of replacer mod in Oblivion. When you say ArchiveInvalidation to me, this is what you mean.

Skyrim has no such problems.

The native BSA files that are installed with Skyrim are registered in Skyrim.ini, and therefore lowest on the priority tree. Loose files do overwrite them, as intended by replacement authors. ArchiveInvalidation, which was necessary in Oblivion to use any replacer mod, is not necessary to use with Skyrim.

Moreover, there is no "ArchiveInvalidation" per se in Skyrim, and therefore no buggy implementation thereof.

Hence, the difference.

The only time that there is a problem is when two separate unregistered (read: mod-added) BSAs contain the same file. This is a condition that didn't come up in Oblivion (or, at least, was not worried about beyond the usual ArchiveInvalidation issues), because all BSAs were treated the same and because ArchiveInvalidation didn't work anyway. Hence, this is a completely separate issue.

If it weren't for Steam Works, this issue would probably not even be that serious; BSAs weren't used that often by mod authors, and had their own problems anyway.

But now Steam Works mandates the use of BSAs, so we have to deal with this new issue with how BSAs are loaded. Obviously, both ArchiveInvalidation and this issue have to do with how BSAs are loaded, but the issues plaguing each game, the circumstances in which there are problems, and the solutions to be employed are all very distinct. Conflating them makes nothing easier to understand, because people's responses will have to be different.

Moreover, the general sentiment that "Bethesda lied to us about ArchiveInvalidation being unnecessary" is fundamentally flawed. That statement was effectively true. Bethesda has always been at least somewhat short-sighted when it comes to mods: they think in terms of one mod modding the game, not in inter-mod interaction. They are aware of this tendency in general, I believe (I don't know if any of them are yet aware of this particular issue), and the statement is just one more example of that. ArchiveInvalidation, and ArchiveInvalidation fixes, were necessary in Oblivion to use replacers. They are not necessary to use in Skyrim for replacers, only for conflicting replacers. Which wouldn't have even been a very big deal, except for Steam Works.


Ultimately, ArchiveInvalidation refers to a bug-riddled intended feature in Oblivion. This issue is not a feature that has bugs, but rather a poorly-designed but well-functioning feature. We don't need them to fix bugs, we need them to change their design.


I don't have a particularly good name for this feature. I hope that Bethesda will fix it shortly, so we won't have to have a name for it.

Unregistered Archive Priority works for me. UAP is to the point and accurately descriptive of the problem.

No man I get it - I really do. http://www.gamesas.com/topic/1338421-relz-wrye-bash/page__view__findpost__p__20270300 just about all the findings in this thread (before the thread was posted) except what the actual order was for how BSAs win between themselves. That was beyond my patience level.

So I've been using the term 'Archive Invalidation' as shorthand to mean any need to override a BSA by any loose file - whether that be a BSA registered in the ini or via a loading plugin. To me it seemed to be the same dynamic. To say that Archive Invalidation (the problem not the method) is not an issue (what Lojack told me) isn't that correct when it still is a problem with regard to esp loading BSAs.

The problem is the same problem and all that Bethesda did was handle it for the registered BSA and not the plugin loaded ones.

So I'm fine with the name for the problem being Unregistered Archive Priority (UAC since modders love acronyms - totally to be confused with User Account Control). Or whatever other name I was just trying clarify that when I and a few others said Archive Invalidation problem it was referring to the problem of needing to override and not the methods that lead to that success (however partial). So the laziness was born out of a lack of a better term.

And yes this needs to be addressed by Bethesda before tool makers create more than needed.

Did the construction set package mods as BSAs? If it did I never used it and no one ever used that feature lol. I actually don't like the thought of it. Maybe its because of my issues with skyrim HD and the DLC (which I completely unpacked and installed manually)...

A nice smackdown by a file with "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~....esp" (or ?)

(Oh my now they will make Workshop mods with the tilde, damn.)

Nope.

mmmm..... in another thread I was told that loose files win over the BSA, but now here see that

Registered BSA file wins over the loose files if it is listed in Skyrim.ini.

So if I want to unpack all my Vanilla BSA files and make sure the game only uses the loose files (not the BSA) I need to delete that line (the list of BSA files to be loaded) in the Skyrim INI ?

I think you've got it backwards. Loose BSA files associated with a plug-in override loose files. Loose files override BSAs in your ini.

Registered BSAs are lowest priority - they will get overridden by Loose Files.

It's the Plugin BSAs that are overriding everything. It's fine to leave Registered BSAs intact.

Edit:

Does anyone know the REASON that SW is forcing modders to use .bsa's? Doesn't seem necessary and that seem to be the root of this evil.

Rather than actually writing a real installer code they just drop a bsa file in your data folder, that way uninstalls are a piece of cake.

@Lojack, http://img267.imageshack.us/img267/6914/p1010829l.jpg

Makes sense. Rather pathetic, also.