[!IMPORTANT] BSAs and You

Post » Thu May 24, 2012 11:42 am

Makes sense. Rather pathetic, also.
Yup, especially considering that they could have easily asked the Wrye team or the NMM team for advice. Or were you referring to my complete lack of muscle tone. :P
User avatar
c.o.s.m.o
 
Posts: 3419
Joined: Sat Aug 12, 2006 9:21 am

Post » Thu May 24, 2012 9:03 am

Yup, especially considering that they could have easily asked the Wrye team or the NMM team for advice. Or were you referring to my complete lack of muscle tone. :tongue:
Foam covered nunchaku FTL man.
User avatar
jessica robson
 
Posts: 3436
Joined: Mon Oct 09, 2006 11:54 am

Post » Wed May 23, 2012 9:57 pm

Foam covered nunchaku FTL man.
Wood ones are illegal in Baltimore. :(
User avatar
Ashley Hill
 
Posts: 3516
Joined: Tue Jul 04, 2006 5:27 am

Post » Thu May 24, 2012 5:50 am

I'm hoping someone can help me out here.

My mod works perfectly when my meshes and textures are ran as loose files.

However, I packaged them with the Creation Kit to release on the Workshop.

I tested the mod after removing the loose files (these are new meshes and textures, they don't replace existing files).

For some reason, the game only reads partial information from the bsa. It will read some meshes (in this case: item meshes inside the menu) and fails to read others (in this case: the actual mesh that should appear on the character.) Instead of having the cloak appear on my character's head, his hair disappears (as it should if the mesh appeared) with no cloak to be found. I've checked, double checked and triple checked all mesh paths inside the esp (in the Creation Kit) and all should work. The first time I packaged the bsa, one of the 3 hood meshes was working. The other two not. I deleted the bsa, repacked it, and now none of the 3 work. But it does read the bsa, because it sees the item preview meshes in the menu.

I'm at my wit's end and hoping someone can tell my why on earth this happening @_@
User avatar
Brandi Norton
 
Posts: 3334
Joined: Fri Feb 09, 2007 9:24 pm

Post » Thu May 24, 2012 9:59 am

So I've been using the term 'Archive Invalidation' as shorthand to mean any need to override a BSA by any loose file
Then you are wrong and should stop. That doesn't even make sense.

To me it seemed to be the same dynamic. To say that Archive Invalidation (the problem not the method) is not an issue (what Lojack told me) isn't that correct when it still is a problem with regard to esp loading BSAs.

The problem is the same problem and all that Bethesda did was handle it for the registered BSA and not the plugin loaded ones.
Every statement here is wrong, with the exception of the one beginning with "to me" — in which case, you are accurately reporting your incorrect perception.

They are not the same. It is a different mechanic, with a different "dynamic". The causes are different, the effects are different, and the solutions will be different. That's about as different as things get.

So I'm fine with the name for the problem being Unregistered Archive Priority (UAC since modders love acronyms - totally to be confused with User Account Control).
Wut. How do you get "UAC" from "Unregistered Archive Priority"? There's only one 'c' in the entire string, and it's in the middle of the second word. This statement is just mind-boggling.
User avatar
Elizabeth Falvey
 
Posts: 3347
Joined: Fri Oct 26, 2007 1:37 am

Post » Thu May 24, 2012 11:57 am

Umm yeah I did get the UAC thing wrong - misread his post/brain not working. :confused:

But how is the need to override BSA files (from plugin loaded BSA) with loose files in this case (Skyrim) all that different than the need to override BSA files (both registered and loaded with a plugin) with loose files in previous cases (Oblivion and Fallouts)?

How is that issue not similar if not damn near identical? Even in the light that there now is a logic of how BSA files win out amongst other BSA files.

Why isn't it that with Skyrim the developers have seemed to address the need to let loose files win without AI or BSA redirection with only registered BSA - but perhaps overlooked it with plugin loaded BSA?
User avatar
Michelle Chau
 
Posts: 3308
Joined: Sat Aug 26, 2006 4:24 am

Post » Thu May 24, 2012 11:18 am

Does anyone know the REASON that SW is forcing modders to use .bsa's? Doesn't seem necessary and that seem to be the root of this evil.
The only thing I can think of is that Skyrim ships with some loose files in the data folder. Correct me if I'm wrong (usually am) but these include .STRINGs and stuff in the \scripts folder (which would be scripts, yes/no!?!)
So installing then uninstalling a mod with these as loose files using NMM or BAIN will mean you have them removed from the game entirely!
THIS might be an issue perhaps?
User avatar
koumba
 
Posts: 3394
Joined: Thu Mar 22, 2007 8:39 pm

Post » Thu May 24, 2012 2:31 am

Here's a fix for the INI string value character limit - http://dl.dropbox.com/u/2584752/Nitpick.dll. It's basically been increased to 32KB.
User avatar
Tessa Mullins
 
Posts: 3354
Joined: Mon Oct 22, 2007 5:17 am

Post » Thu May 24, 2012 9:41 am

Here's a fix for the INI string value character limit - http://dl.dropbox.com/u/2584752/Nitpick.dll. It's basically been increased to 32KB.
Getting a 404 error from that link, fyi
User avatar
LittleMiss
 
Posts: 3412
Joined: Wed Nov 29, 2006 6:22 am

Post » Thu May 24, 2012 2:39 am

That period got included in the link text - Fixed.
User avatar
Roanne Bardsley
 
Posts: 3414
Joined: Wed Nov 08, 2006 9:57 am

Post » Thu May 24, 2012 5:43 am

The only thing I can think of is that Skyrim ships with some loose files in the data folder. Correct me if I'm wrong (usually am) but these include .STRINGs and stuff in the \scripts folder (which would be scripts, yes/no!?!)
So installing then uninstalling a mod with these as loose files using NMM or BAIN will mean you have them removed from the game entirely!
THIS might be an issue perhaps?
I don't think so. Both NMM and BAIN only remove what was added by a particualr mod. Wiz0flyd had it right.
User avatar
Jonathan Egan
 
Posts: 3432
Joined: Fri Jun 22, 2007 3:27 pm

Post » Thu May 24, 2012 10:27 am

I'm hoping someone can help me out here. My mod works perfectly when my meshes and textures are ran as loose files. However, I packaged them with the Creation Kit to release on the Workshop. I tested the mod after removing the loose files (these are new meshes and textures, they don't replace existing files). For some reason, the game only reads partial information from the bsa.

Is it possible that your mod worked when you had loose files because some of them were being overridden by another mod's bsa?
User avatar
Marina Leigh
 
Posts: 3339
Joined: Wed Jun 21, 2006 7:59 pm

Post » Thu May 24, 2012 12:05 pm

But how is the need to override BSA files (from plugin loaded BSA) with loose files in this case (Skyrim) all that different than the need to override BSA files (both registered and loaded with a plugin) with loose files in previous cases (Oblivion and Fallouts)?
Because AI was necessary to deal with replacers in Oblivion. It is not necessary for replacers with Skyrim. The issues that Skyrim has, are issues that Oblivion never had. They have different causes, effects, and solutions. They are different issues. Just because they both have to do with how BSAs load does not make them the same.

You are wrong. You are wrong on an objective and technical level. You need to accept this and move on; you are only drawing out an unnecessary argument, and your continued conflating of terms would only confuse people. Again, I ask you, please stop.
User avatar
Justin Bywater
 
Posts: 3264
Joined: Tue Sep 11, 2007 10:44 pm

Post » Thu May 24, 2012 3:35 am

Does anyone know the REASON that SW is forcing modders to use .bsa's? Doesn't seem necessary and that seem to be the root of this evil.

Pretty much the same reason they do not allow esm files --- they want to keep the format simple so that their "One button Download and install" partially works rather than actually spend some time creating a program that can do the job since as everyone that has been modding for awhile has tried to warn others that a One button installer is just not going to work for anything but the simplest mods.
User avatar
Patrick Gordon
 
Posts: 3366
Joined: Thu May 31, 2007 5:38 am

Post » Thu May 24, 2012 2:53 am

Vote sticky! thanks for all of this, makes TONS more sense now :). Hopefully a solution can be implimented indeed.
User avatar
Kelly Upshall
 
Posts: 3475
Joined: Sat Oct 28, 2006 6:26 pm

Post » Thu May 24, 2012 6:27 am

Here's a fix for the INI string value character limit - http://dl.dropbox.com/u/2584752/Nitpick.dll. It's basically been increased to 32KB.

Drop this into the root skyrim directory, yes?

Is this for a specific ini or all of them?

Could this be converted into a SKSE plugin perhaps?

Love the name. :P
User avatar
Adam Baumgartner
 
Posts: 3344
Joined: Wed May 30, 2007 12:12 pm

Post » Wed May 23, 2012 10:59 pm

@r.d.m. - took a little more though into an idea you brought up before.

I think some simple filesystem hacks might make a very viable solution to the whole thing.

This solution would work like so:

First, parse Skyrim.ini to determine which BSAs are registered.
Now, intercept the filesystem calls from Skyrim. Anytime it tries to access a BSA, allow it if the BSA is registered, but tell the game the BSA doesn't exist otherwise (hide Plugin BSAs from the game).
Finally, parse the active Plugin BSAs for the contained files. When Skryim parses the Data directory for file listing, return the interlaced resultes of the actual files, and the files in the Plugin BSAs. And when Skyrim tries to load said files, instead redirect it to the BSA containing it.

Plus we can design this utility to be able to control BSA Load Order in a simple way. No Modification Time nonsense or anything. It could be configurable via INI or other similar config file.

I'm liking this approach best so far, as it won't require any modification to the game's code, so future patches won't break it either.

I thought about this solution but assumed it was impossible or at least impractical. How would the "redirect to the BSA" part work? Maybe you could memory-map the relevant part of the BSA and return the handle to skyrim, but what if the file is compressed? Either you'd have to extract the file to a temporary location, in which case you could just as well extract the BSA at startup and treat everything as loose files, or uncompress on-the-fly which I'm not sure is possible in a way transparent for skyrim.

Here's a fix for the INI string value character limit

Awesome work! Do you mind telling us how you did that? From what I could tell the buffer is allocated on the stack so it shouldn't be possible to change the buffer size without modifying Skyrim code. If you did that, your dll needs to be updated everytime Skyrim gets updated, correct?
Also, have you verified Skyrim actually uses more than 256 bytes from that buffer?
User avatar
Ashley Clifft
 
Posts: 3468
Joined: Thu Jul 26, 2007 5:56 am

Post » Wed May 23, 2012 10:11 pm

Could this be converted into a SKSE plugin perhaps?

The Nitpick.dll already is a SKSE plugin ;)
User avatar
Erika Ellsworth
 
Posts: 3333
Joined: Sat Jan 06, 2007 5:52 am

Post » Thu May 24, 2012 4:24 am

Drop this into the root skyrim directory, yes?

Is this for a specific ini or all of them?

Could this be converted into a SKSE plugin perhaps?

Love the name. :tongue:
Ah, apologies for not being clear. It's a SKSE plugin; drop it in your Data\SKSE\Plugins folder.


Awesome work! Do you mind telling us how you did that? From what I could tell the buffer is allocated on the stack so it shouldn't be possible to change the buffer size without modifying Skyrim code. If you did that, your dll needs to be updated everytime Skyrim gets updated, correct?
Also, have you verified Skyrim actually uses more than 256 bytes from that buffer?
https://github.com/shadeMe/Nitpick. And yes, the plugin will need to be updated for every new build of the runtime. The setting class dynamically allocates its buffer for strings, so no worries there.
User avatar
Floor Punch
 
Posts: 3568
Joined: Tue May 29, 2007 7:18 am

Post » Thu May 24, 2012 10:02 am

Here's a fix for the INI string value character limit - http://dl.dropbox.com/u/2584752/Nitpick.dll. It's basically been increased to 32KB.

Very nice! I'll update the OP with clear links to this as a workaround to the problem. Do you mind if I upload it to Skyrim Nexus?

I'm thinking with some tweaking we can make this work without SKSE (those tweaks I mentioned in my PM), so it won't have to get rewritten on every new patch.

I thought about this solution but assumed it was impossible or at least impractical. How would the "redirect to the BSA" part work? Maybe you could memory-map the relevant part of the BSA and return the handle to skyrim, but what if the file is compressed? Either you'd have to extract the file to a temporary location, in which case you could just as well extract the BSA at startup and treat everything as loose files, or uncompress on-the-fly which I'm not sure is possible in a way transparent for skyrim.
Yeah, the compressed BSAs will be the hardest to deal with, though internally it'll be similar to how the game handles them anyway - decompress on the fly. Uncompressed will be easier - just memory map the files (the game does that for BSAs already) and pass the portion of the BSA that holds the uncompressed file.


Edit: Ok, OP has been edited with instructions on how to use shadeMe's SKSE plugin.
User avatar
Marion Geneste
 
Posts: 3566
Joined: Fri Mar 30, 2007 9:21 pm

Post » Thu May 24, 2012 5:03 am

https://github.com/shadeMe/Nitpick. And yes, the plugin will need to be updated for every new build of the runtime. The setting class dynamically allocates its buffer for strings, so no worries there.

Very cool, thank you!
User avatar
Julie Ann
 
Posts: 3383
Joined: Thu Aug 23, 2007 5:17 am

Post » Thu May 24, 2012 5:55 am

Cheers to the OP.
User avatar
ladyflames
 
Posts: 3355
Joined: Sat Nov 25, 2006 9:45 am

Post » Thu May 24, 2012 3:09 am

Because AI was necessary to deal with replacers in Oblivion. It is not necessary for replacers with Skyrim. The issues that Skyrim has, are issues that Oblivion never had. They have different causes, effects, and solutions. They are different issues. Just because they both have to do with how BSAs load does not make them the same.

You are wrong. You are wrong on an objective and technical level. You need to accept this and move on; you are only drawing out an unnecessary argument, and your continued conflating of terms would only confuse people. Again, I ask you, please stop.
As usual you don't get what it means to have a regular conversation. I'm not trying to draw an unnecessary argument out. I'm not even trying to have an argument - you are. With your oh so predictable commands for people.

Lojack - sorry for any of this being off topic. But I asked several times exactly how the dynamic of needing to override BSAs with loose files is not a need for invalidating archives with no real answer other than it is not exactly like the other game and so no terms approaching the other game's issue should be used and to do so is criminal or some other nonsense. I was actually answering you and should have known better than to dialogue with that guy again as he always focuses more on browbeating.

I'm past the point of caring if anyone else gets confused over the terms like I do and what (if anything) is the real difference, or if a better name is arrived at. I am glad that solutions are being worked on - but encourage continued efforts to let the developers know - and maybe let us know if you get a reply from GStaff. Thanks

[edit] to be clearer - I'm not confused over why this is not ArchiveInvalidation (the method) ... that is as obvious to see as the nose on my face. I simply posted to state that archive invalidation was a similar problem (the need to invalidate what is in an archive with loose files). It was never my intention to make it a new word to label this problem. DragoonWraith just has issues with me and jumped on that conversation to exercise these issues. I repeat - I'm not trying to advocate for this to be the label. So much for open discussion of ideas.
User avatar
Marine Arrègle
 
Posts: 3423
Joined: Sat Mar 24, 2007 5:19 am

Post » Wed May 23, 2012 9:30 pm

Very nice! I'll update the OP with clear links to this as a workaround to the problem. Do you mind if I upload it to Skyrim Nexus?
Not at all, go right ahead.
User avatar
Angel Torres
 
Posts: 3553
Joined: Thu Oct 25, 2007 7:08 am

Post » Thu May 24, 2012 2:50 am

I'm hoping someone can help me out here.

My mod works perfectly when my meshes and textures are ran as loose files.

However, I packaged them with the Creation Kit to release on the Workshop.

I tested the mod after removing the loose files (these are new meshes and textures, they don't replace existing files).

For some reason, the game only reads partial information from the bsa. It will read some meshes (in this case: item meshes inside the menu) and fails to read others (in this case: the actual mesh that should appear on the character.) Instead of having the cloak appear on my character's head, his hair disappears (as it should if the mesh appeared) with no cloak to be found. I've checked, double checked and triple checked all mesh paths inside the esp (in the Creation Kit) and all should work. The first time I packaged the bsa, one of the 3 hood meshes was working. The other two not. I deleted the bsa, repacked it, and now none of the 3 work. But it does read the bsa, because it sees the item preview meshes in the menu.

I'm at my wit's end and hoping someone can tell my why on earth this happening @_@
FYI for anyone curious, Northborn figured the answer to this one out, see http://www.gamesas.com/topic/1347480-bsa-issue-skyrim-seems-to-only-read-part-of-the-file/page__view__findpost__p__20310780.

@DW: No offense, but you do come off as rude quite often (nature of the internet, I suppose), but I understand your intent.
@Psymon: DW's just trying to be sure that you really truly understand the reason why this isn't Archive Invalidation, and that calling it that will only cause confusion. Both similar topics dealing with similar issues, but the underlying mechanisms that cause it are very different. And as a result, the fixes required are very different - this point here is why you don't want to confuse the terms. If people get in the habit of calling it Archive Invalidation, they'll think the same techniques used to solve it in Oblivion will work in Skyrim - and they won't. Anyway, it seems you at least understand his points now, even if you don't agree, so let's call this discussion complete?

Not at all, go right ahead.
Thanks, Nexus link is http://skyrim.nexusmods.com/downloads/file.php?id=9591 (you have editing access).
Seems like the Nexus is under heavy load right now though. Took multiple attempts to get this up there.
User avatar
Chase McAbee
 
Posts: 3315
Joined: Sat Sep 08, 2007 5:59 am

PreviousNext

Return to V - Skyrim