Setting a Master to an .esm, TESVSnip, separate ini files fo

Post » Sat Nov 17, 2012 11:55 pm

Hey guys,
So i was trying to set Skyrim.esm to a large world esm I had some help with from this forum.

Problem 1:
I tried TESVSnip - I couldn't open a file with it, any file, I just kept getting the windows error 80040111. I even tried updating my NET Framework and downgrading it to 3.5. Both the options made no difference, of course I cold booted my PC after every change to NET.
How can I fix this? The program starts fine, but I click open... and the error message appears.

Problem 2:
My modded up first install and this newer vanilla Skyrim install that I have share ini files. Is there any way to point them to separate Skyrim.ini and SkyrimPrefs.ini files, specific to each install and not shared with other?

Problem 3:
Then I tried Wrye Bash to set Skyrim.esm as a dependency on my esm. And it worked beautifully!
Without the dependency, the esm would cause a crash when an interior was loaded, exteriors were fine. This is in a new worldspace.
With the dependency, no crashes!
My problem here is, will this cause any unforseen problems down the line?

Problem 4:
How can I cause the player to enter my world when they click "New" at the main menu with my mod loaded.
I was using a changed MQ101, but I'd rather not do this. Is there a workaround?

I'd appreciate any suggestions.
User avatar
Laura Tempel
 
Posts: 3484
Joined: Wed Oct 04, 2006 4:53 pm

Post » Sat Nov 17, 2012 12:29 pm

1- have you done a search for that error-type? I've been using Snip since November without ever seeing any error like that. I've been editing the recordStructure.xml file lately, and when I forget to have correct syntax (like omitting a ">" after a line) it'll not open the app - but I doubt that's your problem. Are you using v4.2? Anything older I really do NOT recommend.

2- I'm not sure what it is you trying to do.. have different INIs for different mods or completely different installs of Skyrim? If for mods, each mod can have it's own INI file, with the name set the same as your plugin's file (eg- myMod.esp, myMod.ini).

3- I don't use WryeBash, but given that it works as it should, it won't cause any problems. Keep in mind that using a plugin as an ESM has specific issues that you need to be aware of - which are beyond the scope of this post.

4- In Oblivion, there was an INI setting one could change that does exactly what you want.. I actually used it to do some architectural design and visualizations (private/non-commercial). I dunno if one can still do it with Skyrim or not. I think they were under [General] SStartingCell=, SStartingCellY=, SStartingCellX=, SStartingWorld=. Lemme know if it works or if you find something that does!
User avatar
Donald Richards
 
Posts: 3378
Joined: Sat Jun 30, 2007 3:59 am

Post » Sat Nov 17, 2012 6:54 pm

1: Not sure, but you don't need TesVSnip for anything. There are alternatives for every feature.

2: Use Mod Organizer, link in my signature. Perfect for managing multiple Skyrim installs, we use it as a requirement for MERP.

3: No that won't cause problems, in fact it is the way to go. Definitely make your worldspace ESM dependent on Skyrim.esm.
User avatar
Jack Walker
 
Posts: 3457
Joined: Wed Jun 06, 2007 6:25 pm

Post » Sat Nov 17, 2012 7:42 am

Problem 1:
I tried TESVSnip - I couldn't open a file with it, any file, I just kept getting the windows error 80040111. I even tried updating my NET Framework and downgrading it to 3.5. Both the options made no difference, of course I cold booted my PC after every change to NET.
How can I fix this? The program starts fine, but I click open... and the error message appears.
Consider it a blessing. You shouldn't be using TESVSnip at all because it has a data truncation bug in the source code that will corrupt the mod you're working on.

Problem 2:
My modded up first install and this newer vanilla Skyrim install that I have share ini files. Is there any way to point them to separate Skyrim.ini and SkyrimPrefs.ini files, specific to each install and not shared with other?
Sounds like you want the http://www.darkcreations.org/forums/topic/1721-relz-sis-skyrim-installation-swapper/.

Problem 3:
Then I tried Wrye Bash to set Skyrim.esm as a dependency on my esm. And it worked beautifully!
Without the dependency, the esm would cause a crash when an interior was loaded, exteriors were fine. This is in a new worldspace.
With the dependency, no crashes!
My problem here is, will this cause any unforseen problems down the line?
There should be no lasting problems from this.

Problem 4:
How can I cause the player to enter my world when they click "New" at the main menu with my mod loaded.
I was using a changed MQ101, but I'd rather not do this. Is there a workaround?
You're stuck with editing MQ101. That's a hardcoded entry point from the executable. You'll need to set things up in the first stage of that quest to jump to your own chargen sequence instead.

It's the sort of thing that makes total conversions a real pain in the ass to make because you have to have the dependency on Skyrim.esm which carries all the baggage with that.
User avatar
Ross Zombie
 
Posts: 3328
Joined: Wed Jul 11, 2007 5:40 pm

Post » Sat Nov 17, 2012 10:26 pm

Sounds like you want the http://www.darkcreations.org/forums/topic/1721-relz-sis-skyrim-installation-swapper/.

(...)

It's the sort of thing that makes total conversions a real pain in the ass to make because you have to have the dependency on Skyrim.esm which carries all the baggage with that.

I consider Mod Organizer superior to the install swapper, far smarter implementation, although the result is slightly different. :)

MERP uses a stripped down Skyrim.esm of about 5 MB in size to get rid of the vast majority of unnecessary 'baggage'. Again, Mod Organizer is genius for such a situation.
User avatar
Big mike
 
Posts: 3423
Joined: Fri Sep 21, 2007 6:38 pm

Post » Sat Nov 17, 2012 12:47 pm

I assumed that the plugin would be skyrim.esm dependent already... it didn't occur to me that you're doing a COMPLETE conversion (opening the CK and building EVERYTHING from scratch, not just creating a new worldSpace using Vanilla resources). But if not, and you're using any resources stored in skyrim.esm (like gate settings, objects pointing to models, actors, etc) - the game crashes when it tries to load an object which doesn't exist.. which it would think is the case if you used a tree from skyrim.esm without the dependency set. That svcks if a true total convert can't work without dependency.. but I guess it's a small price to pay for using the engine heheh.

As for the dramatic issue of Snip; it has yet to be proven that v4.2 causes data corruption if data compression is disabled and you use it with "hex mode" and "use new subrecord editor" both checked (in the Options pulldown). Also, there are some bugs that may cause drama - such as the r-click menu; but knowing what to avoid and doing so will not cause problems. Until the bugs are worked out, or another 3rd party app becomes capable of doing comparable functions, I suggest using it sparingly - and only after reading up on it.. and backing up your data frequently.
User avatar
Loane
 
Posts: 3411
Joined: Wed Apr 04, 2007 6:35 am

Post » Sat Nov 17, 2012 11:17 pm

As for the dramatic issue of Snip; it has yet to be proven that v4.2 causes data corruption if data compression is disabled and you use it with "hex mode" and "use new subrecord editor" both checked (in the Options pulldown). Also, there are some bugs that may cause drama - such as the r-click menu; but knowing what to avoid and doing so will not cause problems. Until the bugs are worked out, or another 3rd party app becomes capable of doing comparable functions, I suggest using it sparingly - and only after reading up on it.. and backing up your data frequently.
We've been over this more times than anyone should need to. http://www.darkcreations.org/forums/topic/1799-skyrim-handling-compressed-records-do-not-use-tesvsnip/. The bug has been clearly identified in the source code. I don't know what more you want out of it than that. The program corrupts, it's NOT SAFE to use, period.

There's already two projects in the works that are making good progress. Patience is the key here, because it's the impatience of us modders that got us all into this mess to start with. Guilty as charged. I used it myself, and paid the price for it.
User avatar
Melis Hristina
 
Posts: 3509
Joined: Sat Jun 17, 2006 10:36 pm

Post » Sat Nov 17, 2012 2:21 pm

SLuckyD, it depends on what kind of data the plugin being saved by TesVSnip contains. I can use it safely to modify our stripped Skyrim.esm because it doesn't contain any world/cell data whatsoever. If your plugin just contains base forms TesVSnip won't break anything.
User avatar
(G-yen)
 
Posts: 3385
Joined: Thu Oct 11, 2007 11:10 pm

Post » Sat Nov 17, 2012 10:47 am

heheheh.. I assume you created a plugin and then just renamed it "skyrim.esm"? Altering the orig in Snip is just BEGGING for trouble... I think it has something to do with the compression used (probably different than what the CK uses, which I actually found to be the case in several specific instances). But disabled compression in Snip really doesn't make that much of a difference when we're talking a couple meg - and loading/resaving in the CK recompresses it properly anyways.

Just like any 'power tool' though.. in the wrong hands or being careless with it could chop off a finger. And by wrong hands, I don't mean you or anyone in particular - just those who don't know HOW to use it to begin with (ultranoobs), or what bugs and glitches to avoid, etc etc. Kinda like using a chainsaw to cut through the metal body of a car; it can be done if you avoid the frame/hard spots and have a decent guard on it for if/when it jumps back at ya heheheh.

[EDIT: but having compression disabled DOES make a difference when you're talking about a total conversion.. which probably takes up 10s of mb or more. One ESM I have is 9 quads and over 200mb.]
User avatar
Oscar Vazquez
 
Posts: 3418
Joined: Sun Sep 30, 2007 12:08 pm

Post » Sat Nov 17, 2012 1:23 pm

I assume you created a plugin and then just renamed it "skyrim.esm"?

No, it is the original Skyrim.esm. I loaded a copy in TesVSnip and removed all the cells and world data, leaving only base objects. With compressions turned on too. I stripped it down to 5 MB only. We have been working with it for many months, everything works perfectly fine.
User avatar
Rob Smith
 
Posts: 3424
Joined: Wed Oct 03, 2007 5:30 pm

Post » Sat Nov 17, 2012 1:41 pm

Thanks to all for the concrete suggestions and explanations! Very helpful, you have no idea.

The team and I are recreating Westeros. Its a total conversion, and we want to create as much new content as we logistically can, and using as much of the assets already present is also important. A balance between the two to keep that new world feel while reducing the metric tonnage of work ahead is the idea.

My concern is this: won't a total conversion mod with a stripped Skyrim.esm as a base for terrain and LOD, when given a dependency to vanilla Skyrim.esm and loaded together with it, cause too much data to be loaded to RAM? Won't that cause more CTDs? Am I understanding this correctly? I realize its necessary to use vanilla assets, but all that extra stuff! The terrain takes up so much space as it is.


For problem 1, I've just read through some of the threads concerning the compression flag problem and data loss. If other programs can do what TESVSnip does, then I'll use them. After all, since starting this total conversion project, every member of the team have had to learn to use a myriad of different apps to accomplish anything at this early stage. I was using 4.2, I tried an earlier version but the problem was the same.

I really hope TESVEdit or TESVGecko are released in beta in a month or so. That would be fantastic considering their predecessors!

Problem 2: I'll give both Skyrim Install Swapper and Mod Organizer a shot. I'm just really happy there are options for this.

Problem 3: Honestly this fix just occurred to me too easily! So, logically I questioned it, especially considering that other victories with the CK and modding TESV have been so hard fought. I can rest easy, and yet my cheeks are still clenched.

Problem 4: Okay... well, it'll do. I really think the BSG Devs should consider allowing the modder community more leeway. Its a little hypocritical to tout the modder community as a pro of TESV but then turn around and handicap that community. I understand, vaguely, some of the legal ramifications. All I'm saying is, it'd be nice to roam around freely without that ball and chain.


Thank you again, and I hope to be running around in Arda in the future as well!
User avatar
Samantha Jane Adams
 
Posts: 3433
Joined: Mon Dec 04, 2006 4:00 pm

Post » Sat Nov 17, 2012 8:56 am

The team and I are recreating Westeros.

(...)

My concern is this: won't a total conversion mod with a stripped Skyrim.esm as a base for terrain and LOD, when given a dependency to vanilla Skyrim.esm and loaded together with it, cause too much data to be loaded to RAM? Won't that cause more CTDs? Am I understanding this correctly? I realize its necessary to use vanilla assets, but all that extra stuff! The terrain takes up so much space as it is.

Another Game of Thrones mod eh. I think I heard of at least two other teams already who are attempting such a thing. Then again, most teams trying a total conversion don't get anywhere, odds are only one of you will actually make it anywhere. Perhaps it'll be you guys.

Yes, the RAM concern is a valid concern, depending on the size of your worldspace. Because MERP has a 14x13 quads heightmap I created the stripped down Skyrim.esm of 5 MB where all Skyrim-specific content is removed. If you want you can have that stripped Skyrim.esm, but realize that we also removed ALL magic from it because Skyrim magic has no place in MERP. It's easy to move back forms though. Like I said earlier, it is save to edit this stripped Skyrim.esm with TesVSnip.

If you start using a stripped Skyrim.esm, I even more strongly recommend Mod Organizer. It makes use of separate folders for every mod, and when executing the game, the CK or another application it virtually merges these mod folders, also using a priority system where the vanilla Data folder has the lowest priority. So, the stripped Skyrim.esm is in a different folder than the Data folder, but when you start the CK/Skyrim ModOrganizer will offer the application the stripped Skyrim.esm. No risk of accidentally overwriting your vanilla Skyrim.esm and no physical movement of any files is necessary to return to vanilla Skyrim. All you need to do is deactivate your mod folder in Mod Organizer. It's a genius system for multiple installs, avoids any (de)installation file conflicts too. Every mod has it's own folder.
User avatar
bimsy
 
Posts: 3541
Joined: Wed Oct 11, 2006 3:04 pm

Post » Sat Nov 17, 2012 8:08 pm

I'd love to have that stripped Skyrim.esm. I'm really curious how you guys set up a base. As to the magic, I was considering using some of those as part of a combat overhaul, not as a magic system. We don't intend on having much magic, if any, in the world so the stripped magic associated forms won't be an issue.

Honestly I'm just really curious about how you have opted to do things, that's all. Avoiding some pitfalls is always a better choice.

Yeah, I know, there is a lot of attention thrown at GOT because of the success of the TV Series.
I'm attached to The Westeros Project, or at least the latest iteration of this team. There was a forum thread a while back. Benfclark is our Lead. I think this time, we've got enough interest to move forward without imploding. Fingers crossed!
User avatar
ijohnnny
 
Posts: 3412
Joined: Sun Oct 22, 2006 12:15 am

Post » Sat Nov 17, 2012 3:43 pm

First things first.... I just posted my definitive rebuttal for anyone claiming Snip destroys data (entirely basing such claims on a completely non-Snip-related forum thread which Arthmoor linked to above). I won't go into it here, as the proper place for such discussion is the actual app's Comment Section. Since v4.3/discon is hidden, I posted it in v4.2 - which is actually the version I use. (although I do have a non-public version of 4.3.1, but only use it for testing, not for actual modding yet)
http://forums.nexusmods.com/index.php?/topic/514199-tesvsnip/page__st__210__gopid__5904107#entry5904107

Now... you guys may be interested to know that I did a little experimenting with the alteration of Skyrim.esm - contrary to my belief that it should never BE altered. I understand that one may want to do so to create a true-total conversion, using it as a base.. which seems to be the case in this thread. What's so interesting in my tests is that I was able to get a custom 'copy' of Skyrim.esm working in the CK and in-game.

I agree that using a copy that has been saved as-is in Snip won't work - it results in no land, and Player and all NPCs plummet to their deaths. I didn't look into why this happens yet, didn't care at the time - I just wanted to see what you guys were talking about.

So I took that Snip-saved skyrim.esm, flipped the ESM byte, and tried to load it in the CK. Returns a bunch of errors and I just clicked "yes to all" just plow through it. When it finally finished loading (and it takes a WHILE), I immediately clicked "save plugin". Again, tons of errors and it takes FOREVER. All said and done, the load/save process took over 30min on an i5 system with 4gig RAM.

Tested it in-game, the land is there, and it's solid - but every actor was stiff as a board, even when walking. So the anims were somehow being interefered with. THEN the game CTD within a few seconds; which can be extended should you COC to an interior - preferably one with nothing that moves. Still not a viable file.

So how did I get it to work you ask? I found the solution, it's the DOBJ record (the only form under the DOBJ GRUP)... CK saved it as a bunch of 0s and added an EDID for no reason. What causes this is that the CK seems not to be able to correctly save certain record types... presumably because what CAN do so wasn't released in the public version (or may even be a separate app).

So to get a working hack, one must use 3rd party software (I used Snip... queue the grimaces of pain). Replacing the napalmed DOBJ with the Vanilla record allows the hack to work in-game. Here's the exact process I used, twice - the second time to make sure it works and to document the errors.

- backed up my Vanilla skyrim.esm (of course)
- renamed my to-edit copy to "skyrim.esp"
- opened it in Notepad++ and changed the 9th byte to hexvalue 80 (81 is localized ESM), saved it (9th byte as in position, starting from 1; it's labelled #8)
- load ONLY skyrim.esp in the CK (no master selected, no master dependency forced anywhere like in Snip)
- every error it returns press "yes to all"
(*** Don't try to force the CK or excessively switch to other windows then back again or you'll disturb the CK's tranquility during the load and probably crash it. Also, I recommend not running anything resource-intensive while doing this)
- when it finishes, immediately save the plugin and exit the CK
- open it in Snip, and in the same session open Vanilla skyrim.esm (whatever you renamed it to)
(alternatively, and faster/safer, you can hack out the needed GRUP to it's own plugin, save it, exit, then load your hacked skyrim.esp with that little interim plugin)
- delete the DOBJ top-level GRUP (in the hacked skyrim.esp)
- copy over the Vanilla DOBJ GRUP
- flip the ESM switch, and save the plugin as skyrim.esm (no master dependency used or needed)

Tested in-game; no CTD, actors walk around, everything seems normal. I even started a 'new' game from the Main/Title Screen (for the first time since November heheh) to test this. Went through the entire drama until I came out the cave - as if I were Plato.

Be aware that I found SEVERAL DIAL GRUPS missing in the hacked version... presumably they'd need to be Snipped over as well; though I suppose they could be recreated by-proxy in a pureCK ESP using the Vanilla Skyrim.esm. I don't know what else is missing, if anything - but the DIAL top-level GRUP is the only one that had a different number of children.. I didn't check the CELL and WRLD subGRUPs as that would take forever. (I'd be curious to know if this DIAL thing is because I was choking the life out of the CK during the process, or because the v1.6 drama related to dialog - the missing GRUPs would explain a lot of people's drama if it is; especially if it's always the same GRUPs or some kind of pattern that can be circumvented)

Potentially important forms that return errors during the load/save process are as follows (appeared in this order): 13, 19, 14a, 14c, a0, 33, 25, 3d, 146, 1f3, 31 (all preceded by the appropriate number of 0s). Since -31 was the last, it was the first that I checked in Snip (with Vanilla sky.esm)... because I'm super lucky, it just happened to be the DOBJ - which is the record enabling a hacked sky.esm. I have yet to check the others as I ran out of time (yesterday), and today I've been running errands, uploading, documenting, and posting so haven't gotten a chance to yet. I'm not going to until Monday either - I'll be in WashDC all weekend for a fracking protest... down with pollution and greed!

Ultimate point - you may very well be able to do what you want easier than you think! If I can pull off a clone of an almost-entire Skyrim.esm (sans some dialog), I see little to no reason a stripped down version of the utmost content wouldn't work.


btw Maegfaer: regarding Snip with your stripped down sky.esm... that working file saved with 'compression on' may only work if those forms you kept don't have compressed records - otherwise it seems you're super lucky as well. I HIGHLY recommend turning compression off for ALL use with Snip. I actually reloaded my hacked sky.esm/esp into the CK, returns a bunch of ref-related errors that I haven't checked into yet (but seem un-crashingly harmless and probably fixable). Saved the plugin (to induce recompression), and it only resulted in like a 10mb or less savings - on a 240mb file... point being that the compression is not NEARLY as effective as I had expected on such a huge file (and really can be foregone in my opinion).

One more thing before I finish - since I don't have time to create the dedicated thread I want to explain/list this... but during the load/save process, a metric flip-ton of navMesh errors are returned; all of which I exhaustively documented (took about 2hours!). These are VANILLA navMesh errors, scattered through the game - and are ignored by the CK because they are in an ESM. The significance is that there are areas which I previously thought I caused errors in (by editing the navMesh, or some other thing I could never identify). (CK trips errors because copies of those nilla navMeshes are now in the ESP)

It's also significant to the complaints about pathfinding breakage we see in the forums. Myself and others have suspected this for the longest time, but now I beleve this is true without a doubt (at least in many/most cases.. there's always operator error to consider). I am going to post the entire list, so that people can check their mods to it... and sleep soundly knowing that THEY didn't frig up the game/mod, or that what they're trying to do isn't working but that it's NOT their fault...

Thanks for listening and see I'll check back in a couple days - unless I'm arrested in DC.. heheheh
User avatar
Jack Bryan
 
Posts: 3449
Joined: Wed May 16, 2007 2:31 am

Post » Sat Nov 17, 2012 1:09 pm

No, the proper place for the discussion is in the community, where it's taking place, and where consensus on the destructive potential of the app is no longer in question except by you. You may see this as bashing you or bashing your methods, but you haven't yet had to suffer having months of your work thrown out the window for no good reason. Several others already have and there is absolutely no excuse to take the risk, nor is there any reason to bow to demands to take the discussion somewhere obscure in hopes nobody will follow it there.
User avatar
megan gleeson
 
Posts: 3493
Joined: Wed Feb 07, 2007 2:01 pm

Post » Sat Nov 17, 2012 10:43 pm

Obscure forum?? That's utterly ridulous! It's the Comment section of TESvSnip v4.2 on the Nexus!! Whereas your "community discussion" is what takes place on obscure forums if at all! Get a grip! (and no... I haven't had months, days, or even an hour's worth of work lost to anything - because I back up my data frequently and know how to properly use the apps that I do. Stop placing blame on everything but yourself... and PLEASE take the discussion to the actual app's thread - where you'll find HARD EVIDENCE to disprove your nonsense and fingerpointing.
User avatar
Liv Brown
 
Posts: 3358
Joined: Wed Jan 31, 2007 11:44 pm

Post » Sat Nov 17, 2012 11:10 pm

Too bad the hard evidence that invalidates what you're trying to convince people of isn't worth spit considering the bug in the source code of the program has yet to be addressed. This isn't a matter of one not knowing how to use the program. This is purely the program itself being at fault for the issues it causes.

The app's comment thread is as obscure as it can get since that will be buried quickly under a mountain of Skyrim posts to the uploads forum. Dark Creations is not an obscure site, and the post that details Snip's severe defects isn't hiding away from anyone there hoping not to get noticed.

It doesn't do the community any good to sit back in ignorance of someone blowing smoke when the underlying reason for the problem has never been addressed. I can't in good conscience say nothing when it *IS* months worth of peoples' work that has already been corrupted beyond repair, and potentially months more if you keep going around claiming it's perfectly safe despite having been shown evidence to the contrary.
User avatar
Sarah Bishop
 
Posts: 3387
Joined: Wed Oct 04, 2006 9:59 pm


Return to V - Skyrim