Here's the explanation of record compression and the extra data:

A Skyrim plugin is composed of a plugin header which contains a Record with the plugin info and a series of Groups. A Group has some info fields and a set of of sub-Groups or Records. A Record contains some info fields and a set of Fields. A Field is the lowest type of element in the tree, and contains data in a variety of formats depending on context. Both the Group and Record have a set of flags which control their use - the Group flags primarily indicate what type of Group it is while the Record flags are a combination of application indicators and modifiers indicating how the Fields are to be processed. One flag, 0x00040000, indicates that the Field list is compressed. In this case the "data" area contains a four-byte "size of decompressed data" and the compressed stream. We discovered that sometimes the actual decompressed data is longer than the size indicated. The extra data do not appear to be more Fields. We had thought that an undocumented flag - 0x02000000 - was a warning of this but that turned out to be unreliable. TESVGecko will retain the entire decompressed data stream even though we don't know what it is for.

If a plugin utility drops this extra data, it is gone from the plugin. Opening the plugin in the Creation Kit will not necessarily restore this information. Also, if the compression flag is set, then the data must be compressed. Having the compression flag set but providing uncompressed data will cause issues, likely crashing the game or the Creation Kit. We will not be flipping this flag in TESVGecko. If the flag is set, then the data will be recompressed when the plugin is saved.

That's the general overview. We'll have to go through all the records again to identify which ones have the compressed data and in particular the mismatch between stated size and the actual size. Currently TESVGecko doesn't handle different record types any differently - we just list the records in the format they are found in the plugin, so I don't know which records are affected by this issue.

Arthmoor: I enjoy a good debate, but 'parliamentary procedure' doesn't become us (as in mud slinging, insinuations, etc). Adrenaline is one of the most evil things on the planet in my opinion (which I obviously succumb to on the rare occasion.. sorry about that little dig I made).

AndalayBay: I appreciate you filling in a bit more info about this drama. I like to believe that data-anolysis is one of my strong points; though until now it hasn't been applied toward raw programming data, but business. It didn't take me but a couple hours to completely decode the NAVM record and the NVMI subrecord (my results are posted in the NavMesh Bug thread, #4 I think it is)... though as stated, it probably has little value other than to actually know what it all does (and whether it's been corrupted somehow I presume).

My point is that I'll take a look at the NPC_ data and see what I can discern. I'll be on holiday this coming week or two (depending on the weather I suppose heheh)... so I won't be able to post/check too often. Here's the link to the page with the navMesh data format info... that is if you guys don't already have it incorporated into Gecko. There are two posts... one for the NAVM then one further down the page with the NVMI.... just look for my infamous WALLS of text.
http://www.gamesas.com/topic/1367191-navmesh-bug-discussion-thread-4/page__st__90

I've bookmarked your post already actually - I saw that before. NAVM might be one of the records with this extra data - I'm not sure. As I said, with the way we are processing the records right now, we don't care which group or record it is. That will change as we offer customized formatting for some records.

My Gecko partner, who's also my husband, is a data processing expert, having been doing it professionally for more than 30 years. I've been doing it for 22 years. Yeah, we're old. Anyway we'll welcome any input on decoding the record structures. We'll need to be intimately familiar with the structures and the rules surrounding their construction for Gecko.

Here's the Gecko WIP http://www.gamesas.com/topic/1378818-wipz-tesvgecko/ on Beth if you'd like to post your findings there. The main development http://www.darkcreations.org/forums/topic/1536-wipz-tesvgecko/ is on Dark Creations.

Btw: we did go through the Snip source code and found the error. We didn't draw our conclusion from supposition - we found the error in the code. We tried to use the code to anolyze the data and discovered that the code didn't match the data. We also grabbed the Skyrim.esm and re-saved it. I was able to load it up in-game, but there were large blocks of land missing. My character and all nearby NPC's fell through the void into water.

(WARNING: incoming wall of text... avert your eyes and nasty commentary if this is adverse to you. Also, this info was gleaned by whilst running TESvSnip v4.2, Skyrim v1.52605, and using Total Commander v7.5 to independently compare/view)

[I tried to set up an account on DarkCreations, but apparently it didn't like literal answers to the questions (2 heads/male, breathing both air and water).. heheheh. The link to TermsOfService is the same as the registration page too... couldn't find the real one (yes, I actually read those things). I'll have to go back to it some time.]

But anyways... I did get a chance to do some digging into the hot issue of this and those other threads (regarding corruption of data). I found WAY more than I expected.. some may say "I told you so", though some of you may not like the exact results. I plan to create a new thread, here on Ma Beth's forum, dedicated to the discussion and education of this... I'll probably call it "data corruption while using the CK and 3rd party apps" (or something to that effect). It may have to wait until Tuesday (I'm on holiday and don't have decent wifi nearby, nor as much ambition heheh).

[And YES... I know the CK corrupts data under very specific circumstance, and much of it does NOT require an engineering degree to reproduce. I also stopped using FO3Edit in the preCK days because it wreaked so much havoc on Skyrim plugins... definitely a 'view-only' app, even then I only used a backup ESP while viewing. Hence the all-encompassing '3rd party' disclaimer. Semi-non-related: I don't know why anyone would try to alter Skyrim.esm in any way, or what purpose that could POSSIBLY serve, but that's asking for drama by itself.... and is a separate issue.]

In a nutshell, here's a summary of SOME of my findings... keep in mind that I have HARD DATA to support this, and will post it in the dedi-thread once created.

**** #1 most important - Snip DOES play a part in corrupting data, but isn't solely responsible for it. It also only happens in SPECIFIC circumstances... namely: SNIPPING SOMETHING OUT OF SKYRIM.ESM, then opening and resaving in the CK. I have yet to see how far this goes, but messing with NPCs... this holds true. It messes with the VMAD and FULL subrecords; though some/little of the damage is reversible (during my limited testing). If the plugin is never loaded and resaved in the CK, it actually works as it should.

* I found that a Skyrim.esm-Snipped actor, once loaded and resaved in the CK, will NOT appear in-game... but that's not all. It also 'swapped out' with my pureCK actor... how, I have no idea; I haven't gotten that far yet. (as-in, I placed one ref of each NPC, but the normal one was literally swapped with the Snipped one, details forthcoming in the other thread) The data comparison shows 'flags and unknowns' to be changed, but nothing else (certainly nothing of obvious importance, like body geometry or stats).

I assume this is what causes some people's issues with Snip - that they spliced something directly out of Skyrim.esm instead of doing a legit clone in the CK (then altering whatever in the Snip.. which works perfectly, every time I tested). This does NOT apply to records Snipped from one mod to another... given those records were originally pureCK-created. Snip does NOT corrupt these records whatsoever... NPC_, CELL, WRLD, GRUP, REFR are some that I anolysed.

#2 very important as well: As I said before (and now have hard evidence to support it), Snip decompresses the plugin and saves it as such. No matter WHICH settings/options I tried, I could NOT get Snip to save a compressed record whatsoever. I found that by setting that flag, Snip SHOWS it's been changed (when viewed in the same session), but that change is never actually SAVED in the data. I don't know what you guys did to get whatever corruption/compression/etc... but let me know so that I can see for myself (and then avoid it in the future).

#3 - Comparing a file that was Snip-altered (opened, add/delete a record, saved in Snip) vs a Snip-altered file that was THEN loaded in the CK (though resaving does NOT have to occur, happens as soon as you click 'ok'); I found that the ONLY difference was the 'File Version'... which CK changes to .85 (and the obvious/expected change in record-count). This does NOT happen before CK tries to load it, and there is NO difference before that. It also appears to have no effect whatsoever if LEFT as .85. See for yourself, and post back your proof if you're gunna dispute that.. my proof will be in the future thread.

#4 - It doesn't really matter because of Gecko's imminence, but I found an instance that Snip's code reads and displays the wrong data-block.. the LAND record shows the VCLR subrecord's size instead of the VNML subrecord's size (no big deal at all). The actual data and size-bytes are correct, so this is almost certainly a superficial bug. I also know that Snip will display certain data-blocks wrongly; such as showing a float when it should be an int... again, no big deal - and the actual data is still correct.

BUT - in discovering this, I found something VERY interesting.... Skyrim.esm's WhiterunPLainsDistrict03 has no LAND record at all. It's created by the CK for a mod though. Instances like this (data missing in Skyrim.esm ITSELF) may be what causes certain issues. I also found that the true-nilla Asbjorn has all 0s for his OBND subrecord (bounding box)... it is also corrected by the CK in any plugin that contains it. (I HAD to randomly choose the ONE actor that is nilla-fubar, didn't I?)

#5 - This may cause issues with navMesh, but apparently these issues are going bye-bye when the new Skyrim v1.6final is released. A mod that contains a WRLD record will not have any RNAMs in it, whereas Vanilla/skyrim.esm's has several dozen. RNAMS in WRLDs contain REFRs and NAVMs. What makes it suspicious is that the WRLD record retains it's orig 00- formID, so does it preclude Vanilla's in-game or is it somehow ADDED to the SAME formID's data (the formID dynamically changed from 00- to 0X-)? I have yet to look further into this, and only tested the WhiterunWorld. [I believe the RNAMs indicate the 'portals' to other WRLD/CELLs via navMesh-doors/etc, but that's a prelim hypothesis]

Moving right along... I see this post is already way larger than I had wanted, and I haven't really begun to list what I've found. So once I get the data and my notes in a presentable form, I'll do up that new thread and we can talk about it there. Also, I downloaded the trial-Gecko and will be checking it out before then. The info/anolysis I have is only with uncompressed records, so I dunno if that will help with the development of Gecko; but It'll be posted in that new thread, so you can decide for yourself. I AM interested in helping, especially since Snip is defunct... even if it's in a beta-testing kinda capacity, we'll see though.

[EDIT: I forgot to mention that I never found a single instance of any size-bytes different from the actual size of that subrecord/GRUP/etc. In other words, if there IS something added/deleted (which I haven't found), the data which stores the 'size' always corraborates.]

[EDIT2: Perhaps Ma Beth 'did something' to the Skyrim.esm before release... something which cannot be replicated with the public version of the CK or any 3rd party app to-date. (presumably regarding compression)]

I am absolutely certain I did not do any such thing: nevertheless my NPC turned into Camilla Valerius. As you've found NPCs get swapped with Snip, that seems all the more reason to suspect that Snip was responsible, but you cannot have found all the circumstances that lead to it happening.

Hehe, yeah we're still working on the terms of service for DC. An abbreviated version is displayed when you register and we have a thread in the forums that state the rules.

Looking forward to the new thread on Beth.

Gecko will recompress all compressed records upon save. We only decompress for display, and won't save them that way. I'm keen to discover more about the record structures because we'll need to have a defined set of rules so that mods can be merged.

I too have never done anything of the sort and both of the NPCs in my alt-start mod were never "quite right" and has issues animating their lip movements and had jerky responses in general which have all since vanished entirely after rebuilding that mod without exposing it to Snip.

I've also never snip copied anything else out of Skyrim.esm into my own stuff. I only ever tried to use Snip to remove unwanted data.

I've also already had a few reports come in that the "blue voids" issue is no longer happening with it loaded. So whatever there is to that is confirmed as far as I'm concerned, which is why I'm now proceeding in earnest with a total rebuild on Open Cities.

There's also the fact that the alt-start mod had data loss confirmed once I was able to build the new one out to the exact same changes as the old one. So whatever that may be worth.

So while you may have findings of your own, they aren't in any way going to invalidate my own findings. They seem to merely add to them.

Not sure how, but my Gildergreen Regrown mod seems to have been spared the damage. Probably just got lucky there and the CK repaired the specific issues that would have cropped up.

As SLuckyD is finding, this is really specific. It will be interesting to see if a certain record always has extra compressed data, or if it only has it under certain circumstances. I'm starting to think that it's only under certain circumstances.

So you may have a record that doesn't have the extra compressed data. Then if you change a parameter in that record, you get the extra compressed data. I'm hoping that SLuckyD's anolysis can uncover those situations. Because I may wind up in the situation that during a mod merge, I have to create this extra data! And this is giving me nightmares.

I think the obvious conclusion is not to use TESVSnip. It just doesn't seem worth it. Let's wait for JTES5Edit instead.

If you are going to publish mods that were edited with Snip, please mention it in the description so I can avoid using it.

Yes, I get that it might well be specific situations. Clearly I have run into them already once, and in all likelihood ran into them again with OCS.

Thingy Person more or less has the right idea. Just avoid using Snip to edit mods. Lesson learned, etc. For those of us not writing apps where the specifics are critical, it's just easier for us to stick with using the CK and nothing else for now. The details screen may svck, but at least it doesn't corrupt mods.

This is a bit offtopic but after reading through this thread I was wondering if other tools that create esp's such as the npc editor is likely to corrupt as well?
I was originally planning to do a npc overhaul with that and then merge it with tesvsnip but since merging _npc forms is sure to corrupt I guess I shouldn't.

speaking of Gecko...are there any alternative apps with mod merging capabilities out yet? tes-snip is really the only one I've ever used

Ingenue: I think you may have taken what I said a bit out of context or added a bit to it (as some are want to do.. ahem). I agree that I don't know all the circumstances that do whatever... I only know that 'whatever' doesn't happen EVERY time, and is therefore based on specific conditions - SOME of which I listed. (please don't get my tone wrong, which is usually mis-conveyed in text, I'm only explaining myself.. not chastising your response)

Thingy Person: For the faint of heart and/or people who don't want to spend the extra play-test time ensuring a flawless mod, I agree that not using ANY 3rd party app is the way to go. I try to adhere to that very rule myself.. unless there's something that I could more-easily do (such as deleting a record, changing a FormID, or correcting a type-o in a edID name). I haven't even looked at the 'details' thing in the CK yet... maybe it's about that time heheh. All in all - I believe the discussion is kinda meant for those who DO use Snip (presumably knowing the potential issues), and ongoing warnings to NOT use it are kinda uncalled for. (again, please don't misconstrue my text's haughty tone... I assure you guys that my real-life voice would convey my empathetic intentions much better)

Renzeekin: From what little I played with the NPC Editor app, it basically does what it states... but it's kind of a pain in the b's to go into the meshes and repath everything to a new plugin's name (otherwise you'd get the 'grey-head' syndrome). Something I MIGHT suggest (but have yet to actually try the technique myself), would be to do up a character in NPCEditor... then 'port' that data into a pureCK actor (via the CK or Snip). I foresaw it being a relatively complex process, so I just did a couple minor esthetic CK changes to my spodum actors' defaults and left it at that for now. I plan to update that mod soon, so I'll probably revisit NPCEditor for its convenience and ease and giving things 'purty moufs'. [and once again, Snip corrupting data is not GIVEN.. but be aware that it MAY happen if you choose to use it]

Arthmoor: The blue-void issue.. I haven't seen it (and do NOT deny its existence) so have no idea how to look for as to what causes it. Do you think it was fixed with the Skyrim v1.6beta? (I believe you participate in that beta.. so that may be a factor in its being corrected) I figured you would've been 'commentarilly' happier to know that I really DID find that Snip affects things.. how disappointing.

But anyways - you'll note that whatever Snip DOES affect, in my LIMITED testing I found that, in-game, the mod will still function properly. It's the CK that DIRECTLY causes a mod to malfunction (in the circumstance that I experienced)... presumably because of what Snip did. I believe the CK sees uncompressed data then, during the interpretation and/or re-compression process, it changes that usable data into something else. A specific example of this is the FULL subrecord of anything Snipped directly from the Skyrim.esm.

Perhaps ESMs in general will have this problem (something I plan to look into at my earliest convenience).... which (if the case) would cause your/others' issues. I proved this to be the case in the NPC swapping (which was the FIRST time I ever seen or heard of it)... though only with skyrim.esm (and I know you have yet to see the hard data supporting this, so wouldn't blame your skepticism, if any).

Specifically, Snipping something out of Skyrim.esm causes the FULL subrecord to delete its LAST byte (of four)... but ONLY after the CK loads and resaves that plugin - beforehand, it still appears normal in-game (because it still has that 4th byte.. CK deletes it for whatever reason). In-game, you'll see something's name as two ascii characters - which is the 'translation' of the data; which links to something inside Skyrim.esm (presumably). This is some kind of compression that the CK does NOT replicate... hence my parting comment on 'something being done to skyrim.esm which was proprietarily not included in the public CK'.


Regarding the new thread I want to create to further discuss all this 'properly': I'm not sure I'll be able to get my notes and data into a presentable format as soon as I had wanted to (tomorrow), but I'll get there eventually (later in the week, hopefully). I just didn't want to data-dump a bunch of gibberish... some of which I actually 'decoded' since that last post.

I downloaded the GeckoPreview, and have made some notes on it (to be posted on the DarkCreations forums, maybe tomorrow since my laptop's battery is dying). That app, combined with my notes and Snip, kinda allow me to expand my notes' details... hence my hesistation. It's got a ways to go yet, but eventually it should prove a formidible alternative to the TESvSnip - and I haven't even tried viewing non-compressed data in it yet!

AndalayBay: I take it that means you would welcome my notes on non-compressed data? I briefly glimpsed the raw data of a compressed plugin, but didn't get far... so I'll be little help if that's what you meant. I DID find a bunch of things that I assume are intended, but simply haven't been gotten around to yet. Again. I'll take that stuff to your DC forum.

Confirmation = good. Confirmation also = bad. So... neutral mood it is

No, I don't think anything the 1.6 beta did would have fixed that. The problem only went away with my mod after I spent the time fully rebuilding it using only the CK.

To say that the CK is the one corrupting the data seems pretty daft to me as it's the official tool and ANYTHING done by a 3rd party is by definition outside the norm, which therefore makes it risky to use. Elminster never once claimed TES4Edit could guarantee you a trouble free life, and I think it would be irresponsible to ever think any existing or future tool could do that for Skyrim either.

As I said before, I've never snipped anything from one mod into another. I've only ever used the program to remove unwanted edits and that action alone was enough to drive me into rebuilding my alt-start mod, and after seeing positive results there, has driven me into a full rebuild of Open Cities now too. God. I hate doing navmeshes SO much, but in the end, it'll be worth it since I've also learned better ways of dealing with stubborn quests that won't release their markers.

My husband and I are developing the new version of Gecko now. It will be several months before it's finished though.

--------------------------------------------------------

SLuckyD - I don't believe the CK corrupts data under normal circumstances. The problem we've got is that TESVSnip decompresses the records and doesn't recompress them upon saving. Now there does appear to be a way to configure it so that it will recompress the data, but that doesn't help for the data that were stripped.

We have confirmed by looking at Snip's source code that it only processes the stated length of the decompressed data. However there are numerous instances where there are more data than that. Snip snips that off. So even if you configure Snip to recompress the data, the extra data are lost.

Secondly, if you don't configure Snip to recompress the data and flag 0x00040000 is still set to 1, which means the data are compressed, the CK will corrupt the data. You've told it the data are compressed, so it tries to decompress them. Obviously that won't work.

We'll welcome any of your findings regarding the record structure. We are processing all the groups, records and fields, but we need to understand the rules on how the records need to be structured in order to merge mods. My concern is that we can't just take the records from one mod and merge them with the records from another. I'm wondering if there are cases when these extra data need to be created. In a single mod on its own, these extra data aren't required and therefore they weren't created. However if you merge those records into another mod, perhaps those extra data now need to be populated. Kind of like having to create the ONAM records when you convert a plugin to a master.

You know one thing I don't envy you on with this is determining when a record has changed. Like when the size of the official kill-cam records was changed in 1.5. There was no corresponding bump in the official file version number, so you'd have no way to know how old the mod was and will simply have to assume it's using the updated size. Which tends to end badly if it's wrong :/

Yeah. For the most part we're immune to changes because we don't rely on a specific structure. For example, when Beth added the new INCC record, we didn't have to change anything. I also think we're ok when it comes to record lengths. We're just looking for the next HEDR, for example, so we assume everything else is the current record. One thing I will have to remember is to have fall-through code on everything though because we will have mapping tables so that records are displayed in a readable format. So if a new record type is introduced, we'll have to display it as is until we add it to the table.

Bit off-topic for this thread, but if the other utilities process the records in this fashion, they will be more robust and less prone to garbling records.

I don't understand how you can make this statement given the enormous numbers of errors and corrupted files coming straight from the CK. I have personally experienced the very errors you have been blaming on Snip in mods that have never been touched by Snip. As a point of fact, before I even discovered Snip, the CK, all by its little lonesome, deleted all the navmesh, all the unique ids, and all links (doors, merchant chests, etc.) and renumbered every item in the cell, including all of the containers and markers, DURING THE UPLOAD TO STEAM. As long as you have been doing this I am sure you can understand what a total unmitigated disaster that was. To say that the CK is not corrupting data is what seems daft to me. In my experience, the CK is a wonderful free tool, but like most free things I have ever received, it has it's flaws and some of them are enormous. I am sorry Arthmoor, I am not trying to get on your bad side, but I am puzzled at your concept of the CK as "normal" and everything else as "outside the norm." I'm not saying the CK isn't a wondrous tool, but in my months of extensive use of Snip, i never once experienced a problem that I could attribute to Snip. Of course, i never used it to copy things from Skyrim.esm either.
No matter how we mod, there will always be bugs and glitches that we need to be diligent in looking for. Turning out a quality product takes much more than avoiding the "risky" tools although that certainly seems like a good plan. However, the CK itself is just as risky as any tool we have or ever will have. Maybe that is simply part of the fun, but it certainly can be frustrating.
Now the part that really got me going. Someone earlier, it doesn't matter who, said that they were now so scared by Snip that they never wanted to use a mod that had been touched by it. If a mod is flawed and causes problems I certainly can understand avoiding it or fixing it. Problems such as those you attribute to the use of Snip should be so apparent on testing that such a flawed mod would never be released. But avoiding the mod just because it has been touched by Snip seems ridiculous if the mod otherwise functions as intended. I agree with the assessment that there is no need to take the risk of using it going forward since the CK is bad enough all by itself. But suggesting that all of the mods out there that have seen snip are broken just because snip has touched them seems like complete overkill. There are plenty of mods out there that really are flawed and that have clearly never seen Snip or they probably wouldn't have the problems they have.

Because aside from Giskard, you're now the second person to make such an outlandish claim. If it were true that the CK was as dastardly as you're saying, then everything you just chastised me over about Snip would be easily detected 100 fold with legitimate CK generated mods. I've seen no evidence to support this wild theory that the CK corrupts mods at random. I say this as someone, as you point out, who has been doing this for some time now. CK-generated stuff = works right 1st time. Snip-touched stuff = twice broken.

Your incident with the upload to Steam svcks balls, but I'm having an awfully hard time seeing how it would even be possible for it to mangle a file that's being read. To me that would indicate a more severe problem with your underlying OS or hardware than it would with the kit as READING a file is not an operation that even Windows can normally foul up to that level.

That is of course assuming you didn't try and push an ESM up to Steam, because that doesn't work and would be the only thing that could explain it.

As far as "normal" vs "not normal". That's easy. Standards. Bethesda's tool is the de facto standard tool and produces the de facto standard format REGARDLESS of what other people later claim is the case. Snip finds an "error"? It's a problem with Snip, not the CK. SkyEdit says a record isn't the right size? Time to update SkyEdit's record definitions. The bottom line here being that the CK produces the only file format that can be considered official.

It's more or less like trying to claim that creatures added to Oblivion by MMM are official lore and all others are imposters. That's not how this stuff works.

So I guess if you really want to light the fire and play around next to it, go right ahead. I'm not going to stop you. Just don't expect me to consciously choose to use a mod that's been put through Snip, and don't expect me to stop warning people about it either. It would be doing a massive disservice to everyone to try and bury this one in the sand.

Yes, it did happen. No I don't have hardware or OS issues. No, I don't have network issues either. No, I did not try to push an ESM up to Steam. The Kit won't do that anyway as it won't set an ESM as active. No, I don't play around with flipping tags to fool the CK as I know you and I both agree that is foolish. Yes, it did mangle the file that badly, believe it or not. I was up all night fixing it. And no, I have no idea how the CK could so completely mangle an otherwise clean file as it is uploading it to Steam and also mangle the copy on my computer. If it hadn't trashed the file on my computer, I would have simply uploaded it again. My memory may be flawed but I don't think it even asked me if I wanted to save the changes.

It has only happened to me once, but it happened nonetheless. And that is not the only example of the CK mangling files, just the most outrageous example I have yet run across. I have had plenty of other crashes that trashed a file and I have had many conversations with other modders who have had the same experience. The damn thing just crashes all by itself if you leave it open long enough. In the early days, it deleted pieces of files constanty but that seems to be a rare occurence now since the 1.5 update. It still shoots room bounds and portals all over the cells without rhyme or reason and I have had a constant problem with it disconecting merchants from their chests although some of those I am remembering were back in the Snip days of a few long months ago. It still creates crazy errors that no one seems to understand or be able to fully identify. I could go on and on and I don't even script. I understand from those that do that the scripting side is infintely more frustrating. You and I actually agree on so much but we clearly do not agree on this one point. The game and the CK are clearly amazing works of art but they are also both very much still flawed. They might be the best ever. The best there is. The best we can ever hope for. I recognize the astounding complexity of the game and the CK. But they have bugs. Those bugs are part of the standard. When you run across them, it is all too easy to blame them on some other tool you might have used, but that does not change the fact that those same bugs have been observed and reported on numerous occaisons in the forums and elsewhere by modders who aren't even aware that Snip exists.

Trying to pretend that the CK is perfect as if it was gifted to us by God is just nuts. You have spent so much time crying over the Navmesh bug, I don't see how you can feel that way. Would the bug have even existed if the CK were incapable of doing wrong? It only happed with modded files. If it were perfect, the CK would have corrected the mesh all by its perfect little self. I know you are going to make up reasons why that is not a flaw in the CK, but it doesn't really mattter. I know it is a problem in the game but the CK and the game are intricately related and intertwined. Geez, I know you have read the amazingly long CK bug thread. I guess you will no longer be complaing about bugs in the CK as it is now the flawless standard.

I am not pushing Snip. I will also no longer use Snip. I agree with you 100 percent on that. All I am saying is that the CK can create all the same errors and corruptions as you are blamimg on Snip. If I had a better alternative, I would never use the CK again either. Why do I make such sacreligous claims you cry in horror? Simply because it is true. If the CK were perfect and modders could do no wrong as long as they use only the blessed tool, then there would not be nearly so many flawed mods out there causing CTDs and confiicts with each other. Keep crying about Snip. I am sure glad that AndalayBay and her partner are working on an alternative even though you can never use it as it it will clearly violate the Bethesda Standard. Enough said. I knew it was going to be a complete waste of time when I started it.

Starting to get somewhere here. There's a pretty significant difference between normal operation and what it does when it crashes. All bets are off once any program is in a crash state. I've seen numerous reports of the mangle jobs it does when it crashes as it's saving. Which is the ONLY way it's going to destroy your file utterly. There's simply no possible way it can just randomly say "hey Windows, trash this" without you knowing about it.

This is precisely why it's so insideous that it's only been outed this long after the game's release. Remember, people have been snipping mods together since November. Had the CK somehow never shipped, I'd simply never have used the mods because while I would have (at the time) trusted Snip to edit something CK-generated, I rarely put that kind of faith behind something that's been entirely generated using it or any other tool.

Oh, you have no idea then

Papyrus is a double-edged sword. Very powerful. Can do plenty that previous games only dreamed of. But maddening in its quirks and flat out rage-worthy in the way that it sticks to your save game and there's nothing you can do about that.

They're an entirely different class of recognizable bugs though, and as such, they can be avoided. That's not true when you're off the reservation with a program that's essentially a black box to everyone except its author - who has vanished long ago.

I never once claimed it was perfect. Let's not start doing this, ok? You're trying to stuff an awful lot of things down my throat that were never uttered by me and have never been my position on these matters.

I've spent a god awful amount of time hammering on the navmesh but because it was real, concretely provable, and only fixable by Bethesda. It was also *NOT* exclusively a CK bug. They fixed whatever the CK was doing wrong with them back in 1.5.24. In March. We've been waiting for the other shoe to drop ever since. So yes, the bug existed despite the CK doing everything right with 1.5.24 onward. It required a game side fix to fully repair it. Despite that, the records actually saved by the CK were not at issue. They were properly formatted. The actual bugs in the system lay elsewhere in the code, beyond our reach.

Sorry, but on this point we can never agree, because were this true, then my rebuild of my alt-start mod would have blown up in my face in spectacular fashion. It did not, and was the vehicle by which I proved that Snip induces data loss. Plus every other mod I've made would be equally screwed up, and the Skyrim community in general would be a sea of thousands of corrupt files. Since this is demonstrably not the case, I can't see how you arrive at this conclusion.

This is entirely a scenario of your own making, don't try and make it out like I called the CK the everglowing infallible all seeing all powerful god tool. You know damn well I never once claimed this. I simply stated the fact: It's the de facto standard tool for the job and as such all records produced by it are canon. Even if people have somehow got it in their heads that 3rd party tools are producing the proper results.

Besides, if you had been paying the slightest bit of attention, I've been groaning about not having a TES5Edit for quite some time now. Why? Because what it does is irreplaceable in this community. While the CK may be producing properly formatted records, it also loves to casually mark things as edited that really aren't. Which isn't corruption btw, that's just par for the course dirty edit crud. I'm sure AndalayBay can attest to just how much groaning I've done and how much prodding she's getting on moving Gecko along. I'm also rooting for JTES5Edit for similar reasons.

Each tool will have its purpose and will be capable of something the CK isn't. At the same time, with active developers on hand, and the awareness of the issue raised by Snip's shortcomings, repeating those mistakes doesn't seem very likely.

Snip itself could eventually be redeemed should the author resurface and fix the bugs. Until that happens though, I will avoid it.

i dont know how much this is worth, but i think its worth at least noting here:

having used the CK exclusively to rebuild my corrupted amethyst hollows mod for the past month or so, i have had the thing running on fumes with sessions that last up to 30+ hours at a time. this new version of my mod has never seen or touched Snip in any way shape or form, and the CK has never crashed or bugged on me or caused further data corruption. wheras when i was working on it the first time around (with plenty of Snippage) and also my other mod which was also very long hours of work, was also experiencing quite a large number of crashes unexpectedly. granted, the difference betweent hen and now are several iterations of updates, but even still, i find it hard to believe it's just a coincidence.

i am convinced at this point that Snip is ultimately deadly to your mod from inception, even if it seems like nothing is going wrong. I think the random crashes in the CK are attributed to Snippage, as i have had no incident for the past several weeks using a strictly sterile CK only environment.

of course there are still a multitude of bugs in the CK but most of those are documented with known causes, and thus can be avoided

My experience largely mirrors yours, AD. Outside of the XAudio2 thing I've posted about, I see maybe one crash in the CK for every 100 hours or so I've put in to it. Always inconvenient of course but even then it's never once corrupted a file that was actively open in it. Even when Steam choked itself to death when I was releasing Ars Metallica.

It has yet to crash since I stopped loading the snipped versions of LAL and OCS.

You keep talking about corrupted files, how does one know that has corrupted files? Thanks.

Anyway, CK is bugged enough, no working LOD, Heightmap, duplicating worldspaces doesnt work, etc.

It's pretty easy to tell when your mod causes voids to appear in dungeons, then mysteriously doesn't do that anymore once you've rebuilt it

Since there are no reliable data anolysis tools out yet, it isn't really possible to produce hard evidence one can turn into screenshots to satisfy the silly rule on the internet of "pics or it didn't happen". Using Snip to confirm Snip broke something isn't really going to be all that useful. You have to observe how your mod behaves and pay attention to user reports of issues. After awhile, it becomes too large a problem to ignore.

Or, you could bust out a hex editing program and manually check the data, but gah, that's just asking for torture.

That the CK has bugs isn't really at issue here. NONE of what you just listed results in corrupted files, so I don't know why people keep raising the strawman of "it has bugs" as meaning anything in this.