Except I'm not talking about "false-flag esmifying". Rather, I mean why don't they change Steam so it (Workshop) accepts "real" esm files associated with corresponding esp - just like we can do with Nexus?

You'd have to ask them about that, but I suspect it's because it isn't intended to carry master files like that. And again, being forced to convert things to .esm format is ALSO a hack. That's not how the CK is built for us.

...Unless all plugins had their internal ESM flags ticked, load order was dependent upon file extension rather than ESM flag's state, and the CK were to be able to have ESM flagged plugins as "Active" and automatically created the ONAM lists upon saving when applicable. The community didn't introduce ONAM lists, *Beth did (although they've, to date, never given us the means to utilize 'em =/). Yeah, plain 'ol ESPs, hopefully, can be made to cut it or if the ONAM lists are so instrumental, perhaps ESPs will also have them...?

*The Pitt for Fallout 3

Yes, but the fact that we can't generate them outside of 3rd party utilities means the CK as provided to the end user is not intended to be used that way. Which they are still trying to fix.

It probably isn't going to be easy though since this navmesh thing has been around since Fallout 3 so it's pretty deeply embedded into the system.

I'm still curious about how http://i.imgur.com/VaIrx.png. Perhaps the CK can do it with one of its parameter or something. That NavMeshes and their ins/outs are so embedded might mean they stick with what they know works, ONAM'ification. Might be a CK update comes out with a new patch and all plugins get ONAM lists... Can't wait to see how this plays out...

It's probably one of the minor differences between the in-house version of the CK and the public version. Maybe even part of the version control we don't have available.

I know people asked way back when why Bethesda didn't simply switch on the ability to save a .esm file with GECK for FO3. If these ONAM records are necessary to prevent certain crashes from occurring, then they ought to just take the easy road out and give us the same ability now and let things settle out how they will with load order.

There will initially be a lot of angry modders because of it, but waiting to decide this a year from now will only anger even more and quite possibly turn people off of the idea of modding for this game entirely. The userbase will be outraged and blame us anyway.

If it is part of their VCS, that would explain how they got an ONAM list into their Update.ESM (probably the same setup they use for localization?). Would be great if the CK had those abilities built in if that's the case.

'Us' as in modders or 'us' as in beta testers?

Us as modders, though perhaps both

You may very well be right. However, I find that problems are more difficult to solve when people who are working on them fail to recognize connections between seemingly unrelated circumstances. Sometimes it is very difficult to recognize all of the connections when, in fact, in any closed system, whether mechanical, electrical or software, it is not unusual that seemingly unrelated problems are, in fact, related. I have no proof these two are connected, only suspicions. Just as you have no proof they are not. The only way this becomes a problem is when people who are working on the solutions either assume they are connected or assume they are not, instead of keeping an open mind so they can better recognize possible solutions.
The two problems seem to occur only together. Both issues are the result of the exact same chain of events. I have never had one without also having the other. Although, once you have the crash, its hard to go back and verify the navmesh bug. However, the door bug has a prior indicator that tells me it is going to happen. When I see that, I check for the navmesh bug and verify that it has also been triggered. The recent problems caused by the attempt to fix the navmesh bug are so similar they started me wondering about a possible connection. As a result, it would seem to be a plausible theory that the teleport failure is a result of the broken navmesh. I also realized that while I was using 1.5.24, I did not notice an occurrence of the same door bug and I had hoped it was fixed, although, to be fair, there were so many door crashes it would have been hard to tell sometimes. It came back with 1.5.26, of course. The possibility seems even more plausible now that I know for a fact how closely interconnected the navmesh and teleport systems are. But is still just a possibility, until somebody figures it out and fixes it one way or the other.

.
Maybe this might sound bananas coming from me, but I'm not monkeying around when I say I totally agree with you on this point.
.
.
Something I've noticed is when the mesh used to disappear in Whiterun, it did so indiscriminately. So, what if that means the markers or the door at the other end are occasionally disappearing on loading? In that case it could be creating a contingency the game is unprepared for, and whoa, what are all these desktop items doing in my atrium!
.
But this brings us back to the core of the problem. Why fail to load a random number of meshes in the first place? I'll wager that the meshes load in a given order - and that if only one or two don't load it will just be that one or two. There is one other question. When moving to an interior cell caused a CTD, did you recall any load screen coming up (even the mist appearing at the bottom of the black screen)? The weird thing about this problem, when it manifested itself in the last NavMesh bug, is that the load screen would not load - not even the mist effect at the bottom; just black screen and thence to the desktop. Have you seen this behavior in your cell-load CTDs?
.
Another similar problem, which I've seen twice in southeast Blackreach, is that textures simply won't load sometimes - but the mesh is still there as an invisible obstacle. Reload the game et voila! There it all is. As it turns out, even some of the lights were missing and these are, essentially, markers - like door markers but with a different function. Admittedly I haven't seen this in-game since I last brutalized my operating system. However, the Creation Kit still does the same thing from time to time, and then announces that it cannot load certain textures. But in this case, we can avoid the whole rigeur of restarting the Creation Kit and hit F5 - and, to my eternal surprise, this does the trick every time. If only there was a magic reload textures key in the game - like the one in the Creation Kit, and if only the game could recognize when textures were not loading (like the Creation Kit) a solution to this little problem might be very simple. Who knows, maybe a little conditional statement filled in by a little cut and paste from the Creation Kit just might do the trick. But it may not necessarily solve the problem I once saw in Whiterun - or that you are experiencing in the game at the moment if this happens to be a separate issue.
.
That is not to say that it is not worth trying, but that troubleshooting complex systems tends to be a very messy business and it is probably better not to get one's hopes up too early in the process.
.
That's why I think that having threads running like this one is a very good idea. We can work away at our plug-ins, keep an eye out for the bug and, when it shows itself - take some notes and post them here. That way, Bethesda get a little more detail on an unintended behavior in this complex system and, from what I've seen so far, they will make a serious attempt to nail the bug once it has adequate definition - and that's half the problem of determining the little beastie's origin. Who knows, maybe one of us will stumble on a solution in the process.
.
[EDIT]I neglected to mention a http://www.gamesas.com/topic/1363266-mod-crashing-when-entering-modified-or-new-cells/ and might be useful for some of us[/EDIT]

RealmEleven: In my testing, whenever it CTD without a load-screen it was ALWAYS because of a corrupt saveGame. You tried console-coc'ing there straight from the Title Screen (without loading ANY saveGame)? To fix it, all I had to do was load my saveGame (with the appropriate ESP enabled, eg: the old version when the new one causes CTD), then save in a different location... re-enabled my new version, disable the old, loads right up - no more CTD/NLS. Other things cause this as well, but are non-related to navMeshes (like having certain collision types in custom NIF files, or certain resolutions in the DDSs).

Everyone: I just found another manifestation of this bug... long-lost cousin, Jethro Navmesh. As I described in earlier posts, my mod (which was afflicted by 1.524 CTD) has an interior cell which links several doors to Tamriel and exteriors all around the world. For 1.524 I deleted the Vanilla navMesh data, refinalized the interior those doors linked to, and my mod worked fine (except those Vanilla areas).

NOW, since 1.526 was released, I replaced my Vanilla navMeshes and reuploaded my mod. But what I found while doing this, was that the old version (the one for 1.524) stopped allowing followers to go TO my interior to the world. They could go FROM my Home, but not back in again; the follower meeting up with Player several fast-travels later (implying the doors' link was severed, and they tried to find an alternate route.. ie: walking across half the world). I tried refinalizing the Vanilla navMeshes, I tried refinalzing my interior's; but the only thing that worked was deleting the interior navMesh and remaking it.

I did a recast-autogen of the entire cell, hand tweaked it, finalized... works fine again. The mystery is that the CK didn't change from ..24 to ..26; ALTHOUGH, that cell MAY have been navMeshed with an old version of the CK (trusty v1.472). Also, other cells which link to Vanilla space worked... it was only that one interior cell which needed to be remade completely. I didn't try re-linking the doors (then re-finalizing); it DID seem the doors were partly at fault, as they worked in one direction. How and why this happened... I dunno - maybe some kind of data was stored in the saveGame which wasn't recognized by the new ESP. There's precious little else which could cause the difference, the CK itself didn't change (from 24 to 26).

If you are going to try this, I suggest (backing up your ESP first of all) loading your mod in the CK. Then select your problematic cell in the cellViewWindow; select and delete any Navmesh data you see at the top of the list there. Save your ESP, select another cell, then re-select your cell... it should now be free of navMesh. Make a little 3 or 4 tri navMesh on top of a door (which links to Vanilla).. finalize and test it. If it doesn't work, something else is causing your problem... and you'd revert to your backup ESP discarding this failed attempt.

My ultimate point is that strange things happen for whatever different reasons... the speculation on how and why may go on for another several years (this 'bug' has been around for that long already). All I know is I've found 3 different navMesh-related bugs, all of which I found cures for... what helps you may be something completely different.

Barring all other solutions, my opinion would be to either lobby Steam heavily to allow ESMs (or whatever is disallowed for whatever reason) or to not use Steam to distribute your mods until a solution or compromise if reached. Either way I believe it would be prudent to go with 'Ye Olde Solution for navMesh Bugge' of splitting your mod into ESM/ESP. I know how repugnant that seems to many of you - but if there's been three or four game-titles released with the same bug/limitation (since FO3)... how can we expect them to fix it just for THIS game? ... and given the last attempt at doing so, it probably won't be any time soon, if/when they ever do at all. I know it's a grim prognosis... and I try to be optimistic, but hey - I'm a realist when it comes to spending valuable time on seemingly lost-causes.

Lobbying Valve to do this isn't enough. Bethesda would need to update the CK to be allowed to save and edit ACTIVE .esm files, which I'm sure you can quickly guess would lead to people hacking up Skyrim.esm. Something I'm sure Bethesda doesn't want people doing because it will generate too much noise in tech support.

Not using SW would be pretty sad at this point. It may not be the greatest platform for usability to modders, but it has orders of magnitude more exposure to the users than anywhere else. I rather like the fact that it's drawing in more potential talent and encouraging Bethesda to provide more official support. Abandoning that sends the wrong message.

For me, I'm not wasting my time on hackish solutions like splitting things into a .esm/.esp pair. Nor am I going to deal with having to put up with MasterUpdate. If Skyrtm modding seriously comes to that level, then I won't waste the effort in producing anything that would require a navmesh edit beyond really small simple locations. Just like you, spending time on a lost cause is out of the question. If this sort of thing results in lots of people deciding the same way and Skyrim is reduced to a whole lot of smaller mods, some of us will just go back to modding Oblivion where these headaches don't exist.

Should this happen, I'd predict the quick death of the Skyrim modding forum. The only reason it's still alive and well right now is because Bethesda is still actively trying to fix things. People have lots of patience in that scenario, but if it's announced that this bug won't be fixed, count on an exodus the likes of which you've never seen.

Sad, but true. *sigh*

I'm a little puzzled by the attitude toward using an ESM/ESP combination. Just because they did not give us all of the tools to easily do that in the CK doesn't necessarily make it a "hack". In a sense, everything we do in a mod is a "hack" of Skyrim. I know I am far from an expert at this, so maybe someone can give me some direction, but it seems that the mods I want to build will not work any other way, regardless of the navmesh/transport bugs. I want to build separate rooms/buildings that connect to each other and release them as separate mods so users can take what they want and leave what they don't. I want to have some outdoor areas and some indoor areas combined. Since I cannot make one ESP reference objects in another ESP, I cannot connect them directly together. I must either connect them through an exterior cell or have an esm to connect them to. Esms won't do landscape, so I have to leave some parts in ESP format. The only other solution I see is to have the entire complex in one esp and, maybe I'm wrong as I haven't tried anything bigger yet, but there appears to be a 1meg limit on the size of the file that the SW will allow you to upload. I'm already using two thirds of that apparent limit with my first building. My initial guess would be that limit, combined with the ESP restriction, means they only want small mods to begin with. Until someone can point me to something better, my solution is to post a few pieces on the Workshop and post the full mods on other sites that allow esm files.

And I suspect that Bethesda could probably find another way to restrict editing Skyrim.esm if they really wanted full size mods that would potentially compete with their own dlc. I don't mean that in a bad way. It's their property and they have every right to restrict what we can do with it. And I want them incentivized to give us some rich dlc. I've already spent 1000+ hours actually playing Skyrim and modding is just my way of biding time until new content comes out. Which I also suspect might be the whole point to giving us the CK in the first place.

The most obvious reason for it is because the CK is not built to support actively editing them. You can't work on one as a legitimate official mod. You can't produce the necessary ONAM lists to allow an ESM to edit landscape, or include new landscape. Among other things. Splitting things up into an ESM/ESP pair kills off any possibility of using the Workshop as it exists now. Shezrie probably won't care, but I will. Lack of official support from the CK means that's a no go.

In order to make up for this, one is currently forced into an unreliable solution of using tools created for Fallout to add ONAM lists to Skyrim files. I don't care what side of the camp you come down on, this is objectively a bad idea. You have no idea what that tool may be doing to the rest of your file it doesn't understand. If you think having certain unnamed individuals screaming from the sidelines about corrupt files is bad, just wait until you accept doing this in the mainstream and mainstream modders openly oppose it.

We don't yet have proper tools for doing this kind of work in Skyrim, and even if we did, as I said, I'm done. Not going there. It did nothing but create a big mess for Fallout 3, and it won't be any different this time around either.

I define as a "hack" anything you can't do with the CK under normal use. Which basically means generating .esm files is a hack no matter how you slice it. Yes, there are legit uses for those, but until Bethesda gives us official means of generating them, they're all hacks.

There can't be a 1MB limit on files btw. Open Cities Skyrim has an ESP that's already at 1.97MB with just 3 cities out of 5 done and it's up on the Workshop just fine.

every single armor/weapons mod that introduces new meshes is also a hack (which accounts for a significant percentage of released mods)

i have serious doubts we will ever be able to make "legitimate" mods outside of very simple vanilla edits, regardless if they fix the navmesh issue or not.

True, but I'm also pretty sure you knew what I meant too

yeah i know. i do want bethesda to fix the esp issues, but i think i'm just too impatient hahaha

Thanks Arthmoor. I appreciate your opinion. I certainly agree about the ONAM lists. That seems unnecessarily risky to me. However, I still don't see an alternative to using a combination of masters and plugins so maybe what I want to do is just a bad idea. I'm glad to hear that the Workshop limit is not real. As I upload the mod though, it says I've used xxx of my 1024mb available. At least, that's what I remember it saying. I haven't hit it so far, but I am getting concerned. In the mean time, I have been going to a great deal of trouble trying to be sure my two ESP mods at going to work together in adjacent exterior cells.

the limit is 1GB (1024mb)

Is that a 1GB total allotment or 1GB limit on each mod being uploaded?

each mod from my understanding (but i could be wrong, maybe it s a total allotment? i dunno)

Thanks guys. It wasn't clear to me and I have been worried I was reaching the limit. I wasn't going to panic over it until I had to though. I still don't want to combine the mods if I can help it.

Arthmoor: Some may scream from the sidelines - but occasionally it's with relevant solutions to problems related to this thread. It seems you have a much more purist attitude toward modding than I do, as I'm sure others will agree. You also have way more confidence than I do - in Ma Beth's favorable resolution of this navMesh limitation before TESVI is released.

I do think you're right that we shouldn't be using old tools for previous games... and some mods could benefit from being remade completely in the CK (which I am an advocate of). But I don't use FO3Edit, and I don't use ONAM lists either. I use TESvSnip to change ONE byte flag, and then a few more bytes to add the master dependency to the ESP. This is easily undone when I want to edit the ESM in the CK (which is rare anyway); and I've been doing this kind of thing with pure-CK mods since it's been released... all uneventfully and without the slightest bit of drama you predict.

Now if minimal changes like those get my mod (and others) working just like mods did for Oblivion.. than only the most staunch of purists would deny such a solution and urge others to avoidance as well. I also think you have surprisingly little faith in the Skyrim fan-base, pertaining to your prophecy of some kinda 'mass-departure' of people - because they can't deal with this, right? ... and to go back to a game released like five or six years ago? Wow..

One more thing I don't agree with... people could have been editing Skyrim.esm all along - we certainly have the means to; albeit it's with what you consider an impure hack, but the more 'unscrupulous' individuals among us (like me) care not about getting our hands (and maybe our mods) a little dirty. We could alter Fallout, Obliv too; but have you ever once heard of a single problem with a mod uploading or somehow changing one of these files.. like ever-ever? (It may actually violate the EULA to upload such a thing.. I'd have to re-read it)

Everyone: All in all, there has been a reasonably safe workaround/solution to this navMesh bug for years, which more people modding Skyrim should be made aware of, rather than nay-saying or waxing prophetic. By navMesh bug, I mean the one which has apparently gone back to breaking AI in v1.526 (or disappearing actors/critters altogether?). By solution, I mean placing all the custom cells (and their contents, except Vanilla-linked doors) in the ESM, all navMesh data (for everything) in the ESP, all changes or additions to anything Vanilla in the ESP (this means interior doors linked out to Vanilla... they go in the ESP too). This does NOT require ONAM lists or using old software; just 30 seconds with TESvSnip (literally).

... and since some have been waiting for me to say it.. again (though this obviously isn't directed toward them); if this method doesn't work, then you likely have corrupt saveGame files you need not be testing with, corrupt data from a previous version of the CK (or preCK software), or shoddy mods/tweaks/etc you need not to have enabled when testing or saving a test game. So there you have it, my unsolicited opinion... now back to modding...

[EDIT: oh yea... about the Workshop... it's a great idea and all, but until modders get the knowledge, experience, and proper tools to make all mods 100% trouble-free (which is unrealistic in itself), it just won't work to its full potential. Too many people don't adhere to 'proper' ways, and too many things can go wrong (not disabling old mod's ESPs, etc).

My main stance on the 'lobby Steam' theory was that if they want people to release meaningful content, they have to let us upload that content in the form we currently need to make it in (ESMs, and certain loose files from what I understand.. or have they changed that yet?). To actively prevent us from doing so is contrary to the purpose of the Workshop; though I can appreciate the possibility that legalities or other unknowns may be preventing these changes. In the meantime, have they (whoever) released an official statement on the subject (ESMs on Steam)?]

Actions speak volumes, and Bethesda has demonstrated to me that they mean business this time. 5 months after Oblivion hit the scene, there was no mention of ongoing support. No finger lifted to fix stuff, and the only substantive patch to come along eventually didn't fix the one remaining game critical bug, not even on the 360.

There has been a sizeable effort this time around. So much so you've probably noticed they're handing out free content that goes above and beyond what the game originally called for. They ran a closed beta, and continue to take reports from those of us in that group to this day. They even respond to them and act on them, which is why the navmesh bug is still getting a ton of attention. All signs point to it getting fixed long before TES VI. Heck, they point to it quite possibly being solved before the first DLC. What other game company do you know of in the last 5 years that was willing to delay their DLC in favor of nothing but bug patching?

And this could be done just as easily with Wrye Bash. The point is, why should we have to? Why contort ourselves doing things the system isn't designed for when the end result is ALWAYS a mass of people [censored]ing about how broken things are as a result of using non-standard methods? This isn't a bashed patch we're talking about here. This is fundamentally altering how the game handles data files.

You have your opinion, I have mine. I feel strongly enough about being forced down a dead end road that I'm willing to simply reverse course and go back to something that works and doesn't carry a single one of these navigation problems. I think you're being far too dismissive of the severity of the situation since you seem capable of magic and claim it doesn't affect you.

The Workshop, like it or not, is the only reason Bethesda is still paying attention. You are essentially advocating that modders abandon it. Doing so sends the message to the company that we don't want their involvement, that we don't want things fixed, and that we're willing to accept substandard quality of work in products we've paid good money for.

That's not something I will ever go along with. Abandon it at your own peril.