Agreed; I think that is exactly it.

In my case I do not at all prefer 'stream of consciousness' play, and I expect a well structured
* main campaign [gameplay aside] for any RPG to be interesting.
Fact is, I do not need a game for... what amounts to a digital daydream; I can do better without its framework and limitations.
What I can't do (and do seek out in games), is tell myself an illustrated tale that comes as a complete surprise.
*(And in this... [for me] personally crafting a PC is unimportant so long as the PC I get seems well done; but it's always a welcome bonus if the story of the game gets tailored to a specific PC that I create.)
*(Here, "well structured" does not necessarily mean linear; and preferably not so.)
Here's what I would love: a grand narrative that is compelling, branching, and filled with consequence. Now: if you let me opt out of it, I'm with you 100%. I'm not opposed to good story-telling. The problem is, being a writer myself, I often get frustrated with poorly implemented narratives. Sometimes, no story at all is better than a shoddy one.

What I want is a game that supports 'stream of consciousness' in a very robust way. (@Gizmo specifically: re my comments re classes and factions) I want realistic rules and restrictions, but I never want to be told how to play. The perfect game (imo) would be a game like Skyrim that has incredibly well-executed, long, complex, compelling narratives filled with interesting characters, choices that have a profound impact on your character, etc.;
lot's of those narratives (one for every guild, plus several non-guild-related quests); and tons and tons of interesting little side-quests around interesting secondary characters.
Then I want to be able to opt out of every single one of them so that every single time I play the game I can construct a story around my character based on a different selection of quests. Being forced to use one character and follow one narrative (even with lot's of branching paths) is not my idea of a good RPG, no matter how great the story is. That's just my opinion, of course, and no one has to share it. That's just what
I want. BGS games come closest to that formula, which is why I play them more than other games.
Yess.... and no.
It depends on what you mean by "tell your own story". Like any medium, there are limits on how we can tell the story. The first mission of the thieves guild, for example, seems better suited to being a dark brotherhood first quest (sneak in, eliminate the mercenaries that get in your way, murder the owner of Goldenglow, escape through the sewer) than the thieves guild. It's remarkable what a difference stabbing all the mercenaries makes in that quest. .
Meanwhile the first mission of the Mages Guild (Under Saarthal, I mean) could be completed really easily by any kind of character, there is nothing to differentiate it for a mage at all. Considering the theoretical lesson in the college and the overall deadliness of magic, it would have been quite possible for Bethesda to have engineered that quest so that you were forced into a magical duel with a lich, as an example. (Or rather, the son of Gaulder bombards Tolfdir with magic while he brings down the orb-shield, and it is left to you to shield Tolfdir with some kind of ward before you both finish Gaulderson off in whatever way you deem appropriate).
The thing is, if you left me to tell my own story, I could do so in a much more fluid way than I am able to do through a game, which puts limits on what can happen, puts limits on what it delivers, and all in all ends up with a "Jack of All Trades, Master of None, which isn't at all better than a Master of One" game..
It's interesting that you brought up the first Thieves' guild mission: I played that without attacking a single person. There was one guard that I had to use an invisibility potion to sneak past. I never even saw the owner of estate. I didn't start the guild until my character was level 20's and I play on master with 100 Health. There's no way I could have killed even a single guard. It actually felt exactly like a good thief quest to me; but I played it under the assumption that I was supposed to avoid detection.
The problem with a lot of these 'the game doesn't force you to do it' arguments is that they seem, to me, to be based more on bad role-playing than anything else. (I'm not directing that at you, I just mean arguments of this nature in general.) I haven't joined the Mages' guild yet because I haven't made a mage for Skyrim (highly uncharacteristic for me, since they're my favorite class, usually). When I do, I'll join the guild and I'll be using magic to do pretty much anything. The fact that the game doesn't
force you to use magic doesn't mean that the quests are broken. Don't get me wrong: it would be incredibly cool if there were puzzles/objectives in those quests that could
only be completed by using a spell, but the fact that there isn't (I'm guessing) is a missed opportunity, not a broken mechanic. You can't complain that BGS didn't force you to RP. You can say that the RP would have been better if there were more occassions in which you are forced to use your class skills. I'd agree with that.
That goes for the JOAT argument as well. I don't typically play characters like that (though I have). A lot of my characters are variations on pretty basic archetypes. I just play them in character. I don't feel compelled to play JOAT at all. If I make a mage or a warrior and it doesn't fit my RP, I don't pick locks. If I make a thief or a warrior, I don't cast spells. I don't find my enjoyment of the game suffering because of it. Yeah, my mage can't open chests and get extra loot. So what? He can brew potions and enchant items if he needs money. Money has never been an issue in their games. Same for my warrior: I can just loot corpses and sell their gear or smith stuff and sell it or hunt and sell hides, etc. In none of these cases do I feel compelled by the game to act out of character or any inducement to play a JOAT.
Certainly, the game needs to give you something to work with: you can't RP in the void. But I think a lot of the complaints about Skyrim's lack of structure are based more on different priorities, preferences, and play-styles than they are about actual absences in the game's mechanics. But maybe that's just me. I have no problem RP'ing in this game, and most of the RP complaints I see are about people complaining that the game didn't force them to act in character, which just seems silly to me.
My conclusion: different people have different capabilities when it comes to RP.