The game is most fun during levels 10-30/40 (depending on build). After that I tend to restart.
Edit: The big problem here is that I have typically done nothing in the game by the time I reach level 30/40.
Says who? The overwhelming majority of RPGs keep the difficulty pretty constant throughout the game. You become more powerful, but face more dangerous opponents as well. That's how it should be. No one wants to play a game where you switch your brain off and steamroll a bunch of gnats.
Not entirely true. When high in level I expect the early enemies I faced to become easy. I don't want endless variants in power of the same creatures (brown bear, cave bear, snow bear etc). And I also don't want Oblivions constant supermonsters (Ogres, Minotaur Lord) in places I certainly didn't expect to find them. And for hardware consistency, neither do I want hordes of low level enemies so that low level systems can't enjoy the game at high level characters.
For the characters I tend to make (sneak hunter/archer), combined with my rather careful playstyle (I run away from nearly anything in the beginning

), I've found that with a little bit of focus and not a whole lot of "take advantage of everything you know" gaming, master difficulty should actually be the normal difficulty. In old dice games, no matter your level, any players who didn't try hard to roleplay to the best of the characters abilities, tended to be punished by the GM by getting killed or at least some near death thingie going on. The GM knew your abilities and based the game around that.
Could such an anolytical approach be made algorithmically? Like, monitoring your successrate and adjusting accordingly? I don't know, sounds like a difficult task, and well known exploits (like "faking the difficulty") might be hard to avoid.
Btw, I try to limit my own crafting such that I only pick perks in one (typically only strength perks and arcane perks in the smithing tree), while leveling up the others (no perks) solely to play the money game, since some skills doesn't level naturally for me. I don't feel like I'm exploiting anything playing this way, only playing what I expect you are supposed to play.
Frustration on master difficulty, is only obtained in the beginning of the game and only if you expect to win everything you face and/or don't ever run away. I don't want the end game to be constantly frustrating of course, but SOME challenges every now and then would enhance the game infinitely. Unreasonable self gimping or "pure destro builds" (which doesn't make sense to me in the first place, from a survivability standpoint), is not the way to go. We need some added frustrations also for other classes, especially in the end game.