I see posts where everyone is jacking up the difficulty to the highest level because combat is toooo easy. But whats more fun. Making the combat harder and thus having to take a lot of combat perks or leaving it at normal and being able to afford perks in non combat skills like lockpick or alchemy

whats fun for you isnt fun for others...

I've not played Skyrim yet, but with other TES games I leave it at normal.
The challenge/entertainment of the game isn't in the combat difficulty.

def higher difficulty.better Al and stronger enemies is the way to go

Unless things have changed this go around, all the difficulty does is make enemies do more damage while you do less. That's it.

I just leave it whatever the default is generally, though in Oblivion (etc.) I play with a lot of mods that increase the difficulty in different ways (new enemies, more spawns, etc.)

For me, Elder Scrolls was never about boss battles and getting your hide handed to you. I mean, yes, I was frequently trounced in Oblivion, but adventuring rather than action was what always attracted me to this series.

After Dark Souls, I need to crank up the difficulty. Otherwise I feel like it's a walk in the park. DS has kind of conditioned me to get my ass kicked, so it feels kind of weird not to have that tension. But Skyrim is utterly amazing so far. I'm at the wizard college, feeling like I'm in Hogwarts.