Many of the above suggestions are great (No fast travel, turning off music, HUD, etc.).
In addition, I have found great enjoyment in my second run through by NOT helping everyone. Choosing the "too bad for you" conversation options and thereby not getting a hundred miscellaneous quests frees you up to choose what
you want to do.
Don't go into caves or ruins just for the sake of loot- only if your character has a reason to explore or find something specific.
My current character is seeking out Daedric artifacts. As a result, I am only doing Daedric quests (and then Companions as they seemed interesting). Crafting, alchemy and enchanting when I get the chance but NOT power levelling.
Walking everywhere, I am at level 32 and have discovered far more locations then my first run through when I tried to do
everything.
Not to mention, I have discovered some awesome scenery.