That naked mage undernesth the meadery... During the thieves guild quest
That guy's brutal. It's the only quest I've ever done where I had to leave to get better equipment and then come back. Invisibility potion + backstab. In close melee he's a wuss though. So even if you don't kill him in one blow he's still very killable thereafter.
For me it's trying to fight elder dragons at the college in Winterhold. It's gotten to the point where every time I go there I will get attacked by one. Even with a follower to distract them it's still a huge fight. What I've learned to do is flee into town and hide under the porch of the inn. For whatever reason, they can't get to you there. Then when you figure out where they're going to land, you can run out, hit them a couple of times with a bow or a spell and then get back under cover.
I'll join in with some others who have mentioned wisp mothers. Brutal.
For those who hate Frost Trolls and that are on lower levels, they're actually easy and present great soul capturing opportunities. Paralyze, soultrap, and then hack away at them while they're down using a fire weapon or fire magic. Otherwise, all you get for the trouble is a little bowl of Trollfat. Pretty weak. As for up close and personal toe to toe fighting with trolls: don't do it.