For me, it was Giants, Mammoths at start. Then i realized that i should avoid to being hit and move fast+much in fight. Well, in almost all type meele fight you wouldnt just standing still, so its realistic.
At lvl 39 on expert, they are not a problem. At lvl 39, with 92dmg great sword, 380 light armor, was nothing a problem. Thats bad because if i start new char, ill die much again because i will forget that i should move in fight.
I had a great duel with a friendly npc. I had bounty on me so got a bug, where some friendly npc will attack me. The fight was decent, When i tried to power attack her, she bashed me, then attacked, blocked my normal attack etc. I missing the selfhealing NPCs. If she would heal herself, the fight would last longer

(The timeslowing effect when you block a powerattack, is bad thing imo. made the game much much easier for me)
Edit: Ohh i forgot... mages are still a problem for me