Way too hard? I'm playing on master and i still find it fairly easy and have not broken the game in any way.
I presume you are playing on novice right? If you find it hard it's the reasonable thing to do. I don't know how novice is like but a friend of mine plays who plays on apprentice just 1 hits everything. You must be doing something seriously wrong.
Up until level 50 I was playing on Apprentice and found the game to be pretty challenging. I was dying a lot, and no one who has seen me play would say I was doing something wrong. This game is a lot harder than Oblivion, it provides a challenge no matter what level you are.
So at the OP, it is not just you. The game was designed this way, you just need to adjust how you play for different encounters. I will fight a single Draug Overlord face on if need be, but I will never fight against two at the same time. I am level 55 now and I still run away from fights because they are too difficult. All part of what makes this game so much fun.
When you enter a dungeon for the first time (usually), everything in it levels to a specific range based on what your level is. Obviously any overlords in there will be a level or two higher, while standard draugs will be a few levels lower. So if you find a fight too difficult you can leave and come back after you've levelled up several times.
That does not work every time though. There are certain dungeons (like Labyrinthian I believe) which has a specific level requirement. And many of the Daedric quests have specific level requirements as well. You can get them whenever, but you will be fighting mobs preset to a specific level and so might find the fight almost impossible.