I've only gone down twice. Once to a Draugr Overlord because, in true zombie movie fashion, I didn't notice that he had three other lackeys moving to attack me, and once when two snow saber cats teamed up on me before I could even pull out my weapons.
I was killed three times trying to best a Krosis before I finally succeeded. I was killed by spriggans twice. I was eaten by a dragon. I was smashed by a giant, on two occasions. I slipped and fell into a river, dying. I was brained by a bandit with a warhammer. I was set ablaze by a fire atronach, twice. I was lightning'd to death by necromancers, twice. I was mauled by a vampire. I was smashed by a mammoth. I was ripped to shreds by both a Krosis and a dragon at the same time.
I think I've made my point. There's a lot of danger in Skyrim, and you have to have caution.