Horses are useless. They aren't fast and logistics will bore you really quickly.
Say, you're wondering with your horse,
- Saw a bear? If you immediately drop of the horse, your horse will get pissed and start attacking the bear, if you're playing on higher difficulties, bear may kill your horse before you get a chance to draw your weapon/spell.
- See a rare herb? Get off your horse, and pick it up, get on it again.
- See a dragon far away? Try to find a place where nothing will attack your horse, but will be close and distinctive enough that neither the horse or dragon will attack each other, and will allow you to find it again after you kill the dragon.
- Dragon jumps on you before you could see it? Tough luck, because that horse will attack the dragon no matter what. And the dragon will probably ignore your attempts to take it away from the horse and will land on the horse as soon as you get out of the range.
- You use a follower? LOLOLOLOL
This game really relies on your character to be on foot. Constantly getting of the horse to actually DO stuff gets very old, very fast.
Price? 1k gold? Pfft, pocket change. My characters don't loot anything below glass/ebony, I'm still swimming in gold just by looting smart. Last time I was encumbered was a week ago, for looting 2 dragons on a row. So price is not an issue, the issue is the mechanic itself.
Its kind of like putting your armor away before you sleep, utterly useless mechanic wise, perfectly logical role-playing wise.
TL;DR Don't use them, they're not worth the
time you spend on them, they are only
worth it, if, you like
role-playing. Even then it gets old.