I think to make them more useful, a few things need to be considered.
1: Can I name my horse when I buy it? This might seem minor, but when you name it, it becomes less of a 'vehicle' and more of a secondary companion. It stops being "a horse" and becomes a character. Similar to enchanting weapons and naming them, it gets you into the game. Immersion!
2: Can I interact with my horse? i.e. tell it to go somewhere, to wait somewhere and not attack (ridiculously necessary), hold items?
3: Can I upgrade it at a stable? i.e. increased carrying capacity, increased armor (which would be similar to human armor, with the appropriate levels and types, like iron, steel, ebony, daedric) This would keep it from dying easier, help it deal more damage if it has to. Enchantments could be interesting as well. Imagine, enchanting ebony horse armor with fire, and being able to deal fire damage to bandits you trample under hoof. Badass.
4: Can I attack from my horse? Nothing advanced, but similar to darksiders, whatever my primary weapon is, I can take a swing from my steed as I gallop past. Or draw a bow while standing still. (Camera swings in to over shoulder for aiming).
5: Can the horse have a health and stamina bar while riding? Similar to the enchantment level that appears by your weapon when walking around, this would be useful so you know when you're going to have to save some stamina when galloping from a fight, or to one. Or when you need to get your 1000 septims out of dodge before it goes down.
If anyone has other ideas, feel free to add some. Or, berate me for being an idiot, either one.