1) Horse movement is changed to equal character movement-W moves forward, S moves back (without turning the horse), and A and D strafe left and right (instead of turning the horse). Alternatively, A and D can still turn the horse, but we absolutely must have a key binding that allows us to strafe the horse left and right.
2) Horse movement is not bound to camera direction: this means that the movement of the horse has no effect on how the camera moves, and vice versa. For example, I could pan the camera left or right without changing which direction my horse moves.
3) When using bows (or other ranged weapons, like staves), my character can yaw a full 360 degrees, and pitch 270 degrees. If you have no idea what I'm talking about, look up "yaw" and "pitch" on wikipedia.
4) Although not mechanically necessary to make mounted combat worthwhile, we should be able to cast spells while mounted. It makes no [censored]ing sense that we can use both hands for a bow and arrows, but a one-handed lightning bolt is impossible.
The reason I chose these particular measures is that literally everything I suggested has either already been implementd in other video games (i.e., Red Dead Redemption, Assassin's Creed), or can be done in real life by real horses with real mounted riders (i.e., mongols, equestrian clubs, medieval re-enacters, etc.).
Spells are an obvious exception to the above justification.