OK, I was always buying horses with my first character, but got tired of them dying or having to track them down if I had to dismount for a fight. They would fight unless it seemed like one too difficult for them, then would run off out of the way, then I had to find where they ran off to or fast travel so they'd show up.

Built the houses with second character, and put horses in the stable for the aesthetics, because it was a nice touch. They stayed put, I thought it pretty cool. Yesterday, role played with the kids outside, and decided to take a quick ride around the property to scout for any chicken killing giants, and now the damn horse is just like before, showing up if I fast travel, and won't stay in the stable. I don't feel like going back to a previous save, as I spent way too much time upgrading loot and traveling to sell it all, so I am just dealing with it. If something doesn't kill that horse soon I may do it myself just to be rid of the annoyance factor.

So, FYI, if you don't want your horses from your manor houses following you, don't ride 'em! Just leave them in the stable for looks....

Wondering if I do kill the horse, will I be able to have Gregor buy another? Hmm. May have to save and try that.

I think you can just kill it/have it killed and buy another for 1k

Thank goodness I didn't ride the others. LOL 3 horses following me around? Nooooo. Hate having to kill a horse.

If you did Dawnguard just leave your horse in the stable, conjure up Arvak and ride him for a few seconds. He vanishes when you do not need him and your real horse will stay in the stable. And you can only have one horse travelling with you, so even if you rode all three, only the last one is going to travel with you, it has been the same in the game since day one.

My first character finally got Shadowmere to stay in the Riften stable with his other horse when a stray horse turned up near Riften. Left Shadowmere there, ran down the road, got on the stray for a bit and then left it. Stray horses do not normally stay with you when you fast travel. Although Braggen acquired a stray named Frost from a deal gone wrong who stayed.

Well, I'm 1.5 for 2 on their fast travelling with a character:

-Case 0.5: Stray horse teleported about 4 times with a low level vampire character, coming in handy for an epic flight in broad daylight away from a very insistent wipsmother, then vanished.
-Case 1.0: Stray horse teleported a couple dozen times until a character got sick of his wandering away post-dismount and bought a "real " horse. (Left him at the stable, but he didn't even stay for the oats, just started walking off again.)

I must have been lucky with mine then, he returned to his original location and vanished. Braggen did not fast travel for a while though, since he was exploring the area finding all the locations. He eventually fast travelled back to the Riften stable to check if Shadowmere was still there, and no other horse appeared. Travelling with a horse had annoyed me since Frost and his suicidal need to attack everything. Shadowmere never got out of the stable till we got mounted combat, and Braggen's other horse is still there happily eating the oats. None of my other characters were interested in buying a horse, or being given a horse. Both used strays to test out the mounted combat, and I think all ended up dying, Aussie's through some strange glitch that whizzed it off at super speed and killed it when she dismounted.

Braggen also used a handy stray when chasing a courier for something, and that one vanished never to be seen again as well. He was on a quest so would have fast travelled shortly after.

I had just a regular horse I bought that would attack anything, he didn't last long! They were fun for a while, but I got tired of having to track them down, and trying to keep them alive.

Blasted horses are seriously annoying. That's why I usually stick with taking wagons.

Here's a tip for keeping your horse alive.

When you're about to get into a fight, dismount quickly and shout the 1st word of Kyne's Peace. The horse will become passive & wander off a short distance. All enemies will immediately ignore your horse. Once you've won, your horse will still have the blue, pacified aura around it, making it easy to find.

Cool! Thanks for the tip.

Here's information on how to get this useful Shout: http://elderscrolls.wikia.com/wiki/Kyne