I have come across a few horses in the wild and rode them around, but as soon as I dismount they wander off where they please...

If I purchase a horse, does it stay where I dismounted it? Or is there some way to summon your horse?

And can horses be upgraded with armor?

Any and all info about horses would be great!

I don't tknow very much about them at all but I did buy one and it seemed to stay where I left it, except one time after a fast travel it was there waiting for me (but not later after another fast travel. I couldn't find a way to summon it to me.

If you acquired one without paying, it will wander around. But it will still be there when you fast travel. They tend to die more often than bought horses. But I've only lost one, and I quickly got another from the same place.

Overall, I would say a horse is worth buying. It's only 1,000 gold, which is a small fraction of the money you'll be getting as the game progresses. Horses are pretty useful.

Plus, you can name it Twilight Sparkle or Scootaloo. Or Apple Jack. Or Lord Reginald McBrittington III, esq.

I bought a horse and while it did seem to stay around the spot I'd leave it, if a random encounter with wildlife happens, it could be chased off (mine was almost eaten by a dragon. Lucky for me I was on my way back to the horse when it happened and so I saw the direction the horse ran so I was able to chase it down after the dragon was killed. I'd estimate it moved a good half mile or so from where it had been waiting for me.

Anything to keep in mind is that the horse can be killed as well, so if I didn't get to the dragon in time and kill it, my horse may have ended up as nothing more than roasted horse meat.

Does the horses in the different stables change very much from eachother?
In oblivion, the cheydinal horse was the fines, the bruma was the toughest etc.

So what happens if I dismount my horse and start adventuring.. and forget about my horse. Will I find it a few days later dead in a hole or will he magically appear at the next stable I come across?

He will magically appear when you fast travel somewhere.