Lol, if I had a septim for every time I've crept just inside a room only to have Lydia bumble into me saying, "huh?", I could buy out Erikur and put him on a slow boat to Oblivion.
LOL or you are trying to squeeze by her, while she's blocking your exit and then she says "Ow". I'm like what? Did I step on your toes? Well then you BETTER MOVE! sheesh LOL Illia does the same thing, she even got the same voice as Lydia, but she has a better attitude LOL
But yea, I also noticed that when they are in "alert state with a drawn weapon" they are even moving slower than walking. So, I make certain I only draw my weapon when they are close by and we are in enemy range. I mainly noticed that with my mage followers though, since they are ranged attackers.
I never had any issues with Vorstag. My Nord Brigid and her husband Vorstag, (both warriors, she is in light armor and dual wields and he's wearing Heavy armor and uses sword and board), make an excellent team! They always go on raids together! Unless, the mission requires her to go alone.

LOL
My stealth character never uses a follower, but my other 3 do so on a regular basis. I like to bring a "friend" along. Makes it feel more like an MMO, where you team up with your buddies! hehe