I've been playing a mage recently, but recently I realized my long-term follower, Lydia, wasn't there. I definitely hadn't killed her, so I went back and checked Breezehome. No sign of her. I decided to see if I could get someone else to follow me, and they could, meaning she wasn't following me anymore.

I kind of got over losing ALL my dragon bones/scales, and got a new follower, J'Zargo. One minute he was there, he helped me kill a boss, and when I fast travelled he wasn't there anymore. I check the place, he wasn't there, I checked the college, no sign, and I was allowed another follower.

What's going on?

I have seen my followers just stop for no reason. They usually come back when I go into another area. If you are on a PC, you would be able to get them back. I don't have that luxury. Maybe they are conspiring against you? Were you mean to them?

Lydia seems to get lost quite often, but she also seems to always (so far anyway) turn up eventually; fast travel, or go thru a couple of doors/entrances

Well, you treat them like mules what did you expect. I’ve had followers get knocked into a wall by a swinging blade and if they can be killed they’re not coming back. They all seem to fall behind when running through Skyrim, maybe someone or something attacks them while you’re out in front. Lagers are easy prey. I do it to the Thalmor all the time.

Every once in a while they get stuck in some world geometry and won't come back home. This happened to me once with Mjoll. I spent WEEKS worth of in game time fast traveling, resting, visiting all my homes, and checking Riften top to bottom for her... assuming she had just returned "home" somewhere.

I ended up using the player.moveto console command with Mjoll's ID and it took me to a spot where she was stuck in Blackreach where I had been adventuring a long time ago. I then used the enable/disable command and she was no longer stuck.