Without posting any spoilers, I don't really fancy doing his quest right now. I can ask him to stop following me, but then he goes off doing his own thing (I hoped he would stay near to where I left him). Does he go far, or can he end up just about anywhere? I'd rather leave his quest until later, and if possible have him by my side at that time.

I believe that you go back to the road to Falkreath and you will find him where you originally found him

Sometimes, life inconveniences you.

I always just go to the shrine without him and he spawns there after activating it.

I'm at the part where I've been sent

I'd rather not go all the way to find it because I haven't explored that whole area yet. Falkreath is the furthest west I've been, and I don't want to rush the exploration.

You don't need him for that he will just reappear at the shrine when you go back and activate it.

If you talked to him the first time, he will be at the entrance on where you need to go, if you tell him to not follow you. If you tell him to stop following you during the second half of the quest, he will return and hangout in the shrine room itself or should at least.