In my latest playthrough I'm not using fast travel as much as I used to. Mainly before I got a new PC to run everything at max so walking through a beautiful scenery is actually fun now...

Here's my personal rules about fast traveling:
- Don't use carriages to go to a
new city, You have to discovery it yourself first.
- Don't fast travel to any place *other* than the
big cities.
- You have to walk/ride the way towards your objective from there, you can choose whichever city is closest.
- Fast travel
back to a big city is allowed.
Also very important: don't abuse your horses mountain climbing skills. If you can't walk it normally don't use the horse to go beyond anyway. It cheapens the exploring experience imo.
[Edit:]
My reasoning for the "
no fast travel before the objective/mission but is allowed after" rule is simple: going on a mission is exciting, where the cliche "the journey is more important than the destination" is often in effect. However once you're done, you just wanna unload your loot and start a new journey. No need to be all anol about that and waste your time walking the same path you took to get there.
Just my 2 cents.