Can't you just look at the map?
When we first started traveling I wandered off a bit and they broke off from me so I was using the spell to stay on track.
This. It will have you trying to go through walls, under (or over) mountains, and other, equally inaccessible places. Use your map.
The map isnt really the best way to figure out a where you need to go once you are already lost. There are a few times when it sends me to a rock but for the most part it helps out a lot and is quicker than bringing up the map.
But my point wasnt about the spell walking me back and forth but that Delphine and the guy (forgot his name) just kept walking that same path. I was looking at them for 30 minutes just walking back and forth.