The fox really does seem to lead you to stuff you've not discovered yet.
I've got every single thing on the map, as far as I can tell (all main quests, most other quests all done, level 69 now (with no "exploiting") )
I noticed that when the fox popped up in front of me, it did seem to kind of look back at me and then proceed rather slowly. I followed it the last
couple of times while it was doing that, right up to a thing I'd not really discovered yet (these are not "Map" points, that you "discover", they are just
things out in the wild.) Like one thing it led me up to was a big table, with a skill book or something (I'd already read, unfortunately) and 4 skeletons
who warp in.
I'm going to be kind of looking out for that fox from now on. People on other sites have said, "Yeah, but if you go off in any direction you find things you've not discovered...", but that is not as valid when you've pretty much got every location on your map cleared.
Next to going to jail, or paying a fine ntp a guard (who will then take everything stolen you're carrying or arrest you on the spot), how the heck do you get rid of a bounty??
The third option that I get, is "I'm with the guild and I'll pay 12 gold", or whatever amount. That pays off your bounty at a lower rate (without your stolen items being taken.)
So its good to become a thieves guild member as low as possible.