I think it's more a case of what's appearing on the compass that ruins the immersion. You know, the 'I'm 500m away from that cave entrance and it's on the other side of a mountain but I still know it's there' problem.
Also, the quest markers are a problem (too much 'hand-holding'. Actually having a basic compass and a map suits most people I think.
It's always good to know how to remove it altogether though so nice one!