I explore all over the map and go wherever I want and don't worry about it. The weirdest thing I've encountered is that the cave, tower, whatever repopulates itself even if you've cleared it and you might have cleared it very recently. So I've killed, say, the "head vampire" or whatever and you'd think there would only be one and then he'd be back the next day after I picked up the quest and all the exact items in the location were in exactly the same places. So it's a little spooky in that way...or a fun repeat. Whichever way you look at it.
I've only gotten stuck with one quest item this way:
Spoiler I got stuck with a named drum for someone at the Bard's college. I hadn't picked up the quest yet, and then I got a dialogue option with her in Solitude and she gave me my reward for returning the drum to her but it was stuck in my inventory. Yesterday I found a named Lute and I left it in the location where I found it. I didn't want it stuck in my inventory as well.