But I also find it fun to go to a specific area when I'm short on a certain poison to stock up on ingredients (hjaalmarch marsh is my favorite go to place for poisons).
Agreed. Also, one of the main motivations for me to explore new areas is the possibility of finding new alchemy ingredients. After doing most of the miscellaneous quests in Whiterun, one main motivation for exploring more of Skyrim was to find more alchemy ingredients.
And having to experiment with ingredients to find combinations is also enjoyable. I find the Experimenter perk to be a total waste for two reasons: one, it takes the fun out of finding the combinations; two, it's a waste in the endgame, since it adds no power to your character.
I usually store the bulk of my potions and ingredients in my house, which works great, I have a limited number on me all the time (no more than 3 of each)
I do the same. I'll pick up every ingredient I can when I go out questing or whatever, and then I'll add them to my collection in my house when I return.
I don't mind carrying a whole lot of potions around, however. With a lightly armoured archer/dual-wielder, even with a large stack of restore health/stamina potions, resist potions, and poisons, I rarely get over 120 encumbrance.