So on my few play throughs I often collect items fairly quickly. I mostly end up vendoring them and the rest I'll give to my followers Lydia and Faendal but even they soon run out of space.

In all my play throughs I am usually forced into buying Breezhome so that I can have a safe place to store my goods. However, I don't even spend any time in the damn house. I don't really like buying Breezehome cause I find doing the Civil War quest part a bit immersion breaking upon owning a home in Whiterun after my first playthrough since I didn't really know what happens in the Civil War the first run. I am a Stormcloak through and through.

I prefer my first house to be Hjerim but its a bit expensive so its a while before I can come up with the required gold. However, I was wondering in the mean time where is a safe spot for keeping my belongings?

I know of a few places like the well known Alchemist Shack and Anise's cabin but I ain't looking for places too far out of reach. I am looking for something more accessible.

Also how does storage work in Skyrim? I have yet to figure out how do chests and barrels work as in will I lose anything if I store anything in them outside my own home?

Find the quest "House of Horrors" in Markarth. Telling you anything else would be a spoiler.

Anice's Cabin is safe. Its right by Riverwood.