Consider yourself lucky....I have tested this in several different homes and several kinds of containers. It does happen, and you can in fact lose your souls. The only reliable way I have found around this is to place your filled gems in a chest, and hope that they stack. If you have a stack with, lets say 10 lesser gems in it, you can safely remove 4 of them, sometimes even 6, but when you grab that 7'th one it will be empty, about 75% of the time.
Yeah, stacks seem the least likely to blank in my experience too. But in my testing, it only seemed foolproof (given my limited data) if moving the entire stack in either direction -- and since it *has* to be a stack, any less than six is a pure crapshoot. Even with the stacks though, as I mentioned awhile ago, they can still get "maked" but appear intact until moving them to an otherwise reliable inventory -- i.e. a followers.**
I know this isn't a game-breaking bug but, nonetheless, it's definitely the most annoying one I've bumped into since all the seeming workarounds are also either bugged or limit RP options.
** Edit: I should clarify this I think with one of my tests in Breezehome:
Have ten filled lesser soul gems in my inventory (no blanks) and transfer the whole stack to an otherwise empty container; they stay intact. I move the whole stack back to my inventory; they stay intact. I move partial stack (seven of the ten) to my follower's inventory (again no blanks); the stack of seven loses their souls.