In the last game, anything we took off a shelf or a display case, anything sitting out in the open in a shop or a house, would not ever re-spawn.

I wasn't aware of that at first, and ended up with some very empty looking stores in my game. It kind of killed the mood.

My question is, in this game if I take the loot that's sitting out in the open, will it eventually re-spawn or is it a one time thing?

It will only is a quest directs you to that location.

A quest is going to direct me into the shelves of my neighbors house and the general store?

I think he meant 'only if'

So again... If I steal everything out of the armor shop that's sitting on tables and such, they won't re-spawn unless a quest sends me there (which is pretty unlikely unless I misunderstood something).

Is that accurate?

Funny thing in Oblivion is that food on tables respawned but not the plates and bowls. However if you fireball the table everything would be put back.
So the solution is a bit more complicated, then the cell resets it put everything back on place and repspawning objects is recreated if removed.

Setting bowls and plates to respawning would solve this, but it would be a lot of work and might conflict with other mods who changed the cell.

Another fun thin in Oblivion who has to be tested in Skyrim is to place some armor on the table, on food respawn it will crash with the armor and you get a food explosion.