When I was building my armory, I bought everything that was on the building list. However, for whatever reason, the walls didn't show up. They eventually did show up on the list, and I built them, but I have to say that I rather liked the look of an armory without walls. The two doors inside of your home, instead of taking you into the armory, took you outside to Skyrim. On the outside, it looked like you had a patio. There wasn't any furniture, nor the option to add furniture, but it was an open, outdoor stone patio with a porch covering it from above. The porch was still fully functional, so I could go up to the porch and still have the patio underneath. I really liked the look of it.
I'm not sure if it was a glitch. It may have been, since the walls never appeared on my list, but could you deliberately leave a house unfinished to get the effect? Without the ability to place furniture, an outdoor patio doesn't look very impressive, but maybe you could throw some pelts down there. I think, in most instances, items dropped in the open world tend to stay there. I mean, the roads near Riften are littered with the swords of fallen bandits.