They are considered clutter they are in all Elder Scrolls games. I find it fun to collect the junk that best suits my character and I find a use for it in decorating my houses. I would do that if only they fixed the exploding item bug.
Following someone's advice here I've managed to decorate my house just fine. What you need to do is drop the item(s) you want to use to decorate on the floor of your house. You then need to leave your house. Do not re-enter immediately. Waiting a few seconds is fine. Once you come back in the house, take the items
whitout putting them back in your inventory or in a chest/wardrobe/etc. You just pick them up from the floor and move them to the desired spot, and voilà. Should be okay afterwards.
Works like a charm for me. I put skulls on the chimney, added things on tables, placed quest items, centurion dynamos on shelves, rare potions and poisons near my alchemy table, etc. I was suspicious at first, tried it with only a couple of items, found it worked, and expanded it eventually.