You should still be able to get rid of the items once you complete the quests. When you find an important item, you may or may not have a quest added to your journal. Either way, it's not a glitch; it was the same way back in Oblivion. Basically, it helps to prevent you from selling or losing an essential item and subsequently breaking a quest or even an entire questline. If you have completed a quest and the item is still in your inventory, however, you may have actually have a glitch and you may need to go back to an earlier save.