Being able to drop Quest Items is something they are looking at for the next patch. Until then, just know that they do not add weight to your encumbrance, so other than the room in the list, they take up no capacity in your character's inventory.
They fixed the issues where getting these things early messed up a couple of quests.
I'm on a quest to get 24 special gems. My character has 5.
I don't care if they weigh something, or not. My main complaint is that the stupid things don't stack (note*). Now, in addition to those 5, I have 6 other quest items - all different - so that takes up 11 lines in my inventory. Plus I have some other quest items in my weapons and apparel columns.
*Note: An other character has 4, and they stack nicely.
At least give us an option to store quest items in a chest at home - or make a "quest items" colum/space in your inventory, so all quest items go there while you're carrying them around.
Thanks.
E