I've just completed the 3 quests for the Bard's College in Solitude that require you to get the 3 instruments back: a drum, a lute and a flute. I went back to turn the quest in without any problems but a few minutes after completing the quests i noticed extra weight in my bags, I still have the three instruments in my bag and cannot in an way remove them because they're still marked as quest items even though I've already completed them.

Your lucky you can even get into Solitude. When ever I attempt to enter that city, me and every NPC around fall right through the damned map. I can't continue the main quest because of this bug.