Ever since starting my game, whenever I picked up an amulet of Talos and tried to get rid of it, it would say that it is a quest item. I have a few now that reside in my inventory and I'm unable to get them out. Any tips or help as to what to do?

Protip: Every time you're asking stuff like this, say if you're on PC or not. If you are, you can fix this with a simple console command. If you're on 360/PS3, odds are that you are stuck with them forever, though I doubt that's the case since this is the first time I hear of something like this and there should be quite a fuss about something as major as this if it was irreversible.

You need to do a quest

http://www.uesp.net/wiki/Skyrim:Return_to_Grace

There is another quest for a unique Amulet of Talos, but that has a unique in game name too.

You picked up Roggvir's amulet, didn't you? He's the guy who dies in Solitude. Talk to the little girl, Svari, and complete the quest (it won't actually happen because it's bugged, but you should be able to drop the amulet afterwards).

Alternatively, you can use a console command to get rid of the amulets:
player.removeitem 000cc846 X
where X is the number of amulets you want to drop.

Probably that Roggvir Amulet glitch.

Roam around Solitude for a while talking to the citizens. Eventually you will find out what you need to do. zIf I go any deeper, this would require spoiler tags.

Ok I will try it thanks.

ha definitely Roggvirs amulet. I had some fun with it because it won't get removed from your inventory as it's a quest item so my nord walked around the Thalmor embassy wearing nothing but a Talos amulet and a smile

I only wish Elenwyn had something to say about a nearly nvde nord in her embassy openly defying Aldmeri law

PS3. I fell into this trap unkowingly as well. Alot has been written on this bug, and I was able to fix the inventory issue using that, though the objective is still active in my journal (despite both completing it then killing someone to force it to fail).

She'd probably say something about you threatening he with your "spear".