When you go in to solitude for the first time there is a guy getting killed to the right, he has a talos amulet on him, always!
Yea, I thought about it, but it's glitched and will become a quest item and every other Talos amulet you pick up will also be marked as a quest item and you can't drop it. And if you want to do the quest in order to get rid of the amulet, the person will not recognize that you carry the amulet, because it's glitched. Therefore, I would NOT pick up that amulet, unless you are on PC and can "finish" this quest via console commands.