I'm playing as a high elf mage and lately people have been saying "it's wearing a dangerous spell" or "please stay away from me while you have that spell on you" and they wont talk to me. But I don't have any spells equiped right now or any conjurations following me around! What's going on?

You have an active conjuration spell. Most likely...

Check your Active Effects Tab.