Is it possible to remove these crazy people as an issue totally? Where are all their hideouts?

you can kill them but they are like a bad rash or a crazy ex girlfriend they just keep coming back

They'll respawn just like bandits and most other creatures. But you can find their hide outs strewn just about everywhere, with the most concentrated in "The Reach". Which is a vast area around Markarth stretching deep north and south. Hag's rock redoubt, forsworn ambush point, hag's end, etc. are just a few examples.

If you regularly visit their bases, just do an fast travel round every month it should stop them from respawning.