Happens to me all the time when I'm aiming down the sights and shooting someone, and it has caused me to die more times than I care to remember.

Yes, it's quite sensitive. I got used to it after a few games and loosened my death grip of the controller.

It has happened to me a few times before. I try to battle someone and just end up meleeing someone.

I don't know if it has anything to do with controller sensitivity though because there is either a click in or no input whatsoever. I guess it's just a panic reaction and can be fixed by altering the button layout, although I don't want to.

I thought I was just being too hard, it happens everytime I ADS.

It seems okay to me, I do it sometimes when I tense up and accidently over press the Toggle.

Try playing on alternative button layout (like tactical in CoD). I still have issues with it, as in CoD I would drop-shot when I didn't mean to, and in Crysis I keep crouching in the middle of a firefight and getting shot in the head.