I don't think that that's really fair... or right. I think that a lot of people just don't like seeing the Brotherhood being greyer, yeah you could argue the practical security reasons of the slave-collar and such (I completely agree) but a lot of people just take things at face value and make stubborn judgements far too early.
That may be true to some extent but then why would many of them support an even greyer faction like the NCR instead then? It is interesting. According to the Vault, the BoS leadership's Karma is listed as "Good" (even Hardin) and the NCR leadership's Karma is listed as "Neutral." Even Ambassedor Crocker, the dove, is Neutral.
BoS - Elder McNamara
Karma: Good
http://fallout.wikia.com/wiki/Nolan_McNamara
BoS - Head Paladin Hardin
Karma: Good
http://fallout.wikia.com/wiki/Edgar_Hardin
BoS - Senior Knight Lorenzo
Karma: Good
http://fallout.wikia.com/wiki/Lorenzo
BoS - Paladin Ramos
Karma: Good
http://fallout.wikia.com/wiki/Ramos
NCR - General Oliver
Karma: Neutral
http://fallout.wikia.com/wiki/Lee_Oliver
NCR - Colonel Cassandra Moore
Karma: Neutral
http://fallout.wikia.com/wiki/Cassandra_Moore
NCR - President Aaron Kimball
Karma: Neutral
http://fallout.wikia.com/wiki/Aaron_Kimball
NCR - Ambassador Dennis Crocker
Karma: Neutral
http://fallout.wikia.com/wiki/Dennis_Crocker