Its not that complicated, true. There is a stigma associated with it, and that is that. You can try to deny it, ignore it or justify it. Weird is weird, culturally speaking. No reason to question how comfortable people are with their sixuality, no need to attack anyone.
Ask 20 people at your local supermarket if its weird to wear women's clothes as a guy, how many of them do you think are going to say yes? no?
People on the internet are an odd sort, and pretty liberal when it comes to topics such as these. I don't think that's necessarily a reflection of the people, but a commentary on the anonymity of the internet. Most people are much more conservative in public with these things IRL.
There's nothing wrong with it, I've played female characters, and being attracted to a sprite is likely more of an indicator of one's social life more than it is of one's sixuality.

Different....strokes.... as it were. /shudder
The reason most people choose their own six is because its easier to imagine yourself in the role, the reason for the other?
Boredom, equipment, combat differences, preference, etc...
All preference.