My new character will join them and also go with Yes Man. Will he ask me to destroy the BoS like CL, and House? If so, is there a way to avoid it and still carry on with Yes Man. If there is no way to avoid it, how do I go alone and not help any faction (NCR, CL, House, or Yes Man)

I haven't had him ask me to destroy BoS, just meet with them.

no... yes man is not in charge, the whole point of yes man is that you are. you can tell him that you want to ignore BoS and that's the end of it.

i don't like them though, i destroy them in every playthrough...

so can you tell him that you want to ally with them?

you can't really ally with the BoS. they won't help you during the final showdown like some other factions can.

Ok. So far I've worked with House and CL so I didn't know if it would be a possibility to have them help if you don't destroy them.

Actually, this isn't true. If you are working for the NCR AND are liked by the Brotherhood, then you can ask for the Brotherhood's help in defending the dam. Even though the NCR commander will initially ask you to destroy them, when you go and actually talk to McNamara the option to team up will be offered. If you take this back the NCR commander, she will agree.

Finally, Brotherhood soldiers will appear to help you fight against the legion in the big battle.

I have heard, however, that if you have Hardin replace McNamara, then the option to team up is not available.

I think there's a bug with the Yes Man quest where you are only ever allowed the Optional ignore the BoS no matter how good your standing is with them.

Not true, I went the Yes Man path and was liked by the BoS and liked by the NCR. I chose BoS as to be ignored and later on in the game the option to convince them to help arrived so I took it and they were actually there helping.