instead of joining with them to get info on elder scroll, can i go straight to the mad man?.. i really don't want to join the colledge to be honest..

Yes, you can go straight to the mad man.

You don;t really have to join, just say your dragonborn, Fus Ro Dah her of the bridge ( not really) and then don't take the tour.

You'll find that the College is a very difficult starting point for the journey to the madman. Much easier if you start from somewhere on the north coast of the mainland.

you end up getting quests for the college and that irritates me like the Bryjolf one lol so thanks for the replys ! im going straight to the bretha