So I'm on the main quest where I gotta find info on the elder scroll... Says to talk to the grey beard dude or go to the mage college.. I talked to the grey beard guy first and he said the mages would know about it. I go and talk to the book guy since he has a quest marker over his head. I talk to him and he says something like ill give what I know to the grey beards. But nothing happens after that he still has a quest marker repeats the same thing when I talk to him. Went back to the grey beard and he says the same things also... So is my game over since I can't finish the main story cause I'm gonna be pissed. This would be like the 5th quest I can't finish now

There should be a few books on the desk in front of the librarian guy. Read those and they'll give you the quest marker for the next bit.

Edit: or am I mixing that up with a mage guild quest?

youre not mixing it up, OP needs to read the books.