I got all the money from the 3 people who owed the guild coin, but Brynjolf isnt accepting it. He keeps telling me he needs all of it. I'm confused...

Did you spend it? It's not kept in a separate pile or anything.

Have you tried running all of them again? Perhaps one didn't give all of it?

Make sure you have at least 300 gold on you.
Then make sure your objective says each person payed.

Kill him...that'll show the bastard for not taking your money.