Why does the gauss rifle have an ammo clip thong at the side of it if it's Reloded through the other side?

That's where the actual projectiles are. The microfusion cells simply power the weapon. http://fallout.wikia.com/wiki/Gauss_rifle_(Fallout:_New_Vegas)

My question is why that clip holds an infinite number of rounds.

Elmo says "Science" is the key of all knowledge why ask why when "science" is the answer.

Same way the guns in Mass Effect hold an 'infinite' number of rounds- they fire very small rounds, very fast, so they can fit hundreds or thousands in one small clip. It's the power that's the limiting factor (and in Mass Effect, the cooling- which is why ME2 has coolant clips).

it looks like a clip but i don't thin it is one , becouse all other weapons when they reload in the animation all u see is the Cell type being inserted , i .. never watched a gauss rifle in all time i played fallout to see how it reloads lol

What you think is irrelevant, when the designers have explicitly stated it is one.