The trait I chose at the beginning of the game "four eyes" gives +1 perception when wearing glasses and -1 when not. The problem with this is that when you take the trait it automatically subtracts 1 from the perception stat, and just makes all glasses give +2 perception. The problems lies when trying to obtain perks. Even though my agility should be a permanent 6 I cannot get perks that require 6 agility because of the trait I am stuck at a permanent 5. Please help.

You mean Perception, right? It should not affect Agility.

I suppose if you pick Four Eyes you just need to be aware of this for future perk choices. I'm guessing you wan the critical hit perks that require Perception?

I thought perks went off of what your current stat was, not what it was without armor and implants.

No, perks availability is based on base stats without gear, magazines, drugs, etc.