Why does Caesars Legion attack me when my Reputation with them is Neutral?
I have to get into the camp and talk to Caesar, I had to kinda sneak my way to get to cottoncove so their boat would take me, and now if I get too close to one of the soldiers or something, they all start attacking... Why?
If I don't wear their Armour, they attack me, so I'm wearing recruit Armour... But yeah, as I said, My standing with them is neutral, how do I fix this?

I don't know what combination could work for you but I'm vilified by the Legion and still had to do the quest where you save the Weatherly family in Cottonwood Cove. I first tried saying I was a friend of the Legion but that didn't work. I was attacked. Then I did a speech check and that worked. I think it was about wanting to buy slaves. ick. I was able to free them but why that worked, I don't know. I wasn't wearing any faction clothing and had no companions with me. Speech checks are working well so you might throw some points on Speech when you level or use skill magazines and do it later? Good luck.

Who do you have as a follower?

If you have Boone, it's often an issue that can only be resolved by dismissin' him.

Yeah, Boone doesn't really care for your affairs; if he sees Legion scum, he's taking the shot... no matter what.

Though, I don't think this is the case for you, as Boone warns you when you're near a Legion camp, giving you the opportunity to dismiss him.

Ah, thanks guys, yeah It was because I had Boone, he wasn't with me, but he was still a follower.
Thanks guys, I really appreciate it!

I still had this problem without Boone. I was Neutral with CL and walking up to Caesar's tent, and got attacked randomly. Without my weapons, I died pretty quick

I just killed them all and got the key off of a dead body. Easy XP.

I love fighting legion because of a few things:
1. There armor isn't great so they have low DT
2. They love running up to you
3. I have the shotgun surgeon perk and 100 Gun stat
4. I have Dinner Bell

And yes legion will attack you regardless of your standing with them if you have boone as a companion. Also double check what armor you are wearing

That's funny, I'm vilified by them but they don't attack me when I encounter them in the wastes. If I talk to them they say something like "do me a favour and attack me, so I can kill you." I assumed it was a story thing, you've got to be able to approach caeasar to move the story along (I assume, I haven't got that far yet) so they have to stay neutral up to a point.

You're not wearing NCR armour are you? Don't feel bad, I did that once.

i think if you have any good rep with the NCR they will be KoS, until the point you get the mark of Caesar.

doing the Primm and Nipton stuff will give you good NCR rep iirc.

Do you have Boone with you? I had him when I went to Cottonwood Cove and they were hostile. But I decided to do it again without him and they were green. I was still vilified, but they were not hostile.