You are right, every time I go there there is about 5-10 Legion soldiers that instantly spawn when I walk over there. Should I walk up to CFH and wait for the NCR to wipe out the Legion then walk in?
The large amount of legion troops can take out the soldiers at the camp fairly easily if you're not far enough into the game. Once you hit a certain point there will be heavy NCR troopers and Ranger vets around to take them out but there's a chance the legion guys may kill the NCR troops if you're early into the game. The only way to be sure everyone lives is to help them kill the legion which will obviously net you some bad legion reputation and if your game always freezes here every time you won't be able to do it anyways.
IF the game will allow you my advice is to kill off all of the legion troops and try to do any of the quests within three days before the dead legion guys reset...then avoid the camp like the plague until the problem has been fixed.