I have yet to take sides yet the small patrols of stormcloak soldiers i run into seem rather hostile. Small groups of legionnaires will talk to me (unless they are escorting a prisoner then they get edgy), lone stormclocks will talk to me, yet when there is a few of them they want to fight? :S

Is there a reason for this?