Oh, well can't I stroll into the opposition's headquarters?
Not after this... Of course, you could try. But I highly doubt it'll work. Any of the opposition's camps will be semi-hostile to you (they will tell you to keep walking, and attack if you stay), so I can assume Ulfric and the General will tell you to piss off as well.