I used to think that too, but honestly I can't see the Empire as "good" as the Stormcloaks. I can understand them going for the Greater Good deal(which is much more accurate in my opinion, given the circumstances), but that means that they aren't necessary the "Good guys", even if they are not the "Bad guys". The Empire simply have too much dirt on them to be called "good" as far as I am concerned.
Well, that tends to be the way Empires go. They've done what was necessary to survive.
That being said, I agree with the OP. I think the best course of action would've been siding with each other to drive the Thalmor back.