When you do the peace treaty quest, does the war end? or would they still fight each other? i don't wanna join either side, just want the war to end.

It's not really a treaty it more of a ceasefire and during the dialogue Tullius makes it clear that the war will continue once the dragons are dealt with

The war doesn't end, they just agree to stop fighting for awhile so you can take care of the dragon problem. You still have the option to join the civil war afterwards, even.

So, there is no way to end the civil war peacefully. My Nord wants the war to end, for brother to stop killing brother.

He hates this madness, is there no way for the two sides to be sensible?

The way to end the war quickly is to pick a side and see it through.