I was just wondering if you can attack the side you`re on, say Imperials, and swap sides to the StormCloak rebels? Or can you even accidentally change sides this way?

I`m not changing sides and won`t until a new game so it`s no spoiler to me to know, so I was wondering if that can be done?

You can't.

I presume the guards in the hold will attack you until you pay the fine unless you left no witnesses.

no,

Attacking your side won't let you switch sides. There is one opportunity near the beginning that let's you switch, but it involves how you handle the quest.

You just get yelled at, because they are on your side.

"Hey, I am on your side!"

Note, I have killed several imperials when I was on the imperial side as I was not that accurate in those skirmishes and if you walk in front of my blade, you get ground up.

Yeah, quite a few of my comrades perished from (accidental) friendly fire. They really shouldnt stand between me and the enemy...

As long as you have the Jagged crown, give it to the opposing side's commander and all is fit.

But I have a seperate question, is there a way to just change your mind and stay neutral on the war?

Yes. Don't do civil war quests.

just dont do it?

Do not take either side's OATH and follow the main quest.

Hmm. Ok. Cheers.