I just started to play trough the Stormcloak questline at third time but got this problem. After the battle of Whiterun the battle music stuck on and I can't get it off. Any ideas how to get rid of it? It's quite annoying to run around the Skyrim with looping battle music.

Save the game, exit to desktop (or whatever the console equivalent of that would be, if you're on 360/PS3), and relaunch. This happens to me a lot on the PC, and that's the only way to fix it that I've found. Even if I just quit to the main menu, I'll still get battle music there and when I reload a save.