All of the weird bugs are still present. Like the one in 1st person pov, where your character holds your gun way up in the air, and you can see your arms sticking straight out. Or your guns turning invisible. And some quest bugs where quest related npc's are underground and accessible. And other minor bugs that were present since the games release. I can't remember what else I saw today.

So my question is, do you need to start a new game for the patch to have effect?
What did this patch do besides tweak some of the guns stats?

The latest patch was said only to fix "stability" issues, and also made a lot of balance changes to weapons. I don't believe that it was meant to fix the "gun up in the air" graphical glitch or invisible guns, or any quest/NPC bugs that still existed, mostly it was meant to fix as many freezing issues as they could find (and from some of the forum posts, they didn't find them all) before the next DLC came out.

I would guess the developers would tell you to get the most out of the patch, you'd be best off deleting all your old saves, clearing your cache, deleting any DLC, then reinstalling the patch then the DLC and starting a brand new game. Not the most practical way of doing things, but you can understand how difficult it would be for them to patch things that were already broken in every specific save, and expect those fixes to hold when you'd already saved them when they were broken (if that sentence makes any sense).