I've noticed that whenever I use the D-pad hotkeys to switch between weapons, the game will freeze for several seconds before actually switching weapons. It's not a gamebreaking issue, but it really breaks the flow of combat when I'm trying to swap between my sword and bow.

Is this a common issue? Is there any way to fix it? Any help is much appreciated.

I have the same problem and so do other players. Unfortunately, there is no way to fix this issue as far as I know. You're right it is frustrating though, because sometimes by the time I switch (or draw) my weapon, the enemy has already pretty much killed me. The issue seems to have gotten worse since I installed the Dragonborn DLC, but there has always been a slight delay.

Ah well then. I guess I'll just have to deal with it.