The way they should fix this is by letting us have multiple hot swaps when using the left and right directionals.
The best way to fix this IMO would be to use up and down for items and weapons. So have up be everything for the left hand and down could be everything for the right hand.
Then left the left and right directionals be your left and right handed spells. So you can easily swap them on the fly without having the pause all the time.
It just seems the way its set up now its bugged and you always lose your default set which gets frustrating
I love this game but this is the one thing, they really need to fix with a patch.
AT the very lest when give us a way to go back to our default set after using the left and right D pad.