However with this new card with all settings on maximum the experience is buttery smooth. and never drops below 60fps and I no longer experience problems with big fps dips caused by the VRAM limit.
Just wanted to fill people in if anybody else was in a similar boat and thinking of upgrading.

Probably grabbing a 6 series Nvidia card around Xmas time.