I am on PC, and my save number is about 4000 now. Should I be concerned at all as that number gets higher? I know in some other games it can affect stability.

Save number doesn't matter but a large amount of active saves can cause lag and stuff.

someone on here has claimed to have made over 20,000 saves.

Well I'd recommend deleting a lot of those older saves. Its never a good thing to have all of those small files continuously building and adding up.