I have a hunch Bethesda assumed many PCs would be loaded with allot of stuff running in the background. AV software, chat programs, iTunes and stuff like that all running at the same time can really put a drain on system resources.
Personally I expect my old AMD dual core rig (2.6Ghz) with 2GB RAM, a GT430 and XP to run Skyrim just fine. It didn't even bat an eyelash with Mass Effect 2 on the highest settings.
Agreed, with PCs it is not all in the numbers per se. Some people who actually maintain their systems and keep it running with minimal bloatware and other crap, actually get much better performance out of their systems. You can free up so many resources by simply unloading all of those silly icons in your taskbar, all of the autostart programs that you never use, and using something like Ccleaner to clean up all of the junk files floating around. A good defrag every month is important too.
My former PC commonly got better benchmark scores than other computers that had better specs...just because I optimized it and kept it updated and efficient.