I'm not very clear on what RAM does in terms of increasing a PC game's performance, but surely the fact it uses half a gig of RAM may have something to do with lower performance than expected?

Even with texture mods the most RAM usage I've seen in task manager is about 800mb. I play on ultra.

Hmm, when i start the game i usually have about 1621MB of ram, after about an hour or so i can be up to around 3200mb, that's just over 1500MB

My specs and settings,
1920x1080 all settings ultra, just changed shadow distance (5000) and res (2048 for primary and secondary)
Windows 7
Intel® Core™ i5-2500K CPU @ 3.30GHz ~4.4GHz
8Gb DDR3 665.2 MHz
MSI P67S-C43 Motherboard
NVIDIA GeForce GTX 460 v2 1024MB

To the OP, how much RAM do you have in your system? If you dont have enough, it could be paging to the HDD.

I find the game usually uses about 1.6GB of RAM, and ~2500MB of my GPU memory.

Can't you alter the .ini to dump some aspects of the games graphics into your RAM? I thought I saw something like that for Oblivion tweaks somewhere in regards to adding grids.