One cave up by Solitude associated with a lighthouse quest (can't recall it at the present moment) has a chest in the water, and I remember finding quite a bit of gold and an enchanted weapon in it, so that was interesting. Haven't really found any underwater finds, unless the orcish sword sticking up out of the water near Whiterun counts.
There are two in this cave (Black Marauders home base). One int he first pond at the entrance (down to the left, straight out from the broken boat), and one in the broken, sunk boat at the scene of the end battle, right where the boat is broken in half. There is a book on the top floor of the scaffolding that tells the story of the one in the sunken boat, but no mention of the other one.
There are a few other, one in the ruins where you fight Rahgot, etc. Mostly fluff and fodder though. 10 to 30 gold coins, some weak armor or iron/steel weapons. Nothing significant that I've found.