I'm pretty sure there were three, but I cannot find the information on a third one at the moment. Here are a couple good starting points:
http://en.uesp.net/wiki/Lore:Akavir
PGE 3rd Edition section on Akavir: http://www.imperial-library.info/content/pocket-guide-empire-third-edition-akavir
In 1E 2703, the Akaviri tried to invade through Pale Pass (a quest in Oblivion goes into more detail about it). Reman Cyrodiil and his armies routed them but allowed some to remain in Tamriel. This led to the rise of the Akaviri Potentate Versidue-Shaie to usher in the Second Era. The "2920, Last Year of the First Era" in-game book series goes into more detail about it, although how much truth there is to the books is up to you (perhaps more so than most other books in the series).
In 2E 572, the Akaviri invaded again, led by Ada'Soom Dir-Kamal. Jorunn the Skald-King's Nordic forces, Almalexia's Dunmeri army, and a timely arrival of Argonian troops quashed the Kamal invaders, but it isn't entirely known how they were defeated. This directly led to the formation of the Ebonheart Pact.
One time, it was actually Tamriel who invaded Akavir in 3E 288, which you can find information on in the book "http://en.uesp.net/wiki/Lore:Report:_Disaster_at_Ionith."
I thought there was a third invasion, but I may be mistaken. But I don't see it in UESP or the Imperial Library at the moment.