Assuming you're not interested trying the first two episode and their 2D graphics, let's have a look at the parts with 3D visuals:
Grand Theft Auto III: Only one I haven't really played, because it was incredible power-hungry when it was released. Even the recommended specs weren't enough to run it smoothly, so I skipped and somewhow never tried again since then.
Vice City: Wonderful trip to the 80's, probalby the most profilic GTA ever released. In a nutshell, its Miami Vice meets Scarface, supported by an exceptional OST.
San Andreas: The boys in the hood are always hard, especially with that early 90s style. Incredible variety in activities to do and places to go, but for me, it has less soul than VC.
Grand Theft Auto IV: Overall, its a way realistic experience, which fits as it is the tale of an Eastern European immigrant in the USA. Great experience with a focused storyline, but technically still a cheap console port.
Lost and Damned: Motorcylces, guns and narcotics. And "should've-been-better" gang AI, respectively. Otherwise its a quality expansion, which really shows you another side to the main storyline, and Liberty City itself.
Ballad of Gay Tony: Every gay bar owner needs a straight right-hand man who're not afraid to do dirty jobs. In exchange, enjoy high-end stuffs from the ground to the sky. Another excellent expansion, worth every cent.
Really, you can't go wrong either of them, watch a few gameplay videos on YouTube to decide whether the outdated graphics are bothering you or not. If latter (or you simply has an old PC), start with Vice City, then proceed.