Liked the first one, consider it my favorite game. Martyr explained a good amount of it.
Bioshock 2 I did not like too much. Story was quite bland and not very interesting with the gameplay not having much difference to the first.The addition of the unnecessary multiplayer didn't help either.
I liked Infinite as well. Playing it for the first few days the 'new game' feelings are still kicking and you think this new game is the best game ever. A few weeks later you begin to realize some of its flaws and that it wasn't as good was you really thought. I believe Infinite one is the most overrated one out of all of them and if anything has "dumb downed" itself compare to previous games. The story is good yes, the atmosphere outstand. But the rest? The powers have simply come to just doing damage (excluding Possession). Levels are more linear than ever, previous Bioshock games at least had some expanded levels with plenty of exploration that could be done. In Infinite its sort of a straight line in most places with scripted enemies from one area to the next. I did find the gameplay fun but it wasn't anything too special. Overall, do I love the game and had fun with it.
All of the Bioshock games have a fantastic story however (Bioshock 2 arguably) and I find it to be some of the most fun I could get out of an FPS. Both a good story and some very fun gameplay.