The way I look at it that's not what e-readers are for. Some books are just pretty objects and won't be replaced by technology until we can simulate that experience. E-readers aren't intended to replace art books, book collections, etc. I use mine for portability (again, I can bring hundreds of books with me in a 7 oz. package...a library only does you any good when you're standing in it) and for books I'm reading from beginning to end. I can lie in bed holding a tiny, light device and just tap or click a button (that my thumb is already conveniently resting on) to turn the page. It's the ultimate in literary laziness. 
Just out of curiosity, how often does one need to flip through a novel? Alternatively, can your paper book search for words and phrases in its content? 
There are also two sides to all of these points. When I turn on my Kindle I'm looking at the page where I left off. If I switch to a different book, it remembers where I was. I can also puts marks anywhere in the book that are then indexed for later use, highlight passages (also indexed), search for mentions of characters and places (wait...who was that again? *search for name* Ahh...right.), get statistics about the author's writing style based on volume and frequency of word use, tap a word and see a dictionary entry for it, etc. Sure, there are no pages to flip through, but there are a lot of other related advantages.
Of course, it's all a matter of preference. For me, books are great for admiring and poring over, and e-books are great for the actual act of reading.
I wish Amazon would give me free e-books when I buy a physical book, though. When you buy an audio CD from them a lot of them come with free MP3 downloads (they call it AutoRip), but when I buy a physical book I don't get a free copy of the e-book. That would be fantastic.