Well, it's pretty immersion breaking to me to go to the Imperial Library to get the lore. It should be in-game. And the books don't cover it.
It's a pretty extensive lore. It really can't be covered in one game, not unless there are more books in caves than cliffracers in Morrowind (before Jiub's intervention

). And let's face it, 99% of people won't read it.
Lore should be in-game, sure, but it already is and covers the most important events, places and people.
I'm kinda curious what
exactly it is that you think books don't cover. Obviously, there are references to the events of past games, but they can't be explained in one book. There are parts of history that happened in the novels or in the spin-off games. TES universe is large... There are terms not easily understood by someone new to TES - these are usually the terms that people on the Lore forum can discuss ad infinitum. Again, no easy explanation. And then there are caves and ruins that don't have any history, or their history is long forgotten.
I usually visit the Imperial Library for an in-depth study of a given subject, for things I wouldn't be able to find by myself. That's basically what libraries are for, isn't it? But most of the lore I know I learned from in-game books. Of course, there are always issues not entirely clear to me; no number of books will help me understand, I need to actually talk to people who know more than me and hear what they think. That's what the Lore forums are for. It doesn't break my immersion because it's something that
I do, not what my character does.
The bottom line is, I believe the in-game books already have all the players need to know. It's impossible to put the whole Imperial Library into Skyrim.