But there was the Pocket Guide to the Empire, First Edition. I think a printed version was given away with Redguard.
I also have the 3rd Edition of the Pocket Guide to the Empire, which came with the collectors edition of Oblivion.
Redguard also came with a comic book.
As mentioned before, there's so much official lore - in terms of in-game books as well as in-game events, combined with all official Bethesda books (such as those already mentioned, along with the Keyes novels - plus Arena and Daggerfall came with manuals which had extensive lore introductions and bestiaries) - that a single encyclopedia would have to be very restrictive in what it includes.
Not to mention all semi-official lore (as written by Michael Kirkbride on the lore forums - MK wrote a lot of the official lore in the games and continues to post on the forums in an unofficial capacity) - and not to mention the newly mangled lore introduced by Elder Scrolls online.
Most "lore" in the Elder Scrolls lore isn't always to be taken as the complete truth - the fictional writers of the books approach their writing with a particular bias (remember the phrase "history is written by the victor".) What is "true" for one person, isn't necessary "true" for someone else. An official encyclopedia would either have to recognise this bias (for example "Imperials believe that....") or one particular version of events would have to be selected as the absolute truth. Which kind of ruins a lot of the fun of ES lore in the first place.