They're totally different.
Here is an excellent presentation on id Tech 5, enough to gain a feel for their outstanding features, which are "virtual texturing" and scheduling. Doom 4 and Rage are its launch games. http://s09.idav.ucdavis.edu/talks/05-JP_id_Tech_5_Challenges.pdf
Historically, id Tech engines have been FPS engines. The id Tech 5 scheduling model, which requires a job per actor, suggests that it will work better with small, predictable numbers of actors, rather than the much larger number of an open-world RPG.
Gamebryo has lots of world-building tools, which Bethesda has historically exploited to construct large, detailed worlds. They also have a longstanding relationship with the Gamebryo developers. Other engines have better lighting and better rendering, but Gamebryo is good at doing important optimizations like hidden surface elimination quickly. Thus you can have a lot of objects in the world and in the frame, without the game eating itself.
id Tech has stated they will keep to their policy of releasing engines as open source, as they've done with the engines through id Tech 3 (and id Tech 4, maybe this year).