1- The crime system.
In Oblivion, if someone even saw you tresspassing for one second, every guard in the province would immediatly know where you were and hunt you down relentlessly. While the Skyrim system is a lot easier, which is usually a bad thing, it just feels so much more complex than the previous system. I love how, if a robbery goes bad, I can kill the owners and get away with the crime (like in real life.) I also like how bounty hunters would track you down.
2- The perk system
While I do hate that they removed attributes, I do enjoy the perk overhaul. There is just so much more customization than the Oblivion perk system. It helps let me define my character more, and even if I get all skills to 100, I still feel like I am a certain class because I would have put more perks into certain skills than others
3- The wilderness
For one, it was so much more detailed. In Oblivion, I felt like they had long stretches of generic woods, now in Skyrim, the land feels so much more detailed, not only the general terrain, but the towns as well. It felt like they took the solitary roadside inns of the previous games and built an entire small town. It really made Skyrim feel like people were living outside of the major cities. I also enjoyed the orc strongholds. It made orcs have more flavor and really made them feel like outcasts in this world. Also, giants were a really nice feature for almost the same reason. They made Skyrim feel more alive. This was partly because of the fact that they felt like they actually belonged in the world, rather than just being a generic enemy to attack you.

In Skyrim there's only Nigthgate Inn as an inn not in a town or a village.