It's nice to see the Borders not just being mountains or random invisible Walls, but actually an acknowledgement that it is a border I.e. The wall and gateway/arch east of riften. It does look DLC-ish as they did this with fallout; left open Borders or doorways as possible areas for DLC's / expansions

From what people say, that is the border into Morrowind. Apparently you can see the White-Gold Tower and Red Mountain.

Just from the opening I can see a tower way off in the distance, but obviously it's just a random tower

Theres one south of helgen too, can't see anything interesting but maybe you can on pc, but it you follow the map from that point theres a ravine in the mountains and at the end the mountain has an odd shape to it, like a chunk is missing, kinda reminds me of some of the burrows but like the stone work has been stripped away (or not put in yet...).

The arch to the east of Riften and the arch south of Helgen both mark the borders to Morrowind and Cyrodiil, respectively. If you use noclip and disable border regions in the .ini, you can see that the paths do actually lead down into the provinces - although, at a certain point the ground stops, and all that is there is LOD.