I often have hard times traversing the landscape. A lot of the inclined areas like the sides of some hills and mountains are hard to go up, a lot of the rocks lying around makes it hard to walk to some places, I get stuck in some places, and at times I have missed a turn or so and end up having to backtrack my entire way or something.

The hardest quests are when I cannot fast travel to the location, and the only reason I fast-travel is cause I cannot physically make my way there.

I had this same problem when I played Morrowind and Oblivion, but not as much cause Skyrim is a lot more mountainous and hilly than Morrowind or Cyrodiil.

It seems like the game is programmed specifically for the player to remain on the roads at all times. Maybe it is?

Not at all. You can go wherever you want and do whatever you want.

I understand somewhat when it comes trying to scale mountains, but I have oftentimes climbed them without going up any road or perhaps the "normal" way of going up the mountain.

And I have no trouble zig-zagging down the vertical face of a mountain...

The turning-while-you-jump thing, to scale ledges etc., takes some getting used to.
You might try buying a horse for some trekking, as they can scale mountains better than we can.

If there's a location atop a mountain that you're trying to get to there's always a path somewhere. You just have to keep going around the mountain to find it. If the side of the mountain is too steep to climb than that's not the correct way to go.

3rd person view is easier to navigate around for some than 1st person (unless one is a diehard 1st person view only )

Find the Clairvoyance Spell. Most of the time, it will show you the way to the path you need to take.

While the game does funnel you to the roads, it does allow you travel almost anywhere. But, up steep slopes. Then again, if you were hiking, the same difficulties would be there when you faced a mountain. You have to plan your trip up a slope, and sometimes starting way off to the left or right of where you are is the way to go to get above where you are now.