I liked Markarths scenery as well, at least on the outside it's quite a view.. I don't feel the same about the inside but I don't hate the place.
The Palace of the Kings in Windhelm is a cool place to be in, and though I have only gone inside the Temple of Talos twice now, it's a cool place too. Will probably visit it more often.
I like Solitude partly because if the capital of the Province.. when I first started playing the game I didn't know which city was the capital, and it took me awhile to find out. I also like some of the quests that take place in and around the city.
There's also Jorrvaskr and Dragonsreach in Whiterun but nothing else is of interest to me.
In the end, I suppose I don't really have a favorite hold capital though.. but these are at least my favored. I tend to like the big buildings and the cozy feelings I get inside them.. but I wish there was a place to hang out with NPCs who had decent conversation

Well, I suppose Markarths scenery is good on the inside.. but I just remember how I felt about looking at the city from the outside with mountains in the background. I tend not to pay much attention to the inside but it's still a cool enough place.