Nobody knows. It was there before Whiterun, before Jorrvaskr, before the Nords who first settled in the area arrived to find it. The mer already living nearby didn't know either as apparently nobody could remember a time when it wasn't already there. IIRC they said they stayed away from it because it was cursed.
There is some speculation that the giant hawk figure that dominates it is related to Kyne, which is interesting to me because there is also a Temple of Kynareth in Whiterun (the only one in Skyrim, I believe) and of course the Gildergreen is connected with her cult as well. In Nord tradition she was the war-wife and widow of Shor/Lorkhan and her connection with the world of men and the Mundus is strong, not only for that reason but because it was she who so pitied men that she wanted them to have the Thu'um in order to prevail against Alduin. Some legends say she was also the first to agree to Lorkhan's plan to create a mortal plane and provided him with a space in the Void in which to do so.
I just get the feeling that the area of Whiterun was sacred or special way, WAY back for some reason connected to Kyne and that makes me wonder what went down there, possibly between men and mer, especially since the mer who remained or settled in the area thought the Skyforge - with its gigantic Kyne totem - was cursed even though they didn't know anything else about it or who put it there.