Smarter than Altmer though.
The difference being in the fact that Altmer were once proud and intelligent race like any other race we see today, while giants are not much smarter than from that time.?
I was suprised at how Giants ended up turning out in game. Bethesed in their early marketing preview of Skyrim, mentions that they are gentle creatures that only turn violent when you mess with their mammoths.
So what's the difference?

They are alright dudes, as long as you stay away. They don't even attack on sight, and if you approach too close they give you a friendly warning not to [censored] around them or their mammoths or they'll kill you. Unlike bears who are depicted as creatures who will usually run if in danger and ignore you if you're far enugh in game's loading screen, while I have never ever seen a bear do that, they charge down on me as soon as they see me from the top of the hill.