I agree that if you have a horse with you that your loot limit should increase. But that brings in a lot of other problems. Suppose your horse gets lost? Drowns trying to follow you somewhere. Dies. Etc.
Honestly I'd like to have a mule/cart companion when I go into dungeons.
For Oblivion I preferred Shadowmere, as it was not only one of the best horses in the game, but also essential, so at worst would get knocked out. There were also mods to use mounts as storage, as well as some to command your horse (like call it to you, tell it to wait, or follow etc.).
I fully expect such mods to end up in Skyrim. So even if it's not there in the vanilla game, as long as you're on PC, you'll be good to go.