foot, i only use horse if I'm overweighted with stuff
same here, foot for picking ingrediants, horse for fast travelling, mountain climbing or over wieght
for example.. I just completed a quest- went through a fort killed lots of dudes, got all the armour and weapons... walked outside... placed all the items in a crate.... fast traveled to same location to get my horse there.... picked it all up - got on my horse and fast travelled back.... and BINGO
near enough 1000 in wieght

useful cash and barter build up =]
EDIT:- afterall, alot of wild west movies show a people having their "SADDLE BAGS" filled with their travel gear or other gubbins they find... horses are strong .... and skyrim horses are nordic horses... so they have to be hardy and strong

hell mine thinks he can take on dragons! =p