I don't believe it's possible to cast spells on horseback, and bound weapons require casting a spell. Since you automatically sheathe your weapons when you mount, also, you can't summon your weapon and then get on a horse.
So no. While I haven't actually tried it yet, I think it will be impossible to use bound weapons while riding a horse.