It depends on the Dagger being used. I can't remember which ones do not get the added speed, but equip two Elven Dagers and hold on.
The speed bonus is applied pending the left hand. (For ex) Put a dagger in your left and a mace in your right.
Oh! Where's my manners? Welcome to the forums and have a complementary fishy stick.
http://images.uesp.net/c/c4/Fishystick.jpg