Okay. Some enemies upon death will drop their weapons and/or shields.. Also, if someone misses when shooting then the arrow ends up somewhere on the ground or in water...

I do know that leaving dropped items like this can clutter the save and I do know ways of cleaning up weapons, shields, and arrows... But I do have an issue with Bound Arrows since they cannot be picked up..

I was wondering, do ALL arrows eventually disappear if you miss or will they stay around forever?

I'm not sure. I always make sure to store weapons and shields in bodies so that they would clean up. But I never paid any attention to arrows.

For arrows it's an maximum numbers of used arrows allowed in the game totally, when the number goes to high the oldest one are deleted.
Its also an maximum number of dead bodies, during the huge civil war battles you can see them going away.

Dropped weapon and shields should go away on cell/ dungeon reset. In some dungeons dead bodies goes away after some days but the weapons stays but i'm sure they goes away after 30 days.
However in Oblivion, if you pick up something and put in inventory and later drop it the item will stay. This is because the item belong to you and not the bandit.
Not sure if this is true in Skyrim but I think so, so if you pick up junk store it in an body or container, then it will disappear on reset.

Dropped weapons and shields do not.. I normally pick them up and put them on a corpse or container so it does though.

But I am curious about arrows because there are some bound arrows in this one dungeon that never was cleaned up... And they been there for well over a month by now.