I thought people would agree that courier is way too strong for one man. I play the game with hardest difficulty and have tried to search some difficulty-raising mods to keep the immersion. I don't mean to brag here, just point out how way out of place the courier's capabilities are.
Logically speaking, you're totally right – but, like most narrative video games, this character follows "action movie logic." It makes no sense that a washed-up old cop can take on a building of terrorists alone in Die Hard, or that a ragtag group of misfit hackers can take on a bunch of superpowered computer programs in The Matrix, or that a single martial artist can take on an army of ninjas in just about every martial arts movie ever made ... but that's the sort of genre cliche now. For additional examples in gaming of single heroes against impossible odds, see Splinter Cell, Mass Effect, and Super Mario Brothers.
All of that said, it definitely feels more challenging and a little more realistic when you leave Endurance at 1, take the Small Frame (and either Fast Shot or Kamikaze) trait, and only wear clothing instead of armor. I'm still getting crippled regularly in fights at level 24!