Stop trying to punch people. Shoot them. It becomes a whole lot less necessary.
Having no weapon skills seems to be hurting your ability to fight.
Not exactly. I have high sneak, and am taking all the perks for crits, silent running, those type things. I do use guns, my point is even though I've invested nothing in guns, endurance attribute etc, having high sneak mitigates not investing in anything else (so far of course).
Spoiler I mean, I'm only just past Nipton (sp?) where you run into the Legion first time.
So I might be speaking too early.
My second run through I'm going to try a stealth melee for fun. I don't see me playing through this game on any type build unless it has high sneak though.
Often I can shoot a baddie in the face and just sit there in the shadows and they still can't find me, which makes me feel cool, like an assassin.
EDIT: I do have a silenced .22 and the silencer mod on my Varmint rifle, I like that.