It depends on where else you might want to use those points. If you can afford it, and it feels right for your character, then do it. But have you thought about dual-wielding instead? Granted, there's no defense, but you can do more damage when they get that close, especially if you already have some skill in one-handed.
What level is your character, and what have you perked so far?