Ive played 5-6 hours and am not really challenged so far, is Very Hard a better choice?

go with very hard and don′t come back untill you have played longer but fallout 3 was not that hard so i don′t exspect this to be hard..
:D have not played it By the way

To me, hardcoe seems less about actually "hard" and more about being realistic/simulation. Nothing about it should really impact your ability to surmount some obstacle or situation. It's really just more about making you plan things out. Less godmoding and stomping all over people with your 90,000 minigun rounds that you happen to be able to carry in your pocket.

IMO, you should just keep dynamically adjusting the difficulty until you find what you are comfortable with.