I was playing on expert for quite some time. I only died a couple of times and it was a challenge.

Then I went and did the Companion mission (the one you have to do to be able to actually be a companion) ... and christ it was hard! I turned the difficulty back down and I still found it a little crazy! But I got through it.

Think i shall be keeping it on its default setting for a while now...