Is there a way to mount a troll skull on the wall of your house? Or anyway to decapitate and stuff a bear head or some other beast for decoration?

As awesome as that would be, nope. I have a feeling there will eventually be a mod for it.

No. Some houses come with that stuff though (the house in Riften has a bear head on the wall for instance), once you ask the steward to decorate.