So I went and got my soul gem back, but all it did was add a black soul gem to my misc. is that it? So I have the rest of my soul back? Or do I do something with the black soul gem?

If I remember right, you have to activate it. For some, it made you stronger. For others it made it so that your health, magicka and stamina did not drain nearly as fast.

So did I activate it when I took it from the chest or do I do something to the black soul gem?

I am not sure. I just know from the testers that you have to activate it. I had a bug/glitch with that part of the quest