So I want to make a dungeon that is really dark, not pitch black, but pretty dark. When I do it in the CK it looks dark to me, but in game it does not come off as dark enough.

http://imgur.com/a/cd1g8#Rl49s

the first two photos are what I'm seeing in the CK, and the bottom two are what I am seeing in game.

As far as light settings go, I have the fog set to show up at 0 and its color is set to straight black. Any help is appreciated!

You need to set the directional ambient lighting to black. By default its white. Its the last tab on the cell edit window for interiors. Remember to select your cell first so you don't accidentally change another cells lighting.