I know what you mean.

The thing with doors, drawbridges and the such are, that there is always a lever, handle or chain near by to open / lower it. So you will get used to look out for those.
The lever you are looking for in the Embershard Mine is on the floor. When you have the drawbridge to your left (and the stairs where the first bandits came at you to the right), you just head straight forward. That will lead you to a small room from where you again watch directly back to the drawbridge. There on the floor is the lever you are looking for.