I thought it was like in Oblivion, when you click LMB to swing a weapon. But when I press it in Skyrim, nothing happens. When I press and hold LMB, my character makes power attack draining my stamina. How to make a simple attack/swing?

just click.... no holding.

sounds like you might have a mouse problem

My mouse is fine. I can swing a weapon sometimes, but cannot say how I do this. When I just click a button, nothing happens.

Check your mouse. Make sure its fine.
Just click don't hold.