Annoying when you battle a guy who quits when his HP reach dangerous low and you
don't get the kill. Would love see at penalty for this.

Based on what? How many people are you going to penalize for false positives? Not to mention we have no idea what penalties or ttk is like to make this an even decent worry.

you simply dont allow instant log outs and problem solved

It seems to be standard practice to keep characters in the game world for a bit on disconnect. The death you earn when you log out and the death you would have gotten if you stayed logged in don't have to be dramatically different.

Not sure if that is even possible to control, except while using the logout option.

A timed penalty for quitting in the midst of battle in PvP would also be a penalty sufficient. I could live with that:)