Well, bumping the enemies sneak skill up to 100 would make it easier for them to detect a sneaking player.
As for invisibility....
The cloak spell that dispels sounds interesting.
I wonder....
What if you give an NPC a 'detect life' spell. He probably won't cast it on his own, but if you made him cast it in an 'onhit' event like:
Event OnHit(ObjectReference akAggressor, Form akSource, Projectile akProjectile, bool abPowerAttack, bool abSneakAttack, \ bool abBashAttack, bool abHitBlocked) Debug.Trace("We were hit by " + akAggressor) EquipSpell(DetectLife,2) DoCombatSpellApply(DetectLife,self)EndEvent
Then invisibility+sneak up to the enemy, attack, and see if after casting the detect life spell he comes after the player even when they are invisible