I want to create a perk that reduces damage from spells cast by certain actor type (undead and daedra) by half.

For physical damage, this is not a problem - the Dragonhide spell perk provides a good template, and under "Attacker" tab I just add various keywords for actor types whose damage I want reduced.

For magical damage however, I am having a problem. I choose "Mod Incoming spell Magnitude" and under "Spell" tab add the same keyword conditions as I did for the physical damage part (hasKeyword ActorTypeUndead, hasKeyword ActorTypeDaedra, etc). I tried running these conditions on Target and on Subject, but neither produced any effect. If I take the conditions out, the entry point works (so this seems to be the correct entry point), I just need to restrict the damage reduction to spells cast by certain actor types.

I believe the source of spell is not the casting actor but the spell itself, this is why it works if you remove the conditions. However I have no idea how you can link the spell back to the caster only by conditions....maybe possible if you wish to search deeper.
You might want to try adding a perk to undead actors if nothing else works that will reduce the power of what they cast but note that you have to change all those npc forms as addPerk doesn't work on NPC via scripting.

Well that's upsetting. Thanks though.

Are you sure the "mod incoming damage" entry is limited to physical damage?

In fact I am... I was kind of hoping it wasn't, but it's not the case (which, BTW, makes the vanilla Dragonhide spell a bit less exciting than it seems).

The testing I did was to spawn a hostile flame atronach next to a naked test character with 0 resistances to anything and counting the firebolts it took to kill me.

With the "mod incoming damage" entry point it took 5 firebolts whether I had the perk or not.
With the "mod incoming spell magnitude" entry point and WITH conditions (subject or target hasKeyword ActorTypeDaerdra == 1) it still took 5 firebolts. And yes, the atronach has ActorTypeDaedra keyword (checked with the console).
With the "mod incoming spell magnitude" and NO conditions, it took 10 firebolts to kill me (as expected).

So, the entry point is right, it's just it doesn't seem to give a damn about who the spell is coming from.

I think there is a sort of workaround, however its not 100%.

Have a look at the Magic Resistance perk, that grants an ability called PerkMagicResistance, which blocks 10% of a spells damage.

If you duplicate this and put conditions on the spell that check the target for keywords ActorTypeDaedra and ActorTypeUndead.

When these conditions are met, the ability activates. The downside is if you are in combat with an undead actor and another creature type, the ability would still activate.

Hmm, this works sort of.

If you apply the following condition to the Entry Point (Mod Incoming Spell Magnitude), on the Perk Owner tab

HasKeyword ActorTypeDaedra == 1 Run On Combat Target

The damage reduction only applies so long as it is your current combat target. You have to attack the target in order for the condition to activate. If you do nothing, the damage reduction doesn't apply.

You may be able to create a resistance ability that has Daedra (and whatever else) in the conditions.

That was what i was suggesting in my original post, however there was that caveat that if you had a daedra caster and another type outside of the conditions, then potentially you could receive the damage reduction from both targets.