I've started playing Skyrim again a few days ago, this time on Expert difficulty and with some mods that changes various aspects of the game (for instance Skyrim Monster Mod, Warzones, Apocalypse spell mod and bringing two followers along thanks to UFO), but none - I think - that ought to have any immediate impact on how hard individual regular enemies hit.

While I'm generally a bit of a wimp with the difficulty setting in computer games , I figured Expert ought to be doable since it isn't even the highest. While I'm having great fun, it has - even with my two fairly competent followers along plus a strong conjured spirit cat - proven an unexpected challenge for my light armor wearing mage (around 50 in current armor class at level 13), and in just those 13 levels I've died many more times than I ever did in my previous games on normal difficulty.

Is it to be expected to occasionally die from a single hit from some enemies (humaniod bosses in forts, caves etc.)? I can't even imagine playing my current game without my two companions along to take some of the heat off me, and provide fire support (or in some cases, kill things that sneak up on us before I've barely had time to ready my spells ). And that's just regular exploring - trying to tackle the content of the Warzones mod on my lonesome must be a very scary proposition.

Expert level means the enemies take more hits from you to be killed, and thier hits do more damage. If you are tired of being one shotted when using mods made to make the game less survivable, then lower the difficulty.
I use just Skyrim Creatures Alive, One Giant Spider is going to kill most of my beginning characters on Novice. I live with this inconvenience because at later levels my characters do not feel overpowered.

I'm actually having more fun than I've ever had in Skyrim before. Now many fights - even with my two efficient followers - feel like a life and death challenge (and sometimes a slightly scary one too with Realistic Lighting installed, so I can't, without help, see a damn thing at night or in dungeons, other than spells lighting up around me), it's just those one hit kills that can be a bit frustrating .

Play on master level It's even more fun that way. I'm tired of being the hot stuffs. I want stronger opponents to fight. All I can suggest is to start learning how to dodge and use your shout at the 1 shot archers.

In expert difficulty, the damage you do to your enemy is reduced by 25% and the damage your enemy does to you is increased by 50%. In master, the damage you do to your enemy is reduced by 50% and the damage your enemy does to you is increased by 100%.

Increasing difficulty does some interesting unbalancing for a mage. A destruction mage's magicka expenditure increases by 50% in expert, and by 100% in master, as well as lengthening the battle. This is a very huge impact. However, a conjuration mage is not affected much, since this damage modifier does not affect followers and summons.

I imagine having the Apocalypse spell mod installed should help tip the favour back in my balance. But I'm still low level (and destruction and conjuration magic still only both around 35) so I'm running around with relatively limited firepower.

The game is not that balanced. Some combinations of skills are doomed to be weaker. As long as you have much fun, continue your way.
(Alteration magic can make you tougher, even you wearing robes.)

Bad enemy scaling maybe ? i sometimes run not often but it happens i run into much higher lvl enemies for my level it could be that.

For some people, dying a lot is fun and it's no big deal. For others, big part of the game is trying not to die and not dying. Extreme example of this is the Dead is Dead aka Hard Core people.

Actually some enemies are like that because you are supposed to have a higher minimum level to finish a quest or cave because the rewards are not for low levels etc. Morrowind had a lot of this.

But battles to the death tend to take longer, and in dungeons where you face 8-15 groups of falmer in pairs and it takes 3 minutes to deal with each pair, you will get sick of it haha. I prefer having one hit kills on because to me that IS realistic (for both player and enemy)...a mace to the face is gonna kill you