As in preceding Elder Scrolls games, I've found I've only used a few http://www.uesp.net/wiki/Skyrim:Illusion spell effects, and I'm wondering how they would be useful.

In Skyrim, mostly I've only used the http://www.uesp.net/wiki/Skyrim:Muffle_(effect) effect to enhance stealth; I've not built up a character's illusion skill enough to use http://www.uesp.net/wiki/Skyrim:Invisibility, which would obviously be useful.

http://www.uesp.net/wiki/Skyrim:Fear and http://www.uesp.net/wiki/Skyrim:Pacify seem like they would be useful, though in any situation I might use them, I've tended to use a destruction spell or other attack instead. This is complicated by the limited range of levels on which a given variation of the spell is useful. In past games, I've seen spells with descriptions similar to http://www.uesp.net/wiki/Skyrim:Frenzy to be used in exploits to force a non-hostile character to attack, but I've never had a situation in Skyrim where I've wanted to do that. I have a really hard time believing that http://www.uesp.net/wiki/Skyrim:Rally would be useful.

Does anyone have tips for use of these spells?

• The usefulness of Invisibility is self-evident.
  
• Dual-casting the spells makes them stronger, and taking the relevant Illusion perks makes them even stronger. Once you've taken all the perks, dual-casting the lowest-level spell will affect the highest-level enemy (except dragons: Illusion spells cannot be used on them).
  
• Sneak up near a bunch of enemies (bandits, for example). Cast Frenzy into their midst. Wait for them to mostly kill each other off, then finish off the survivors.
  
• I tend to Pacify animals, especially bears & sabre cats, when I'm en route to an objective and have no time for fighting random wildlife.
  
• Cast Fear (or its variants) on an enemy to send them fleeing. You can then chase them down and finish them off without them fighting back. Useful for mages against warriors who can one- or two-hit them.
  
• I find Rally useful during the Civil War quests, for example, where you're working with a group. I also use it a lot when dragon-hunting with the Blades recruits--they're pretty tough, but a little extra health and stamina never goes amiss. I also often have a follower, and two summons and, once you take the relevant perk, your spells will affect summons like Dremora Lords, so it's useful for when they're fighting a tough enemy.

If you are going for a stealth mage, Frenzy and Calm are the best effects there are. With Frenzy, you can drop the number of enemies you need to kill down to one.
Calm can allow you to walk right by an enemy without conflict. This is great for thieves.

I always play as a sorceress (mage) and the only reason I take the perks in Illusion is for the Aspect of Terror perk, for it has a glitch that makes fire spells more powerful and as fire is the main element I use, this perk is good for me.