If you are doing conjuration, yea. Illusion spells are really powerful and even just getting quiet casting, muffle and invisibility make your character a boss. The only spells that are affected by caps are fear/charm/frenzy all of which are OP. I don't know the technicalities of how the level caps work, but you're playing most of the game under level 50 anyway. It's not an MMO.
I joined the college just to get access to purchasing the invisibility spell. I reread your post and I'm not sure if you're asking whether or not to "master" illusion. If that's the case, then I would say that going to expert level so that you can get invisibility is as much as you really need as a thief.