In my earlier days of Skyrim, I had played an offensive mage and had a lot of trouble dealing with dragon priests and other high-level mages. I felt pretty good about myself when my character had reached the point of bursting down Krosis in a few seconds without running out of magic.
I ran into Krosis early. He was one of the only people to ever kill my mage. He dominated my flame atronachs something silly. If I had waited until I could dual cast Dremora Lords, I imagine he'd be a pushover.
My fully powered mage struggled with only two opponents: Orchendor and the final showdown with the Gauldurs in Reachwater. Sigdis' clones kept FUS'n me all over the place.... and while I was level 40+ on master, I only had 100 health. Limiting my health was the only way to make the game remotely challenging for a conjurer, but it made that fight exceptionally hard. If I was lucky, I could take two hits before dying.