everything's a blast now with maxed conjuration even if I didn't need it... the master level spells that revive high level humanoids and the daedra are a blast. Take the aggro off me as well. I can see how they would be annoying though if your build is not ready to fight them.
I don't play a caster. I play a 1h melee guy with a shield. The only "offensive range" that I use, is the shouts.
That said... it's not so much about fighting them, as it is that I'm purposely not progressing the main quest... I'm doing most of the quests in cities (townsfolk), jarl / thane quests, and faction quests, first. In my particular game (since the dragons are random)... they were just appearing WAY too often, and it was getting more annoying than fun, while I was in the middle of some other quest. I have way more souls absorbed than shouts. So turning off this distraction, for the time being, is what I opted to do. When I runout of souls, I'll probably turn them back on for a while.