Depends on what you define as "effectively". And what you're looking to get out of them.
I tend towards jack-of-many-trades hybrids in games that allow it. My first character in Skyrim was a mix of Warrior & Thief skills, plus Restoration. Kind of a "Ranger-ish" build. I put perks into the skills I was using, as they felt appropriate and if they seemed like they'd be useful/interesting. (And, of course, as the skills got high enough to unlock the perks - with a wide set of skills, none of them leveled terribly fast. Only got one skill to 100, and that was Smithing at level 48.) Playing on Adept, I felt reasonably challenged (yeah, that's subjective

) until in the mid-to-high 40's. Certainly didn't feel overpowered.
My perks? I had them scattered among ten skills, and the only one that was run up to the top level was Smithing (since you had to, to try out Dragon armor). The others generally had around 6 perks in them.
hmm, I'm sure I have it around here somewhere..... rummage, rummage.... ah, there it is!
Alchemy 6 Block 8 Lt Armor 6 1-Hand 6
Speech 4 Resto 3 Lockpick 5 Sneak 1
Smith 6
Was this the most powerful character I could make? No. But that wasn't what I was thinking as I played or as I picked perks. I just went, when I got each level, and looked through the skill chart thinking "ok, where can I put this that seems interesting & useful". That's one of the main reasons I didn't go higher in the 1-Hand skill - the upper level stuff seemed boring and/or that I wouldn't use it much. Or Light Armor - I didn't take the "bonus for matched set of armor" perk, because I never wore matched armor - I mixed & matched between all the random loot I found. Heck, in the 30's or 40's I was still wearing Hide bracers because they had a great enchantment.

Might not be the way other people might want to play, but it worked for me. And perhaps a bit off topic, but I just wanted to illustrate that maxing out a small number of skill trees isn't the only way to perk.
