I want to do a heavy armor sword and board who is also a master conjurer and smith. So, heavy armor, one hand, block, conjuration, smith.

Is that doable completely perked by level 50? I wouldn't mind archery or destruction too, but that would be a bonus. I suppose my summons can take care of ranged.

Also, how would you invest attribute points with such a build? Health, Stamina, and Magic even?

Pretty much anything is viable in Skyrim. If any character build involves Smithing or Enchanting then it's going to be a snorefest past mid game.

As for Health, Magicka and Stamina.. well there isn't much point increasing Stamina. You can perform actions that require Stamina when you only have one point of it.

From what I see that's basically a battlemage, which is perfectly viable.

This

I will be playing on master. If I level smithing to 100, but don't perk it, will it still break the game in your opinion (or anyone else's)?

If you plan to power level smithing (Early On) than you may need to take some perks in it, or you will probably die A LOT

I won't power level it, but once my main skills reach or near 100, I will likely raise it. Anyway, without perking smithing, will it break the game at 100 in the mid game? I have it perked up with one character and he is almost too much of a tank.

Sure it's viable. My character is a heavy armor mace/shield heavily spec'd in Restoration, Alteration, and Alchemy. I'm only going into Smithing as far as Arcane Blacksmith.

As for your formula, I generally go 1 point in magic for every 4-5 I throw into health/stam. Now that I'm well over 200 stamina (+steed stone) I'm mainly dumping into health.

I played on Master on my stealth archer, but it gets tedious and frustrating as sword n' board. Expert seems to be the sweet spot for this play style, in my opinion.