From what I've been told in game, it's because Talos was a man, not a God. Your classed a heretic for worshipping a man and not a god. This is what I was told by a group of travellers (non-nords) anyway, we disagreed on the matter and I had to set things straight.
This is a spoiler and I know this is the right reason. The dominion (elves) do not think talos is worthy of religious worship. Some speculation is because it's that he is human and not elf, they don't want a human worshipped as a god when he is mortal. So as part of the peace treaty for the empire and dominion in the war, they wanted the empire to outlaw the worship of talos. This was another reason as to why Skyrim was very upset about the empire.
Another couple of great answers, thanks guys

I'm a lore junky, and now the game -actually- makes sense to me lol