So an order dedicated to Talos and his heirs says that they were all Dragonborn makes it true? As much propaganda that's come out of Septim's Empire I'm inclined to take what a biased author says as fact. The book makes the assumption that "if they can X they must be Dragonborn", trying to reason previous holes in the Imperial Ideology that comes off more so as propaganda and politicing rather than divine right and blessing.
Makes sense though. Only Martin could wear the Amulet of Kings, the Amulet was not bound to blood, we can see this clearly because the rulers during the first second or third era empire had no blood relations to each other. Martin could wear the amulet not because he was related to the first wearer of the Amulet of Kings, Alessia, but because he was dragon blood. Whether or not he was actually Dragon born or not, I don't know. It is entirely possible that all descendants of the Emperor Tiber Septim were Dragon born, or perhaps they became so when crowned.
More relevant to the question asked in the first post however, learning the voice takes many, many years. It's a long hard process and even then some don't master it. It's easier for the Dragon born if they take in souls but if they don't then they still have to learn the hard way. Meaning quite simply that all of the Emperors from Tiber Septim and up had to learn it the hard way, because there were no dragons to kill for them, and that would have taken them years and years and years, which quite frankly they needed to spend on more important things like governing an Empire.