They aren't loyal to them. More like they won't get involved but will let thalmor move troops through Elsweyr and possibly spare supplies for them
And the thalmor don't antagonize them because they don't want them attacking the dominion.
Typically the Khajiit and Bosmer haven't got along well. And having another force attacking your forces is generally a stupid thing to provoke if it's completely unnecessary.
If the thalmor goals come to fruition though, you'll see elsweyr come to an end shortly after they have no more use for them.