Not as detrimental, but any additional disadvantage to the disadvantage he already makes the situation insurmountable without resorting to the Kaio-ken x3.
Goku can use the standard Kaio-ken comfortably in short bursts, he cannot maintain it, which is a problem. It's similar to what happened between Super Saiyan God Vegeta and Merged Zamasu. Vegeta can employ Blue's full power, but only in bursts, which puts him at a disadvantage against someone who can maintain the equivalent to Blue's full power 100% of the time. If the best Goku can hope for is a stalemate, and he can't even hope for that, he's doomed, that's plainly evident.