Atem King of Niger said:
The ability to adjust to gravity isn't strictly numerical as we see in the series. But Goku does state that the Vegeta zenkai helped him master the 20fold gravity better and after abusing zenkais he could even move in 100x gravity. The zenkais clearly raised his body's ability to adapt to harsher environments, and the reason he concluded the 10-fold Kaioken shouldn't be impossible is because he mastered the 100-fold gravity. There's a clear correlation between Goku's zenkai increases and his Kaioken endurance, especially when the 20-fold Kaioken still did less damage to him than the 4-fold one did on Earth despite Goku only getting a week's worth of training if we don't count the zenkais.
Goku also states that if he doesn't toughen himself up from the basics, he won't be able to withstand many folds Kaioken:
Chapter: 251 (DBZ 57), 10.5
Goku: “If I don’t thoroughly toughen myself up from the basics…I won’t be able to withstand many-fold Kaio-Kens…!”
King Kai states here the Saiyans used the 10x gravity on their planet to toughen up their bodies:
Chapter: 211 (DBZ 17), P7.4
Kaio: “Let me tell you something…The planet where the Saiyans lived had gravity at least as great as this [10 times Earth's gravity]…
They used it to strengthen their bodies. [ ] Do you begin to see where they get their strength? And that's not mentioning their inborn fighting instincts. You can’t even imagine how deadly they are.”
This creates a correlation, that Goku needs to toughen up his body with the gravity training to withstand many folds, suggesting heavily that power
alone isn't the only factor on the Kaioken mastery.
Also, the 4-fold Kaioken didn't actually damage Goku that much if i remember correctly. Yeah, it worn him out, but he was still able to run away from Oozaru Vegeta until the latter caught him off guard when he was charging the Spirit Bomb.