I have a suspicion that Goku got some sort of Zenkai Power or Power Up after his fight against Hit, as that's the only way it makes sense for SSJB Goku to be stronger than Hit in Episode 104.
I've also thought about the possibility that Goku got stronger in his little training with Whis, but if that was the case, Golden Freeza would be more powerful than Ultimate Gohan in the ToP, when it is clearly suggested that they are both on the same level.
Even in Episode 104, Goku SSJB is shown to be only slightly faster than Dyspo, which is suggested, that Goku SSJB (Vs Dyspo) was still on the level of Golden Frieza and Ultimate Gohan.
It seems like Goku got another power up in Episode 107 when he was resting. I think maybe just by seeing Jiren's terrifying power defeating Maji-Kayo, it will have given him some power boost and thus match SSJB Vegeta.