Actually, Goku defeated him with Kaioken in a direct fight and he was forced to use Great Ape.
And he was embarrassed to do that. I'm pretty sure that he wouldn't forget that.
Yeah, Vegeta forgetting Goku's greatest technique when he'd been plotting to go back to Earth and annihilate it for most of Namek is preposterous. I literally don't know where that argument even came from when he still remembers the Genkidama all the way in the Boo arc.