Old Rayleigh was able to easily keep up with Kizaru albeit at a disadvantage. Since Prime Garp would most likely beat Prime Rayleigh by sheer portrayal of being Roger's rival, the same most likely holds true in his old state.
How does this play out. Even if Garp is the powerhouse here, perhaps...