I think it's more that Gohan doesn't know how strong he is either. If his dormant ability is greater than what Goku originally said, (strong enough that no one can beat him) initial Boo should not stand a chance against him. It seems that his true ability is somewhere in the neighborhood of what he got from crusty Kaioshin.
If you assume that Gohan does know his true power, perhaps we can attribute his lack of confidence to his just general inability to access his power. There are possible other readings of that line. You could, perhaps, say that his statement about "maybe I can do something to him" applies to his current self, and that his victory would be assured, if he could only access his true power.