FWI I do agree that Zoro would defeat Act 1 G4 Luffy (50/50 chance even Rooftop Luffy as well)
I think your viewing Kaido's durability the wrong way. His scales are what makes him durable, but different methods can allow someone to hurt him. It wasn't because of Luffy's physical power that he couldn't hurt him, he just couldn't get through his scales. Even the likes of Kinemon can hurt Kaido a bit with advanced CoA. The only way Kidd could hurt him was by crushing his insides.