I do not think they are strong at all. Just as strong as real life heads of state or noble families. They rule through respect for tradition/the status quo/the current legal order of the world and patronage.
Yeah, I doubt Oda's going to add a twist of them being top tier fighters. They probably just control the world through aristocratic blood ties, much like the Tenryubito.