I'd probably say Aoshi, seeing as how he was placed against far more impressive opponents (eg. Kenshin using his best ougi to defeat him and Gein having far better feats than the rest of Enishi's allies), plus Aoshi seemed to last a little longer against Shishio than Saito did.