Yeah either Franky or Brook. Both show up too late in the game, and Usopp might actually learn to be a better shipwright in the future to make up for the loss of the Going Merry.
Berserk and Vagabond go further downhill than Vinland Saga does though even if Vinland Saga wasn't as good as them. I can't really comment on the other two of the "Big Five Seinen" as I haven't invested enough of my time in them to really say (Blade Of The Immortal, Kingdom).
We all know that Naruto and Bleach go downhill after Part I or the Shinigami and Soul Society Arcs and especially after Pain & Aizen, and we all know One Piece isn't the same after the Whitebeard War Saga, but what are some other instances of quality fluctuation?
For me, I think East Blue One...