I dare to say the 21st Budokai had a couple hundred people at best, but each tournament had more people than the last.
I dare to say there's easily 10,000 people in here. The current Budokai arena probably has the capacity of a small football stadium. The 25th and 28th probably had a similar attendance, same for the one in the Bojack movie. The Ano-Yo Ichi Budokai seems to be at least as large, if not bigger, not counting the fact the stands surround the arena.
The smaller ones you can count yourself. Baba's tournament and the U6 one only had Goku's friends. Cell Games was just the Z Fighters. ToP was just the gods watching.