Toward a Better Understanding of Leaderboard

Understand

The leaderboard in machine learning competitions is a tool to show the performance of various participants and to compare them.

• However, the leaderboard quickly becomes no longer accurate, due to hack or overfitting.
• This article gives two pieces of advice to prevent easy hack or overfitting.
• By following these advice, we reach the conclusion that something like the Ladder leaderboard introduced in [blum2015ladder] is inevitable.