Frustrated with Replicating Claims of a Shared Model? A Solution

Understand

Machine Learning (ML) and Deep Learning (DL) innovations are being introduced at such a rapid pace that model owners and evaluators are hard-pressed analyzing and studying them.

• This is exacerbated by the complicated procedures for evaluation.
• The lack of standard systems and efficient techniques for specifying and provisioning ML/DL evaluation is the main cause of this "pain point".
• This work discusses common pitfalls for replicating DL model evaluation, and shows that these subtle pitfalls can affect both accuracy and performance.