Multisource AI Scorecard Table for System Evaluation

Understand

The paper describes a Multisource AI Scorecard Table (MAST) that provides the developer and user of an artificial intelligence (AI)/machine learning (ML) system with a standard checklist focused on the principles of good analysis adopted by the intelligence community (IC) to help promote the development of more understandable systems and engender trust in AI outputs.

• Such a scorecard enables a transparent, consistent, and meaningful understanding of AI tools applied for commercial and government use.
• A standard is built on compliance and agreement through policy, which requires buy-in from the stakeholders.
• While consistency for testing might only exist across a standard data set, the community requires discussion on verification and validation approaches which can lead to interpretability, explainability, and proper use.