Objective evaluation metrics for automatic classification of EEG events

Understand

The evaluation of machine learning algorithms in biomedical fields for applications involving sequential data lacks standardization.

• Common quantitative scalar evaluation metrics such as sensitivity and specificity can often be misleading depending on the requirements of the application.
• Evaluation metrics must ultimately reflect the needs of users yet be sufficiently sensitive to guide algorithm development.
• Feedback from critical care clinicians who use automated event detection software in clinical applications has been overwhelmingly emphatic that a low false alarm rate, typically measured in units of the number of errors per 24 hours, is the single most important criterion for user acceptance.