Grader variability and the importance of reference standards for evaluating machine learning models for diabetic retinopathy

Understand

Diabetic retinopathy (DR) and diabetic macular edema are common complications of diabetes which can lead to vision loss.

• The grading of DR is a fairly complex process that requires the detection of fine features such as microaneurysms, intraretinal hemorrhages, and intraretinal microvascular abnormalities.
• Because of this, there can be a fair amount of grader variability.
• There are different methods of obtaining the reference standard and resolving disagreements between graders, and while it is usually accepted that adjudication until full consensus will yield the best reference standard, the difference between various methods of resolving disagreements has not been examined extensively.