Learning with Non-Convex Truncated Losses by SGD

Understand

Learning with a {\it convex loss} function has been a dominating paradigm for many years.

• It remains an interesting question how non-convex loss functions help improve the generalization of learning with broad applicability.
• In this paper, we study a family of objective functions formed by truncating traditional loss functions, which is applicable to both shallow learning and deep learning.
• Truncating loss functions has potential to be less vulnerable and more robust to large noise in observations that could be adversarial.