CatBoost: unbiased boosting with categorical features

Understand

This paper presents the key algorithmic techniques behind CatBoost, a new gradient boosting toolkit.

• Their combination leads to CatBoost outperforming other publicly available boosting implementations in terms of quality on a variety of datasets.
• Two critical algorithmic advances introduced in CatBoost are the implementation of ordered boosting, a permutation-driven alternative to the classic algorithm, and an innovative algorithm for processing categorical features.
• Both techniques were created to fight a prediction shift caused by a special kind of target leakage present in all currently existing implementations of gradient boosting algorithms.