FFCV: Accelerating Training by Removing Data Bottlenecks

Understand

We present FFCV, a library for easy and fast machine learning model training.

• FFCV speeds up model training by eliminating (often subtle) data bottlenecks from the training process.
• In particular, we combine techniques such as an efficient file storage format, caching, data pre-loading, asynchronous data transfer, and just-in-time compilation to (a) make data loading and transfer significantly more efficient, ensuring that GPUs can reach full utilization; and (b) offload as much data processing as possible to the CPU asynchronously, freeing GPU cycles for training.
• Using FFCV, we train ResNet-18 and ResNet-50 on the ImageNet dataset with competitive tradeoff between accuracy and training time.