Integrating kNN with Foundation Models for Adaptable and Privacy-Aware Image Classification

Understand

Traditional deep learning models implicity encode knowledge limiting their transparency and ability to adapt to data changes.

• Yet, this adaptability is vital for addressing user data privacy concerns.
• We address this limitation by storing embeddings of the underlying training data independently of the model weights, enabling dynamic data modifications without retraining.
• Specifically, our approach integrates the $k$-Nearest Neighbor ($k$-NN) classifier with a vision-based foundation model, pre-trained self-supervised on natural images, enhancing interpretability and adaptability.