Compositional Embeddings Using Complementary Partitions for Memory-Efficient Recommendation Systems

Understand

Modern deep learning-based recommendation systems exploit hundreds to thousands of different categorical features, each with millions of different categories ranging from clicks to posts.

• To respect the natural diversity within the categorical data, embeddings map each category to a unique dense representation within an embedded space.
• Since each categorical feature could take on as many as tens of millions of different possible categories, the embedding tables form the primary memory bottleneck during both training and inference.
• We propose a novel approach for reducing the embedding size in an end-to-end fashion by exploiting complementary partitions of the category set to produce a unique embedding vector for each category without explicit definition.