Multi-Label Product Categorization Using Multi-Modal Fusion Models

Understand

In this study, we investigated multi-modal approaches using images, descriptions, and titles to categorize e-commerce products on Amazon.

• Specifically, we examined late fusion models, where the modalities are fused at the decision level.
• Products were each assigned multiple labels, and the hierarchy in the labels were flattened and filtered.
• For our individual baseline models, we modified a CNN architecture to classify the description and title, and then modified Keras' ResNet-50 to classify the images, achieving $F_1$ scores of 77.0%, 82.7%, and 61.0%, respectively.