The Multimodal And Modular Ai Chef: Complex Recipe Generation From Imagery

Understand

The AI community has embraced multi-sensory or multi-modal approaches to advance this generation of AI models to resemble expected intelligent understanding.

• Combining language and imagery represents a familiar method for specific tasks like image captioning or generation from descriptions.
• This paper compares these monolithic approaches to a lightweight and specialized method based on employing image models to label objects, then serially submitting this resulting object list to a large language model (LLM).
• This use of multiple Application Programming Interfaces (APIs) enables better than 95% mean average precision for correct object lists, which serve as input to the latest Open AI text generator (GPT-4).