Recognition of Heat-Induced Food State Changes by Time-Series Use of Vision-Language Model for Cooking Robot

Understand

Cooking tasks are characterized by large changes in the state of the food, which is one of the major challenges in robot execution of cooking tasks.

• In particular, cooking using a stove to apply heat to the foodstuff causes many special state changes that are not seen in other tasks, making it difficult to design a recognizer.
• In this study, we propose a unified method for recognizing changes in the cooking state of robots by using the vision-language model that can discriminate open-vocabulary objects in a time-series manner.
• We collected data on four typical state changes in cooking using a real robot and confirmed the effectiveness of the proposed method.