STEPS: A Benchmark for Order Reasoning in Sequential Tasks

Understand

Various human activities can be abstracted into a sequence of actions in natural text, i.e.

• cooking, repairing, manufacturing, etc.
• Such action sequences heavily depend on the executing order, while disorder in action sequences leads to failure of further task execution by robots or AI agents.
• Therefore, to verify the order reasoning capability of current neural models in sequential tasks, we propose a challenging benchmark , named STEPS.