Simulating User Agents for Embodied Conversational-AI

ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks

Original

Mohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk, Winson Han, Roozbeh Mottaghi, Luke Zettlemoyer, and Dieter Fox · 1912

Earlier work this paper cites.

The hidden agenda user simulation model

Jost Schatzmann and Steve Young · 2009

Earlier work this paper cites.

A sequence-to-sequence model for user simulation in spoken dialogue systems

Original

Layla El Asri, Jing He, and Kaheer Suleman · 2016

Earlier work this paper cites.

AI2-THOR: An Interactive 3D Environment for Visual AI

Eric Kolve, Roozbeh Mottaghi, Winson Han, Eli VanderBilt, Luca Weihs, Alvaro Herrasti, Daniel Gordon, Yuke Zhu, Abhinav Gupta, and Ali Farhadi · 2017

Earlier work this paper cites.

User modeling for task oriented dialogues

Izzeddin Gür, Dilek Hakkani-Tür, Gokhan Tür, and Pararth Shah · 2018

Earlier work this paper cites.

How to build user simulators to train RL-based dialog systems

Weiyan Shi, Kun Qian, Xuewei Wang, and Zhou Yu · 2019

Cited alongside, same era.

Vision-and-dialog navigation

Jesse Thomason, Michael Murray, Maya Cakmak, and Luke Zettlemoyer · 2019

Cited alongside, same era.

How should agents ask questions for situated learning? an annotated dialogue corpus

Felix Gervits, Antonio Roque, Gordon Briggs, Matthias Scheutz, and Matthew Marge · 2021

Cited alongside, same era.

Dialfred: Dialogue-enabled agents for embodied instruction following

Xiaofeng Gao, Qiaozi Gao, Ran Gong, Kaixiang Lin, Govind Thattai, and Gaurav S Sukhatme · 2022

Cited alongside, same era.

Dialog acts for task-driven embodied agents

Spandana Gella, Aishwarya Padmakumar, Patrick Lange, and Dilek Hakkani-Tür · 2022

Cited alongside, same era.

TEACh: task-driven embodied agents that chat

Aishwarya Padmakumar, Jesse Thomason, Ayush Shrivastava, Patrick Lange, Anjali Narayan-Chen, Spandana Gella, Robinson Piramuthu, Gokhan Tur, and Dilek Hakkani-Tür · 2022

Later among the works it cites.

Think, act, and ask: Open-world interactive personalized robot navigation

Original

Yinpei Dai, Run Peng, Sikai Li, and Joyce Chai · 2023

Later among the works it cites.

User simulation with large language models for evaluating task-oriented dialogue

Original

Sam Davidson, Salvatore Romeo, Raphael Shu, James Gung, Arshit Gupta, Saab Mansour, and Yi Zhang · 2023

Later among the works it cites.

Multimodal embodied plan prediction augmented with synthetic embodied dialogue

Aishwarya Padmakumar, Mert Inan, Spandana Gella, Patrick Lange, and Dilek Hakkani-Tür · 2023

Later among the works it cites.

Helper-x: A unified instructable embodied agent to tackle four interactive vision-language domains with memory-augmented language models