Fetching the paper…
Reading the bibliography…
User Satisfaction Modeling (USM) is one of the popular choices for task-oriented dialogue systems evaluation, where user satisfaction typically depends on whether the user's task goals were fulfilled by the system.
The use of mmr, diversity-based reranking for reordering documents and producing summaries
Jaime Carbonell and Jade Goldstein. 1998 · 1998
Earlier work this paper cites.
Bleu: a method for automatic evaluation of machine translation
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002 · 2002
Earlier work this paper cites.
Rouge: A package for automatic evaluation of summaries
Chin-Yew Lin. 2004 · 2004
Earlier work this paper cites.
Modeling user satisfaction with hidden markov models
Klaus-Peter Engelbrecht, Florian Gödde, Felix Hartard, Hamed Ketabdar, and Sebastian Möller. 2009 · 2009
Earlier work this paper cites.
The elements of statistical learning: data mining, inference, and prediction , volume 2
Trevor Hastie, Robert Tibshirani, Jerome H Friedman, and Jerome H Friedman. 2009 · 2009
Earlier work this paper cites.
Adam: A method for stochastic optimization
Diederik P Kingma and Jimmy Ba. 2014 · 2014
Earlier work this paper cites.
Recurrent conditional random field for language understanding
Kaisheng Yao, Baolin Peng, Geoffrey Zweig, Dong Yu, Xiaolong Li, and Feng Gao. 2014 · 2014
Earlier work this paper cites.
Attention-based recurrent neural network models for joint intent detection and slot filling
Bing Liu and Ian Lane. 2016 · 2016
Earlier work this paper cites.
How not to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation
Chia-Wei Liu, Ryan Lowe, Iulian Serban, Michael Noseworthy, Laurent Charlin, and Joelle Pineau. 2016 · 2016
Earlier work this paper cites.
Hierarchical attention networks for document classification
Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016 · 2016
Earlier work this paper cites.
Gate-variants of gated recurrent unit (gru) neural networks
Rahul Dey and Fathi M Salem. 2017 · 2017
Earlier work this paper cites.
User interaction sequences for search satisfaction prediction
Rishabh Mehrotra, Imed Zitouni, Ahmed Hassan Awadallah, Ahmed El Kholy, and Madian Khabsa. 2017 · 2017
Earlier work this paper cites.
Neural belief tracker: Data-driven dialogue state tracking
Nikola Mrkšić, Diarmuid Ó Séaghdha, Tsung-Hsien Wen, Blaise Thomson, and Steve Young. 2017 · 2017
Earlier work this paper cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017 · 2017
Earlier work this paper cites.
Slot-gated modeling for joint slot filling and intent prediction
Chih-Wen Goo, Guang Gao, Yun-Kai Hsu, Chih-Li Huo, Tsung-Chieh Chen, Keng-Wei Hsu, and Yun-Nung Chen. 2018 · 2018
Earlier work this paper cites.
Measuring user satisfaction on smart speaker intelligent assistants using intent sensitive query embeddings
Seyyed Hadi Hashemi, Kyle Williams, Ahmed El Kholy, Imed Zitouni, and Paul A Crook. 2018 · 2018
Cited alongside, same era.
Sequicity: Simplifying task-oriented dialogue systems with single sequence-to-sequence architectures
Wenqiang Lei, Xisen Jin, Min-Yen Kan, Zhaochun Ren, Xiangnan He, and Dawei Yin. 2018 · 2018
Cited alongside, same era.
Towards deep conversational recommendations
Raymond Li, Samira Ebrahimi Kahou, Hannes Schulz, Vincent Michalski, Laurent Charlin, and Chris Pal. 2018 · 2018
Cited alongside, same era.
Mem2seq: Effectively incorporating knowledge bases into end-to-end task-oriented dialog systems
Andrea Madotto, Chien-Sheng Wu, and Pascale Fung. 2018 · 2018
Cited alongside, same era.
User intent, behaviour, and perceived satisfaction in product search
Ning Su, Jiyin He, Yiqun Liu, Min Zhang, and Shaoping Ma. 2018 · 2018
Cited alongside, same era.
A simple language model for task-oriented dialogue
Ehsan Hosseini-Asl, Bryan McCann, Chien-Sheng Wu, Semih Yavuz, and Richard Socher. 2020 · 2020
Later among the works it cites.
Dynamic fusion network for multi-domain end-to-end task-oriented dialog
Libo Qin, Xiao Xu, Wanxiang Che, Yue Zhang, and Ting Liu. 2020 · 2020
Later among the works it cites.
Towards scalable multi-domain conversational agents: The schema-guided dialogue dataset
Abhinav Rastogi, Xiaoxue Zang, Srinivas Sunkara, Raghav Gupta, and Pranav Khaitan. 2020 · 2020
Later among the works it cites.
Graphdialog: Integrating graph knowledge into end-to-end task-oriented dialogue systems
Shiquan Yang, Rui Zhang, and Sarah Erfani. 2020 · 2020
Later among the works it cites.
Overview of the ntcir-15 dialogue evaluation (dialeval-1) task
Zhaohao Zeng, Sosuke Kato, Tetsuya Sakai, and Inho Kang. 2020 · 2020
Later among the works it cites.
Survey on evaluation methods for dialogue systems
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Offline and online satisfaction prediction in open-domain conversational systems
Jason Ingyu Choi, Ali Ahmadvand, and Eugene Agichtein. 2019 · 2019
Cited alongside, same era.
Bert: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, and Kristina Toutanova. 2019 · 2019
Cited alongside, same era.
Higru: Hierarchical gated recurrent units for utterance-level emotion recognition
Wenxiang Jiao, Haiqin Yang, Irwin King, and Michael R Lyu. 2019 · 2019
Cited alongside, same era.
Jointly leveraging intent and interaction signals to predict user satisfaction with slate recommendations
Rishabh Mehrotra, Mounia Lalmas, Doug Kenney, Thomas Lim-Meng, and Golli Hashemian. 2019 · 2019
Cited alongside, same era.
Using customer service dialogues for satisfaction analysis with context-assisted multiple instance learning
Kaisong Song, Lidong Bing, Wei Gao, Jun Lin, Lujun Zhao, Jiancheng Wang, Changlong Sun, Xiaozhong Liu, and Qiong Zhang. 2019 · 2019
Cited alongside, same era.
Joint turn and dialogue level user satisfaction estimation on multi-domain conversations
Praveen Kumar Bodigutla, Aditya Tiwari, Spyros Matsoukas, Josep Valls-Vargas, and Lazaros Polymenakos. 2020 · 2020
Cited alongside, same era.
Predicting user intents and satisfaction with dialogue-based conversational recommendations
Wanling Cai and Li Chen. 2020 · 2020
Cited alongside, same era.
Jan Deriu, Alvaro Rodrigo, Arantxa Otegi, Guillermo Echegoyen, Sophie Rosset, Eneko Agirre, and Mark Cieliebak. 2021 · 2021
Later among the works it cites.
A sequence-to-sequence approach to dialogue state tracking
Yue Feng, Yang Wang, and Hang Li. 2021 · 2021
Later among the works it cites.
Self-supervised contrastive learning for efficient user satisfaction prediction in conversational agents
Mohammad Kachuee, Hao Yuan, Young-Bum Kim, and Sungjin Lee. 2021 · 2021
Later among the works it cites.
Simulating user satisfaction for the evaluation of task-oriented dialogue systems
Weiwei Sun, Shuo Zhang, Krisztian Balog, Zhaochun Ren, Pengjie Ren, Zhumin Chen, and Maarten de Rijke. 2021 · 2021
Later among the works it cites.
User satisfaction estimation with sequential dialogue act modeling in goal-oriented conversational systems
Yang Deng, Wenxuan Zhang, Wai Lam, Hong Cheng, and Helen Meng. 2022 · 2022
Later among the works it cites.
A multi-task based neural model to simulate users in goal-oriented dialogue systems
To Eun Kim and Aldo Lipani. 2022 · 2022
Later among the works it cites.
Multi-task pre-training for plug-and-play task-oriented dialogue system
Yixuan Su, Lei Shu, Elman Mansimov, Arshit Gupta, Deng Cai, Yi-An Lai, and Yi Zhang. 2022 · 2022
Later among the works it cites.
Assist: Towards label noise-robust dialogue state tracking
Fanghua Ye, Yue Feng, and Emine Yilmaz. 2022 · 2022
Later among the works it cites.
Automatic user experience evaluation of goal-oriented dialogs using pre-trained language models
Mika Rebensburg, Stefan Hillmann, and Nils Feldhus. 2023 · 2023
Closest in time.