Fetching the paper…

Reading the bibliography…

2022

A Multi-Task BERT Model for Schema-Guided Dialogue State Tracking

Kapelonis, Eleftherios, Georgiou, Efthymios, Potamianos, Alexandros

Original
PDF
Post on X

Understand

Task-oriented dialogue systems often employ a Dialogue State Tracker (DST) to successfully complete conversations.

Recent state-of-the-art DST implementations rely on schemata of diverse services to improve model robustness and handle zero-shot generalization to new domains [1], however such methods [2, 3] typically require multiple large scale transformer models and long input sequences to perform well.
We propose a single multi-task BERT-based model that jointly solves the three DST tasks of intent prediction, requested slot prediction and slot filling.
Moreover, we propose an efficient and parsimonious encoding of the dialogue history and service schemata that is shown to further improve performance.

A Multi-Task BERT Model for Schema-Guided Dialogue State Tracking · Around

Built on

J. Williams, A. Raux, D. Ramachandran, and A. Black, “The dialog state tracking challenge,” in Proceedings of the SIGDIAL 2013 Conference . Metz, France: Association for Computational Linguistics, Aug. 2013, pp. 404–413
2013
Earlier work this paper cites.
M. Henderson, B. Thomson, and J. D. Williams, “The second dialog state tracking challenge,” in Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL) . Philadelphia, PA, U.S.A.: Association for Computational Linguistics, Jun. 2014, pp. 263–272
2014
Earlier work this paper cites.
M. Henderson, B. Thomson, and S. J. Young, “Word-based dialog state tracking with recurrent neural networks,” in SIGDIAL Conference , 2014
2014
Earlier work this paper cites.
A. Bordes and J. Weston, “Learning end-to-end goal-oriented dialog,” CoRR , vol. abs/1605.07683, 2016
Original
2016
Earlier work this paper cites.
T. Wen, M. Gasic, N. Mrksic, L. M. Rojas-Barahona, P. Su, S. Ultes, D. Vandyke, and S. J. Young, “A network-based end-to-end trainable task-oriented dialogue system,” CoRR , vol. abs/1604.04562, 2016
Original
2016
Earlier work this paper cites.
N. Mrksic, D. Ó. Séaghdha, T. Wen, B. Thomson, and S. J. Young, “Neural belief tracker: Data-driven dialogue state tracking,” CoRR , vol. abs/1606.03777, 2016
Original
2016
Earlier work this paper cites.
A. Rastogi, D. Hakkani-Tür, and L. P. Heck, “Scalable multi-domain dialogue state tracking,” CoRR , vol. abs/1712.10224, 2017
Original
2017
Earlier work this paper cites.
I. Loshchilov and F. Hutter, “Fixing weight decay regularization in adam,” CoRR , vol. abs/1711.05101, 2017
Original
2017
Earlier work this paper cites.
P. Budzianowski, T.-H. Wen, B.-H. Tseng, I. Casanueva, S. Ultes, O. Ramadan, and M. Gašić, “Multiwoz–a large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling,” arXiv preprint arXiv:1810.00278 , 2018
Original
2018
Earlier work this paper cites.
L. Ren, K. Xie, L. Chen, and K. Yu, “Towards universal dialogue state tracking,” CoRR , vol. abs/1810.09587, 2018
Original
2018
Earlier work this paper cites.
J. Devlin, M. Chang, K. Lee, and K. Toutanova, “BERT: pre-training of deep bidirectional transformers for language understanding,” CoRR , vol. abs/1810.04805, 2018
Original
2018
Earlier work this paper cites.

Similar

M. Eric, R. Goel, S. Paul, A. Sethi, S. Agarwal, S. Gao, and D. Hakkani-Tür, “Multiwoz 2.1: Multi-domain dialogue state corrections and state tracking baselines,” 2019
2019
Cited alongside, same era.
Z. Yang, Z. Dai, Y. Yang, J. G. Carbonell, R. Salakhutdinov, and Q. V. Le, “Xlnet: Generalized autoregressive pretraining for language understanding,” CoRR , vol. abs/1906.08237, 2019
Original
2019
Cited alongside, same era.
A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, “Language models are unsupervised multitask learners,” 2019
2019
Cited alongside, same era.
S. Kim, S. Yang, G. Kim, and S. Lee, “Efficient dialogue state tracking by selectively overwriting memory,” CoRR , vol. abs/1911.03906, 2019
Original
2019
Cited alongside, same era.

Then

C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” Journal of Machine Learning Research , vol. 21, no. 140, pp. 1–67, 2020
2020
Later among the works it cites.
Y.-P. Ruan, Z.-H. Ling, J.-C. Gu, and Q. Liu, “Fine-tuning bert for schema-guided zero-shot dialogue state tracking,” 2020
2020
Later among the works it cites.
M. Li, H. Xiong, and Y. Cao, “The SPPD system for schema guided dialogue state tracking challenge,” CoRR , vol. abs/2006.09035, 2020
Original
2020
Later among the works it cites.
T. Han, X. Liu, R. Takanabu, Y. Lian, C. Huang, D. Wan, W. Peng, and M. Huang, “Multiwoz 2.3: A multi-domain task-oriented dialogue dataset enhanced with annotation corrections and co-reference annotation,” in CCF International Conference on Natural Language Processing and Chinese Computing . Springer, 2021, pp. 206–218
2021

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

J. Wei and K. Zou, “EDA: Easy data augmentation techniques for boosting performance on text classification tasks,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) . Hong Kong, China: Association for Computational Linguistics, Nov. 2019, pp. 6383–6389

2019

Cited alongside, same era.

A. Rastogi, X. Zang, S. Sunkara, R. Gupta, and P. Khaitan, “Towards scalable multi-domain conversational agents: The schema-guided dialogue dataset,” 2020

2020

Cited alongside, same era.

Y. Ma, Z. Zeng, D. Zhu, X. Li, Y. Yang, X. Yao, K. Zhou, and J. Shen, “An end-to-end dialogue state tracking system with machine reading comprehension and wide & deep classification,” 2020

2020

Cited alongside, same era.

Z. Zhang, R. Takanobu, M. Huang, and X. Zhu, “Recent advances and challenges in task-oriented dialog system,” CoRR , vol. abs/2003.07490, 2020

Original

2020

Cited alongside, same era.

X. Zang, A. Rastogi, S. Sunkara, R. Gupta, J. Zhang, and J. Chen, “Multiwoz 2.2: A dialogue dataset with additional annotation corrections and state tracking baselines,” arXiv preprint arXiv:2007.12720 , 2020

2020

Cited alongside, same era.

A. Rastogi, X. Zang, S. Sunkara, R. Gupta, and P. Khaitan, “Schema-guided dialogue state tracking task at DSTC8,” CoRR , vol. abs/2002.01359, 2020

2020

Cited alongside, same era.

J. E. M. Mosig, S. Mehri, and T. Kober, “STAR: A schema-guided dialog dataset for transfer learning,” CoRR , vol. abs/2010.11853, 2020

2020

Cited alongside, same era.

Later among the works it cites.

F. Ye, J. Manotumruksa, and E. Yilmaz, “Multiwoz 2.4: A multi-domain task-oriented dialogue dataset with essential annotation corrections to improve state tracking evaluation,” arXiv preprint arXiv:2104.00773 , 2021

2021

Later among the works it cites.

J. Cao and Y. Zhang, “A comparative study on schema-guided dialogue state tracking,” in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies . Online: Association for Computational Linguistics, Jun. 2021, pp. 782–796

2021

Later among the works it cites.

C. Lee, H. Cheng, and M. Ostendorf, “Dialogue state tracking with a language model using schema-driven prompting,” CoRR , vol. abs/2109.07506, 2021

2021

Later among the works it cites.

J. Zhao, M. Mahdieh, Y. Zhang, Y. Cao, and Y. Wu, “Effective sequence-to-sequence dialogue state tracking,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing . Online and Punta Cana, Dominican Republic: Association for Computational Linguistics, Nov. 2021, pp. 7486–7493

2021

Later among the works it cites.

Z. Lin, B. Liu, S. Moon, P. A. Crook, Z. Zhou, Z. Wang, Z. Yu, A. Madotto, E. Cho, and R. Subba, “Leveraging slot descriptions for zero-shot cross-domain dialogue state tracking,” CoRR , vol. abs/2105.04222, 2021

2021

Later among the works it cites.

J. Zhao, R. Gupta, Y. Cao, D. Yu, M. Wang, H. Lee, A. Rastogi, I. Shafran, and Y. Wu, “Description-driven task-oriented dialog modeling,” 2022

2022

Closest in time.