Fetching the paper…
Reading the bibliography…
We introduce ClarQ-LLM, an evaluation framework consisting of bilingual English-Chinese conversation tasks, conversational agents and evaluation metrics, designed to serve as a strong benchmark for assessing agents' ability to ask clarification questions in task-oriented dialogues.
V. Zue and J. Glass, “Conversational interfaces: advances and challenges,” Proceedings of the IEEE , vol. 88, no. 8, pp. 1166–1180, 2000
2000
Earlier work this paper cites.
M. Purver, J. Ginzburg, and P. Healey, “On the means for clarification in dialogue,” in Proceedings of the Second SIGdial Workshop on Discourse and Dialogue , 2001. [Online]. Available: https://aclanthology.org/W01-1616
2001
Earlier work this paper cites.
K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a method for automatic evaluation of machine translation,” in Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics , P. Isabelle, E. Charniak, and D. Lin, Eds. Philadelphia, Pennsylvania, USA: Association for Computational Linguistics, Jul. 2002, pp. 311–318. [Online]. Available: https://aclanthology.org/P02-1040
2002
Earlier work this paper cites.
C.-Y. Lin, “ROUGE: A package for automatic evaluation of summaries,” in Text Summarization Branches Out . Barcelona, Spain: Association for Computational Linguistics, Jul. 2004, pp. 74–81. [Online]. Available: https://aclanthology.org/W04-1013
2004
Earlier work this paper cites.
K. J. Rodríguez and D. Schlangen, “Form, intonation and function of clarification requests in german task-oriented spoken dialogues,” in Proceedings of the 8th Workshop on the Semantics and Pragmatics of Dialogue - Full Papers . Barcelona, Spain: SEMDIAL, Jul. 19-21 2004. [Online]. Available: http://semdial.org/anthology/017_Rodriguez_semdial2004_catalog_proceedings.pdf
2004
Earlier work this paper cites.
S. Banerjee and A. Lavie, “METEOR: An automatic metric for MT evaluation with improved correlation with human judgments,” in Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization , J. Goldstein, A. Lavie, C.-Y. Lin, and C. Voss, Eds. Ann Arbor, Michigan: Association for Computational Linguistics, Jun. 2005, pp. 65–72. [Online]. Available: https://aclanthology.org/W05-0909
2005
Earlier work this paper cites.
V. Rieser and J. Moore, “Implications for generating clarification requests in task-oriented dialogues,” in Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05) , K. Knight, H. T. Ng, and K. Oflazer, Eds. Ann Arbor, Michigan: Association for Computational Linguistics, Jun. 2005, pp. 239–246. [Online]. Available: https://aclanthology.org/P05-1030
2005
Earlier work this paper cites.
J. Williams, A. Raux, D. Ramachandran, and A. Black, “The dialog state tracking challenge,” in Proceedings of the SIGDIAL 2013 Conference , M. Eskenazi, M. Strube, B. Di Eugenio, and J. D. Williams, Eds. Metz, France: Association for Computational Linguistics, Aug. 2013, pp. 404–413. [Online]. Available: https://aclanthology.org/W13-4065
2013
Earlier work this paper cites.
M. Eric and C. D. Manning, “Key-value retrieval networks for task-oriented dialogue,” 2017
2017
Earlier work this paper cites.
T.-H. Wen, D. Vandyke, N. Mrkšić, M. Gašić, L. M. Rojas-Barahona, P.-H. Su, S. Ultes, and S. Young, “A network-based end-to-end trainable task-oriented dialogue system,” in Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers , M. Lapata, P. Blunsom, and A. Koller, Eds. Valencia, Spain: Association for Computational Linguistics, Apr. 2017, pp. 438–449. [Online]. Available: https://aclanthology.org/E17-1042
2017
Earlier work this paper cites.
P. Budzianowski, T.-H. Wen, B.-H. Tseng, I. Casanueva, S. Ultes, O. Ramadan, and M. Gašić, “MultiWOZ - a large-scale multi-domain Wizard-of-Oz dataset for task-oriented dialogue modelling,” in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing , E. Riloff, D. Chiang, J. Hockenmaier, and J. Tsujii, Eds. Brussels, Belgium: Association for Computational Linguistics, Oct.-Nov. 2018, pp. 5016–5026. [Online]. Available: https://aclanthology.org/D18-1547
2018
Earlier work this paper cites.
M. Saeidi, M. Bartolo, P. Lewis, S. Singh, T. Rocktäschel, M. Sheldon, G. Bouchard, and S. Riedel, “Interpretation of natural language rules in conversational machine reading,” Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018 , pp. 2087–2097, 2018. [Online]. Available: https://aclanthology.org/D18-1233
2018
Cited alongside, same era.
S. Rao and H. Daumé III, “Learning to ask good questions: Ranking clarification questions using neural expected value of perfect information,” in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , I. Gurevych and Y. Miyao, Eds. Melbourne, Australia: Association for Computational Linguistics, Jul. 2018, pp. 2737–2746. [Online]. Available: https://aclanthology.org/P18-1255
2018
Cited alongside, same era.
——, “Answer-based Adversarial Training for Generating Clarification Questions,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , J. Burstein, C. Doran, and T. Solorio, Eds. Minneapolis, Minnesota: Association for Computational Linguistics, Jun. 2019, pp. 143–155. [Online]. Available: https://aclanthology.org/N19-1013
Y. Sun, S. Wang, S. Feng, S. Ding, C. Pang, J. Shang, J. Liu, X. Chen, Y. Zhao, Y. Lu, W. Liu, Z. Wu, W. Gong, J. Liang, Z. Shang, P. Sun, W. Liu, X. Ouyang, D. Yu, H. Tian, H. Wu, and H. Wang, “Ernie 3.0: Large-scale knowledge enhanced pre-training for language understanding and generation,” 2021
2021
Later among the works it cites.
L. Qin, W. Pan, Q. Chen, L. Liao, Z. Yu, Y. Zhang, W. Che, and M. Li, “End-to-end task-oriented dialogue: A survey of tasks, methods, and future directions,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , H. Bouamor, J. Pino, and K. Bali, Eds. Singapore: Association for Computational Linguistics, Dec. 2023, pp. 5925–5941. [Online]. Available: https://aclanthology.org/2023.emnlp-main.363
2023
Later among the works it cites.
B. Madureira and D. Schlangen, “Instruction clarification requests in multimodal collaborative dialogue games: Tasks, and an analysis of the CoDraw dataset,” in Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics . Dubrovnik, Croatia: Association for Computational Linguistics, May 2023, pp. 2303–2319. [Online]. Available: https://aclanthology.org/2023.eacl-main.169
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
2019
Cited alongside, same era.
J. Xu, Y. Wang, D. Tang, N. Duan, P. Yang, Q. Zeng, M. Zhou, and X. Sun, “Asking clarification questions in knowledge-based question answering,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) , K. Inui, J. Jiang, V. Ng, and X. Wan, Eds. Hong Kong, China: Association for Computational Linguistics, Nov. 2019, pp. 1618–1629. [Online]. Available: https://aclanthology.org/D19-1172
2019
Cited alongside, same era.
M. Aliannejadi, H. Zamani, F. Crestani, and W. B. Croft, “Asking clarifying questions in open-domain information-seeking conversations,” in Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval , ser. SIGIR’19. New York, NY, USA: Association for Computing Machinery, 2019, p. 475–484. [Online]. Available: https://doi.org/10.1145/3331184.3331265
2019
Cited alongside, same era.
M. Eric, R. Goel, S. Paul, A. Sethi, S. Agarwal, S. Gao, A. Kumar, A. Goyal, P. Ku, and D. Hakkani-Tur, “MultiWOZ 2.1: A consolidated multi-domain dialogue dataset with state corrections and state tracking baselines,” in Proceedings of the Twelfth Language Resources and Evaluation Conference , N. Calzolari, F. Béchet, P. Blache, K. Choukri, C. Cieri, T. Declerck, S. Goggi, H. Isahara, B. Maegaard, J. Mariani, H. Mazo, A. Moreno, J. Odijk, and S. Piperidis, Eds. Marseille, France: European Language Resources Association, May 2020, pp. 422–428. [Online]. Available: https://aclanthology.org/2020.lrec-1.53
2020
Cited alongside, same era.
J. Quan, S. Zhang, Q. Cao, Z. Li, and D. Xiong, “RiSAWOZ: A large-scale multi-domain Wizard-of-Oz dataset with rich semantic annotations for task-oriented dialogue modeling,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , B. Webber, T. Cohn, Y. He, and Y. Liu, Eds. Online: Association for Computational Linguistics, Nov. 2020, pp. 930–940. [Online]. Available: https://aclanthology.org/2020.emnlp-main.67
2020
Cited alongside, same era.
H. Zamani, G. Lueck, E. Chen, R. Quispe, F. Luu, and N. Craswell, “Mimics: A large-scale data collection for search clarification,” 2020
2020
Cited alongside, same era.
V. Balaraman, S. Sheikhalishahi, and B. Magnini, “Recent neural methods on dialogue state tracking for task-oriented dialogue systems: A survey,” in Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue , H. Li, G.-A. Levow, Z. Yu, C. Gupta, B. Sisman, S. Cai, D. Vandyke, N. Dethlefs, Y. Wu, and J. J. Li, Eds. Singapore and Online: Association for Computational Linguistics, Jul. 2021, pp. 239–251. [Online]. Available: https://aclanthology.org/2021.sigdial-1.25
2021
Cited alongside, same era.
Y. Dai, H. Li, Y. Li, J. Sun, F. Huang, L. Si, and X. Zhu, “Preview, attend and review: Schema-aware curriculum learning for multi-domain dialogue state tracking,” ACL-IJCNLP 2021 - 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, Proceedings of the Conference , vol. 2, pp. 879–885, 2021. [Online]. Available: https://aclanthology.org/2021.acl-short.111
2021
Cited alongside, same era.
M. Aliannejadi, J. Kiseleva, A. Chuklin, J. Dalton, and M. Burtsev, “Building and evaluating open-domain dialogue corpora with clarifying questions,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , M.-F. Moens, X. Huang, L. Specia, and S. W.-t. Yih, Eds. Online and Punta Cana, Dominican Republic: Association for Computational Linguistics, Nov. 2021, pp. 4473–4484. [Online]. Available: https://aclanthology.org/2021.emnlp-main.367
2021
Cited alongside, same era.
2023
Later among the works it cites.
2023
Later among the works it cites.
H. A. Rahmani, X. Wang, Y. Feng, Q. Zhang, E. Yilmaz, and A. Lipani, “A survey on asking clarification questions datasets in conversational systems,” Proceedings of the Annual Meeting of the Association for Computational Linguistics , vol. 1, pp. 2698–2716, 2023. [Online]. Available: https://aclanthology.org/2023.acl-long.152
2023
Later among the works it cites.
2023
Later among the works it cites.
H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, D. Bikel, L. Blecher, C. C. Ferrer, M. Chen, G. Cucurull, D. Esiobu, J. Fernandes, J. Fu, W. Fu, B. Fuller, C. Gao, V. Goswami, N. Goyal, A. Hartshorn, S. Hosseini, R. Hou, H. Inan, M. Kardas, V. Kerkez, M. Khabsa, I. Kloumann, A. Korenev, P. S. Koura, M.-A. Lachaux, T. Lavril, J. Lee, D. Liskovich, Y. Lu, Y. Mao, X. Martinet, T. Mihaylov, P. Mishra, I. Molybog, Y. Nie, A. Poulton, J. Reizenstein, R. Rungta, K. Saladi, A. Schelten, R. Silva, E. M. Smith, R. Subramanian, X. E. Tan, B. Tang, R. Taylor, A. Williams, J. X. Kuan, P. Xu, Z. Yan, I. Zarov, Y. Zhang, A. Fan, M. Kambadur, S. Narang, A. Rodriguez, R. Stojnic, S. Edunov, and T. Scialom, “Llama 2: Open foundation and fine-tuned chat models,” 2023
2023
Later among the works it cites.
L. Zheng, W.-L. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. P. Xing, H. Zhang, J. E. Gonzalez, and I. Stoica, “Judging llm-as-a-judge with mt-bench and chatbot arena,” 2023
2023
Later among the works it cites.
C. Li, Y. Gan, Z. Yang, Y. Chen, X. Qiu, Y. Lin, M. Purver, and M. Poesio, “Analyzing and enhancing clarification strategies for ambiguous references in consumer service interactions,” in Proceedings of the 25th Annual Meeting of the Special Interest Group on Discourse and Dialogue . Kyoto, Japan: Association for Computational Linguistics, Sep. 2024
2024
Closest in time.
2024
Closest in time.
O. Shaikh, K. Gligoric, A. Khetan, M. Gerstgrasser, D. Yang, and D. Jurafsky, “Grounding gaps in language model generations,” in Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , K. Duh, H. Gomez, and S. Bethard, Eds. Mexico City, Mexico: Association for Computational Linguistics, Jun. 2024, pp. 6279–6296. [Online]. Available: https://aclanthology.org/2024.naacl-long.348
2024
Closest in time.