Revisit Input Perturbation Problems for LLMs: A Unified Robustness Evaluation Framework for Noisy Slot Filling Task

Coucke, A., Saade, A., Ball, A., Bluche, T., Caulier, A., Leroy, D., Doumouro, C., Gisselbrecht, T., Caltagirone, F., Lavril, T., Primet, M., Dureau, J.: Snips voice platform: an embedded spoken language understanding system for private-by-design voice interfaces (2018)

2018

Earlier work this paper cites.

Ma, E.: Nlp augmentation. https://github.com/makcedward/nlpaug (2019)

2019

Earlier work this paper cites.

Reimers, N., Gurevych, I.: Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084 (2019)

Original

2019

Earlier work this paper cites.

Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T.J., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners. ArXiv abs/2005.14165

Original

2020

Earlier work this paper cites.

Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are few-shot learners (2020)

2020

Earlier work this paper cites.

Gopalakrishnan, K., Hedayatnia, B., Wang, L., Liu, Y., Hakkani-Tur, D.: Are neural open-domain dialog systems robust to speech recognition errors in the dialog history? an empirical study (2020)

2020

Earlier work this paper cites.

Namysl, M., Behnke, S., Köhler, J.: NAT: Noise-aware training for robust neural sequence labeling. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. pp. 1501–1517. Association for Computational Linguistics, Online (Jul 2020). https://doi.org/10.18653/v1/2020.acl-main.138, https://aclanthology.org/2020.acl-main.138

2020

Earlier work this paper cites.

Peng, B., Li, C., Zhang, Z., Zhu, C., Li, J., Gao, J.: Raddle: An evaluation benchmark and analysis platform for robust task-oriented dialog systems. arXiv preprint arXiv:2012.14666 (2020)

Original

2020

Earlier work this paper cites.

Ruan, W., Nechaev, Y., Chen, L., Su, C., Kiss, I.: Towards an asr error robust spoken language understanding system. In: Interspeech 2020 (2020), https://www.amazon.science/publications/towards-an-asr-error-robust-spoken-language-understanding-system

2020

Earlier work this paper cites.

Gui, T., Wang, X., Zhang, Q., Liu, Q., Zou, Y., Zhou, X., Zheng, R., Zhang, C., Wu, Q., Ye, J., Pang, Z., Zhang, Y., Li, Z., Ma, R., Fei, Z., Cai, R., Zhao, J., Hu, X., Yan, Z., Tan, Y., Hu, Y., Bian, Q., Liu, Z., Zhu, B., Qin, S., Xing, X., Fu, J., Zhang, Y., Peng, M., Zheng, X., Zhou, Y., Wei, Z., Qiu, X., Huang, X.: Textflint: Unified multilingual robustness evaluation toolkit for natural language processing (2021)

2021

Earlier work this paper cites.

Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: Lora: Low-rank adaptation of large language models (2021)

2021

Earlier work this paper cites.

Lin, J., Men, R., Yang, A., Zhou, C., Zhang, Y., Wang, P., Zhou, J., Tang, J., Yang, H.: M6: Multi-modality-to-multi-modality multitask mega-transformer for unified pretraining. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery Data Mining. p. 3251–3261. KDD ’21, Association for Computing Machinery, New York, NY, USA (2021). https://doi.org/10.1145/3447548.3467206, https://doi.org/10.1145/3447548.3467206

2021

Earlier work this paper cites.

Moradi, M., Samwald, M.: Evaluating the robustness of neural language models to input perturbations. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. pp. 1558–1570. Association for Computational Linguistics, Online and Punta Cana, Dominican Republic (Nov 2021). https://doi.org/10.18653/v1/2021.emnlp-main.117, https://aclanthology.org/2021.emnlp-main.117

2021

Cited alongside, same era.

Wu, D., Chen, Y., Ding, L., Tao, D.: Bridging the gap between clean data training and real-world inference for spoken language understanding. arXiv preprint arXiv:2104.06393 (2021)

Original

2021

Cited alongside, same era.

Dong, G., Guo, D., Wang, L., Li, X., Wang, Z., Zeng, C., He, K., Zhao, J., Lei, H., Cui, X., Huang, Y., Feng, J., Xu, W.: PSSAT: A perturbed semantic structure awareness transferring method for perturbation-robust slot filling. In: Proceedings of the 29th International Conference on Computational Linguistics. pp. 5327–5334. International Committee on Computational Linguistics, Gyeongju, Republic of Korea (Oct 2022), https://aclanthology.org/2022.coling-1.473

2022

Lei, S., Dong, G., Wang, X., Wang, K., Wang, S.: Instructerc: Reforming emotion recognition in conversation with a retrieval multi-task llms framework. arXiv preprint arXiv:2309.11911 (2023)

Original

2023

Closest in time.

Li, C., Yuan, Z., Dong, G., Lu, K., Wu, J., Tan, C., Wang, X., Zhou, C.: Query and response augmentation cannot help out-of-domain math reasoning generalization (2023)

2023

Closest in time.

Li, X., Wang, L., Dong, G., He, K., Zhao, J., Lei, H., Liu, J., Xu, W.: Generative zero-shot prompt learning for cross-domain slot filling with inverse prompting. In: Findings of the Association for Computational Linguistics: ACL 2023. pp. 825–834. Association for Computational Linguistics, Toronto, Canada (Jul 2023), https://aclanthology.org/2023.findings-acl.52

2023

Closest in time.

Liu, J., Wang, L., Dong, G., Song, X., Wang, Z., Wang, Z., Lei, S., Zhao, J., He, K., Xiao, B., Xu, W.: Towards robust and generalizable training: An empirical study of noisy slot filling for input perturbations (2023)