Do Large Language Models Have an English Accent? Evaluating and Improving the Naturalness of Multilingual LLMs

Bleu: a method for automatic evaluation of machine translation

Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002 · 2002

Earlier work this paper cites.

ROUGE: A package for automatic evaluation of summaries

Chin-Yew Lin. 2004 · 2004

Earlier work this paper cites.

Weisfeiler-lehman graph kernels

Nino Shervashidze, Pascal Schweitzer, Erik Jan Van Leeuwen, Kurt Mehlhorn, and Karsten M Borgwardt. 2011 · 2011

Earlier work this paper cites.

A kernel two-sample test

Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander Smola. 2012 · 2012

Earlier work this paper cites.

On the features of translationese

Vered Volansky, Noam Ordan, and Shuly Wintner. 2015 · 2015

Earlier work this paper cites.

Crowd-sourcing NLG data: Pictures elicit better data

Jekaterina Novikova, Oliver Lemon, and Verena Rieser. 2016 · 2016

Earlier work this paper cites.

Translationese: Between human and machine translation

Shuly Wintner. 2016 · 2016

Earlier work this paper cites.

Treat the system like a human student: Automatic naturalness evaluation of generated text without reference texts

Isabel Groves, Ye Tian, and Ioannis Douratsos. 2018 · 2018

Earlier work this paper cites.

How human is machine translationese? comparing human and machine translations of text and speech

Yuri Bizzoni, Tom S Juzek, Cristina España-Bonet, Koel Dutta Chowdhury, Josef van Genabith, and Elke Teich. 2020 · 2020

Earlier work this paper cites.

Universal Dependencies v2: An evergrowing multilingual treebank collection

Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Jan Hajič, Christopher D. Manning, Sampo Pyysalo, Sebastian Schuster, Francis Tyers, and Daniel Zeman. 2020 · 2020

Earlier work this paper cites.

Stanza: A python natural language processing toolkit for many human languages

Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D. Manning. 2020 · 2020

Earlier work this paper cites.

Grakel: A graph kernel library in python

Giannis Siglidis, Giannis Nikolentzos, Stratis Limnios, Christos Giatsidis, Konstantinos Skianis, and Michalis Vazirgiannis. 2020 · 2020

Earlier work this paper cites.

Transformers: State-of-the-art natural language processing

Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020 · 2020

Earlier work this paper cites.

Bertscore: Evaluating text generation with bert

Tianyi Zhang*, Varsha Kishore*, Felix Wu*, Kilian Q. Weinberger, and Yoav Artzi. 2020 · 2020

Earlier work this paper cites.

Measuring massive multitask language understanding

Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021 · 2021

Earlier work this paper cites.

Naturalness evaluation of natural language generation in task-oriented dialogues using BERT

Ye Liu, Wolfgang Maier, Wolfgang Minker, and Stefan Ultes. 2021 · 2021

Earlier work this paper cites.

Language model evaluation beyond perplexity

Clara Meister and Ryan Cotterell. 2021 · 2021

Earlier work this paper cites.

Mauve: Measuring the gap between neural text and human text using divergence frontiers

Krishna Pillutla, Swabha Swayamdipta, Rowan Zellers, John Thickstun, Sean Welleck, Yejin Choi, and Zaid Harchaoui. 2021 · 2021

Cited alongside, same era.

Evaluating the evaluation of diversity in natural language generation

Guy Tevet and Jonathan Berant. 2021 · 2021

Cited alongside, same era.

Machine translationese: Effects of algorithmic bias on linguistic complexity in machine translation

Eva Vanmassenhove, Dimitar Shterionov, and Matthew Gwilliam. 2021 · 2021

Cited alongside, same era.

FairLex: A multilingual benchmark for evaluating fairness in legal text processing

Ilias Chalkidis, Tommaso Pasini, Sheng Zhang, Letizia Tomada, Sebastian Schwemer, and Anders Søgaard. 2022 · 2022

Cited alongside, same era.

A natural diet: Towards improving naturalness of machine translation output

Markus Freitag, David Vilar, David Grangier, Colin Cherry, and George Foster. 2022 · 2022

Cited alongside, same era.

Direct preference optimization: Your language model is secretly a reward model

Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023 · 2023

Later among the works it cites.

Firefly(流萤): 中文对话式大语言模型

Jianxin Yang. 2023 · 2023

Later among the works it cites.

Judging LLM-as-a-judge with MT-bench and chatbot arena

Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023 · 2023

Later among the works it cites.

Phi-3 technical report: A highly capable language model locally on your phone

Original

Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Harkirat Behl, et al. 2024 · 2024

Closest in time.

Iterative translation refinement with large language models

Pinzhen Chen, Zhicheng Guo, Barry Haddow, and Kenneth Heafield. 2024 · 2024

Closest in time.