Fetching the paper…
Reading the bibliography…
Despite rising global usage of large language models (LLMs), their ability to generate long-form answers to culturally specific questions remains unexplored in many languages.
Bertscore: Evaluating text generation with bert
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2019 · 1904
Earlier work this paper cites.
Eli5: Long form question answering
Angela Fan, Yacine Jernite, Ethan Perez, David Grangier, Jason Weston, and Michael Auli. 2019 · 1907
Earlier work this paper cites.
Bleu: a method for automatic evaluation of machine translation
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002 · 2002
Earlier work this paper cites.
Tydi qa: A benchmark for information-seeking question answering in typologically diverse languages
Jonathan H. Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski, Vitaly Nikolaev, and Jennimaria Palomaki. 2020 · 2003
Earlier work this paper cites.
Measuring massive multitask language understanding
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Xiaodong Song, and Jacob Steinhardt. 2020 · 2009
Earlier work this paper cites.
Evaluating question answering evaluation
Anthony Chen, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. 2019 · 2019
Earlier work this paper cites.
XQA: A cross-lingual open-domain question answering dataset
Jiahua Liu, Yankai Lin, Zhiyuan Liu, and Maosong Sun. 2019 · 2019
Earlier work this paper cites.
On the cross-lingual transferability of monolingual representations
Mikel Artetxe, Sebastian Ruder, and Dani Yogatama. 2020 · 2020
Earlier work this paper cites.
MLQA: Evaluating cross-lingual extractive question answering
Patrick Lewis, Barlas Oguz, Ruty Rinott, Sebastian Riedel, and Holger Schwenk. 2020 · 2020
Earlier work this paper cites.
Bleurt: Learning robust metrics for text generation
Thibault Sellam, Dipanjan Das, and Ankur P. Parikh. 2020 · 2020
Earlier work this paper cites.
Quantifying human-perceived answer utility in non-factoid question answering
Berkant Barla Cambazoglu, Valeriia Bolotova-Baranova, Falk Scholer, Mark Sanderson, Leila Tavakoli, and W. Bruce Croft. 2021 · 2021
Earlier work this paper cites.
ELI5-Category: a categorized open-domain qa dataset
Jingsong Gao, Qingren Zhou, and Rui Qiu. 2021 · 2021
Earlier work this paper cites.
Evaluating the efficacy of summarization evaluation across languages
Fajri Koto, Jey Han Lau, and Timothy Baldwin. 2021 · 2021
Earlier work this paper cites.
Hurdles to progress in long-form question answering
Kalpesh Krishna, Aurko Roy, and Mohit Iyyer. 2021 · 2021
Earlier work this paper cites.
its5q/yandex-q - datasets at hugging face
its5Q. 2022 · 2022
Earlier work this paper cites.
Label Studio: Data labeling software
Maxim Tkachenko, Mikhail Malyuk, Andrey Holmanyuk, and Nikolai Liubimov. 2020-2022 · 2022
Earlier work this paper cites.
Towards a unified multi-dimensional evaluator for text generation
Ming Zhong, Yang Liu, Da Yin, Yuning Mao, Yizhu Jiao, Peng Liu, Chenguang Zhu, Heng Ji, and Jiawei Han. 2022 · 2022
Earlier work this paper cites.
Do all languages cost the same? tokenization in the era of commercial language models
Orevaoghene Ahia, Sachin Kumar, Hila Gonen, Jungo Kasai, David R. Mortensen, Noah A. Smith, and Yulia Tsvetkov. 2023 · 2023
Earlier work this paper cites.
ordinal—Regression Models for Ordinal Data
Rune H. B. Christensen. 2023 · 2023
Earlier work this paper cites.
Gptscore: Evaluate as you desire
Jinlan Fu, See-Kiong Ng, Zhengbao Jiang, and Pengfei Liu. 2023 · 2023
Earlier work this paper cites.
Hallucinations in large multilingual translation models
Nuno M. Guerreiro, Duarte M. Alves, Jonas Waldendorf, Barry Haddow, Alexandra Birch, Pierre Colombo, and André Martins. 2023 · 2023
Earlier work this paper cites.
Are large language model-based evaluators the solution to scaling up multilingual evaluation?
Rishav Hada, Varun Gumma, Adrian de Wynter, Harshita Diddee, Mohamed Ahmed, Monojit Choudhury, Kalika Bali, and Sunayana Sitaram. 2023 · 2023
Earlier work this paper cites.
Is chatgpt a good translator? yes with gpt-4 as the engine
Wenxiang Jiao, Wenxuan Wang, Jen tse Huang, Xing Wang, Shuming Shi, and Zhaopeng Tu. 2023 · 2023
Cited alongside, same era.
Amr Keleg and Walid Magdy. 2023 · 2023
Cited alongside, same era.
Viet Dac Lai, Chien Van Nguyen, Nghia Trung Ngo, Thuat Nguyen, Franck Dernoncourt, Ryan A. Rossi, and Thien Huu Nguyen. 2023 · 2023
Cited alongside, same era.
Factscore: Fine-grained atomic evaluation of factual precision in long form text generation
Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen tau Yih, Pang Wei Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023 · 2023
Cited alongside, same era.
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
Gemini Team. 2024 · 2024
Closest in time.
Themis: A reference-free nlg evaluation language model with flexibility and interpretability
Xinyu Hu, Li Lin, Mingqi Gao, Xunjian Yin, and Xiaojun Wan. 2024 · 2024
Closest in time.
Tigerscore: Towards building explainable metric for all text generation tasks
Dongfu Jiang, Yishan Li, Ge Zhang, Wenhao Huang, Bill Yuchen Lin, and Wenhu Chen. 2023 · 2024
Closest in time.
Comparing hallucination detection metrics for multilingual generation
Haoqiang Kang, Terra Blevins, and Luke Zettlemoyer. 2024 · 2024
Closest in time.
Click: A benchmark dataset of cultural and linguistic intelligence in korean
Eunsu Kim, Juyoung Suk, Philhoon Oh, Haneul Yoo, James Thorne, and Alice Oh. 2024 · 2024
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Extracting cultural commonsense knowledge at scale
Tuan-Phong Nguyen, Simon Razniewski, Aparna S. Varde, and Gerhard Weikum. 2022 · 2023
Cited alongside, same era.
FORK: A bite-sized test set for probing culinary cultural biases in commonsense reasoning models
Shramay Palta and Rachel Rudinger. 2023 · 2023
Cited alongside, same era.
Detecting and mitigating hallucinations in multilingual summarisation
Yifu Qiu, Yftah Ziser, Anna Korhonen, E. Ponti, and Shay B. Cohen. 2023 · 2023
Cited alongside, same era.
wangrui6/zhihu-kol - datasets at hugging face
Ray Wang. 2023 · 2023
Cited alongside, same era.
Copal-id: Indonesian language reasoning with local culture and nuances
Haryo Akbarianto Wibowo, Erland Hilman Fuadi, Made Nindyatama Nityasya, Radityo Eko Prasojo, and Alham Fikri Aji. 2023 · 2023
Cited alongside, same era.
A critical evaluation of evaluations for long-form question answering
Fangyuan Xu, Yixiao Song, Mohit Iyyer, and Eunsol Choi. 2023 · 2023
Cited alongside, same era.
xdial-eval: A multilingual open-domain dialogue evaluation benchmark
Chen Zhang, L. F. D’Haro, Chengguang Tang, Ke Shi, Guohua Tang, and Haizhou Li. 2023 · 2023
Cited alongside, same era.
Judging llm-as-a-judge with mt-bench and chatbot arena
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023 · 2023
Cited alongside, same era.
Closest in time.
Huihan Li, Liwei Jiang, Nouha Dziri, Xiang Ren, and Yejin Choi. 2024 · 2024
Closest in time.
Culturally aware and adapted nlp: A taxonomy and a survey of the state of the art
Chen Cecilia Liu, Iryna Gurevych, and Anna Korhonen. 2024 · 2024
Closest in time.
Cheaper, Better, Faster, Stronger
Mistral AI. 2024 · 2024
Closest in time.
Having beer after prayer? measuring cultural bias in large language models
Tarek Naous, Michael Ryan, Alan Ritter, and Wei Xu. 2024 · 2024
Closest in time.
Model release blog: GPT-4o
OpenAI. 2024b · 2024
Closest in time.
Multi-fact: Assessing factuality of multilingual llms using factscore
Sheikh Shafayat, Eunsu Kim, Juhyun Oh, and Alice Oh. 2024 · 2024
Closest in time.
Understanding the capabilities and limitations of large language models for cultural commonsense
Siqi Shen, Lajanugen Logeswaran, Moontae Lee, Honglak Lee, Soujanya Poria, and Rada Mihalcea. 2024 · 2024
Closest in time.
Global mmlu: Understanding and addressing cultural and linguistic biases in multilingual evaluation
Shivalika Singh, Angelika Romanou, Clémentine Fourrier, David I. Adelani, Jian Gang Ngui, Daniel Vila-Suero, Peerat Limkonchotiwat, Kelly Marchisio, Wei Qi Leong, Yosephine Susanto, Raymond Ng, Shayne Longpre, Wei-Yin Ko, Madeline Smith, Antoine Bosselut, Alice Oh, Andre F. T. Martins, Leshem Choshen, Daphne Ippolito, Enzo Ferrante, Marzieh Fadaee, Beyza Ermis, and Sara Hooker. 2024 · 2024
Closest in time.
Veriscore: Evaluating the factuality of verifiable claims in long-form text generation
Yixiao Song, Yekyung Kim, and Mohit Iyyer. 2024 · 2024
Closest in time.
Cultural bias and cultural alignment of large language models
Yan Tao, Olga Viberg, Ryan S. Baker, and Rene F. Kizilcec. 2024 · 2024
Closest in time.
All languages matter: Evaluating lmms on culturally diverse 100 languages
Ashmal Vayani, Dinura Dissanayake, Hasindri Watawana, Noor Ahsan, Nevasini Sasikumar, Omkar Thawakar, Henok Biadglign Ademtew, Yahya Hmaiti, Amandeep Kumar, Kartik Kuckreja, Mykola Maslych, Wafa Al Ghallabi, Mihail Mihaylov, Chao Qin, Abdelrahman M Shaker, Mike Zhang, Mahardika Krisna Ihsani, Amiel Esplana, Monil Gokani, Shachar Mirkin, Harsh Singh, Ashay Srivastava, Endre Hamerlik, Fathinah Asma Izzati, Fadillah Adamsyah Maani, Sebastian Cavada, Jenny Chim, Rohit Gupta, Sanjay Manjunath, Kamila Zhumakhanova, Feno Heriniaina Rabevohitra, Azril Amirudin, Muhammad Ridzuan, Daniya Kareem, Ketan More, Kunyang Li, Pramesh Shakya, Muhammad Saad, Amirpouya Ghasemaghaei, Amirbek Djanibekov, Dilshod Azizov, Branislava Jankovic, Naman Bhatia, Alvaro Cabrera, Johan Obando-Ceron, Olympiah Otieno, Fabian Farestam, Muztoba Rabbani, Sanoojan Baliah, Santosh Sanjeev, Abduragim Shtanchaev, Maheen Fatima, Thao Nguyen, Amrin Kareem, Toluwani Aremu, Nathan Xavier, Amit Bhatkal, Hawau Toyin, Aman Chadha, Hisham Cholakkal, Rao Muhammad Anwer, Michael Felsberg, Jorma Laaksonen, Thamar Solorio, Monojit Choudhury, Ivan Laptev, Mubarak Shah, Salman Khan, and Fahad Khan. 2024 · 2024
Closest in time.
Jianhao Yan, Pingchuan Yan, Yulong Chen, Judy Li, Xianchao Zhu, and Yue Zhang. 2024 · 2024
Closest in time.
Mm-vet: Evaluating large multimodal models for integrated capabilities
Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. 2024 · 2024
Closest in time.
Does mapo tofu contain coffee? probing llms for food-related cultural knowledge
Li Zhou, Taelin Karidi, Nicolas Garneau, Yong Cao, Wanlong Liu, Wenyu Chen, and Daniel Hershcovich. 2024 · 2024
Closest in time.
Blend: A benchmark for llms on everyday knowledge in diverse cultures and languages
Junho Myung, Nayeon Lee, Yi Zhou, Jiho Jin, Rifki Afina Putri, Dimosthenis Antypas, Hsuvas Borkakoty, Eunsu Kim, Carla Perez-Almendros, Abinew Ali Ayele, Víctor Gutiérrez-Basulto, Yazmín Ibáñez-García, Hwaran Lee, Shamsuddeen Hassan Muhammad, Kiwoong Park, Anar Sabuhi Rzayev, Nina White, Seid Muhie Yimam, Mohammad Taher Pilehvar, Nedjma Ousidhoum, Jose Camacho-Collados, and Alice Oh. 2025 · 2025
Closest in time.
GeoMLAMA: Geo-diverse commonsense probing on multilingual pre-trained language models
Da Yin, Hritik Bansal, Masoud Monajatipoor, Liunian Harold Li, and Kai-Wei Chang. 2022 · 2055
Closest in time.