Fetching the paper…
Reading the bibliography…
Post-training of language models, either through reinforcement learning, preference optimization or supervised finetuning, tends to sharpen the output probability distribution and reduce the diversity of generated responses.
A diversity-promoting objective function for neural conversation models
Li, J., Galley, M., Brockett, C., Gao, J., and Dolan, B · 2015
Earlier work this paper cites.
Quality diversity: A new frontier for evolutionary computation
Pugh, J. K., Soros, L. B., and Stanley, K. O · 2016
Earlier work this paper cites.
Deep reinforcement learning from human preferences
Christiano, P. F., Leike, J., Brown, T., Martic, M., Legg, S., and Amodei, D · 2017
Earlier work this paper cites.
Proximal policy optimization algorithms, 2017
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O · 2017
Earlier work this paper cites.
Hierarchical neural story generation
Fan, A., Lewis, M., and Dauphin, Y · 2018
Earlier work this paper cites.
The curious case of neural text degeneration
Holtzman, A., Buys, J., Du, L., Forbes, M., and Choi, Y · 2019
Earlier work this paper cites.
Spoc: Search-based pseudocode to code
Kulal, S., Pasupat, P., Chandra, K., Lee, M., Padon, O., Aiken, A., and Liang, P. S · 2019
Earlier work this paper cites.
Fine-tuning language models from human preferences
Ziegler, D. M., Stiennon, N., Wu, J., Brown, T. B., Radford, A., Amodei, D., Christiano, P., and Irving, G · 2019
Earlier work this paper cites.
Language models are few-shot learners
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al · 2020
Earlier work this paper cites.
Evaluating the evaluation of diversity in natural language generation
Tevet, G. and Berant, J · 2020
Earlier work this paper cites.
Trading off diversity and quality in natural language generation
Zhang, H., Duckworth, D., Ippolito, D., and Neelakantan, A · 2020
Earlier work this paper cites.
Evaluating large language models trained on code
Chen, M., Tworek, J., Jun, H., Yuan, Q., de Oliveira Pinto, H. P., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., Ray, A., Puri, R., Krueger, G., Petrov, M., Khlaaf, H., Sastry, G., Mishkin, P., Chan, B., Gray, S., Ryder, N., Pavlov, M., Power, A., Kaiser, L., Bavarian, M., Winter, C., Tillet, P., Such, F. P., Cummings, D., Plappert, M., Chantzis, F., Barnes, E., Herbert-Voss, A., Guss, W. H., Nichol, A., Paino, A., Tezak, N., Tang, J., Babuschkin, I., Balaji, S., Jain, S., Saunders, W., Hesse, C., Carr, A. N., Leike, J., Achiam, J., Misra, V., Morikawa, E., Radford, A., Knight, M., Brundage, M., Murati, M., Mayer, K., Welinder, P., McGrew, B., Amodei, D., McCandlish, S., Sutskever, I., and Zaremba, W · 2021
Earlier work this paper cites.
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al · 2022
Earlier work this paper cites.
Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al · 2023
Earlier work this paper cites.
fairseq2, 2023
Balioglu, C · 2023
Earlier work this paper cites.
Quality-diversity through ai feedback
Bradley, H., Dai, A., Teufel, H., Zhang, J., Oostermeijer, K., Bellagente, M., Clune, J., Stanley, K., Schott, G., and Lehman, J · 2023
Earlier work this paper cites.
Chung, J. J. Y., Kamar, E., and Amershi, S · 2023
Earlier work this paper cites.
Tinystories: How small can language models be and still speak coherent english?
Eldan, R. and Li, Y · 2023
Earlier work this paper cites.
Gemini: a family of highly capable multimodal models
Gemini, Anil, R., Borgeaud, S., Alayrac, J.-B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A. M., Hauth, A., Millican, K., et al · 2023
Earlier work this paper cites.
Efficient memory management for large language model serving with pagedattention
Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J. E., Zhang, H., and Stoica, I · 2023
Earlier work this paper cites.
Alpacaeval: An automatic evaluator of instruction-following models
Li, X., Zhang, T., Dubois, Y., Taori, R., Gulrajani, I., Guestrin, C., Liang, P., and Hashimoto, T. B · 2023
Earlier work this paper cites.
Large language models generate functional protein sequences across diverse families
Madani, A., Krause, B., Greene, E. R., Subramanian, S., Mohr, B. P., Holton, J. M., Olmos, J. L., Xiong, C., Sun, Z. Z., Socher, R., Fraser, J. S., and Naik, N · 2023
Cited alongside, same era.
Whose opinions do language models reflect?
Santurkar, S., Durmus, E., Ladhak, F., Lee, C., Liang, P., and Hashimoto, T · 2023
Cited alongside, same era.
Beyond human data: Scaling self-training for problem-solving with language models
Singh, A., Co-Reyes, J. D., Agarwal, R., Anand, A., Patil, P., Garcia, X., Liu, P. J., Harrison, J., Lee, J., Xu, K., et al · 2023
Cited alongside, same era.
Llama 2: Open foundation and fine-tuned chat models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al · 2023
Cited alongside, same era.
Beyond reverse kl: Generalizing direct preference optimization with diverse divergence constraints
Marco, G., Rello, L., and Gonzalo, J · 2024
Later among the works it cites.
Simpo: Simple preference optimization with a reference-free reward
Meng, Y., Xia, M., and Chen, D · 2024
Later among the works it cites.
Murthy, S. K., Ullman, T., and Hu, J · 2024
Later among the works it cites.
Turning up the heat: Min-p sampling for creative and coherent llm outputs
Nguyen, M., Baker, A., Neo, C., Roush, A., Kirsch, A., and Shwartz-Ziv, R · 2024
Later among the works it cites.
Asynchronous rlhf: Faster and more efficient off-policy rl for language models, 2024
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Wang, C., Jiang, Y., Yang, C., Liu, H., and Chen, Y · 2023
Cited alongside, same era.
Some things are more cringe than others: Preference optimization with the pairwise cringe loss
Xu, J., Lee, A., Sukhbaatar, S., and Weston, J · 2023
Cited alongside, same era.
Large language model as attributed training data generator: A tale of diversity and bias
Yu, Y., Zhuang, Y., Zhang, J., Meng, Y., Ratner, A. J., Krishna, R., Shen, J., and Zhang, C · 2023
Cited alongside, same era.
Exploring precision and recall to assess the quality and diversity of llms
Bronnec, F. L., Verine, A., Negrevergne, B., Chevaleyre, Y., and Allauzen, A · 2024
Cited alongside, same era.
Self-play fine-tuning converts weak language models to strong language models
Chen, Z., Deng, Y., Yuan, H., Ji, K., and Gu, Q · 2024
Cited alongside, same era.
Diversity-rewarded cfg distillation
Cideron, G., Agostinelli, A., Ferret, J., Girgin, S., Elie, R., Bachem, O., Perrin, S., and Ramé, A · 2024
Cited alongside, same era.
Adaptive decoding via latent preference optimization
Dhuliawala, S., Kulikov, I., Yu, P., Celikyilmaz, A., Weston, J., Sukhbaatar, S., and Lanchantin, J · 2024
Cited alongside, same era.
Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al · 2024
Cited alongside, same era.
Noukhovitch, M., Huang, S., Xhonneux, S., Hosseini, A., Agarwal, R., and Courville, A · 2024
Later among the works it cites.
West-of-n: Synthetic preference generation for improved reward modeling
Pace, A., Mallinson, J., Malmi, E., Krause, S., and Severyn, A · 2024
Later among the works it cites.
Does writing with language models reduce content diversity?
Padmakumar, V. and He, H · 2024
Later among the works it cites.
Beyond the binary: Capturing diverse preferences with reward regularization
Padmakumar, V., Jin, C., Kirk, H. R., and He, H · 2024
Later among the works it cites.
Park, R., Hsu, D. J., Roland, C. B., Korshunova, M., Tessler, C., Mannor, S., Viessmann, O., and Trentini, B · 2024
Later among the works it cites.
Online dpo: Online direct preference optimization with fast-slow chasing, 2024
Qi, B., Li, P., Li, F., Gao, J., Zhang, K., and Zhou, B · 2024
Later among the works it cites.
Direct preference optimization: Your language model is secretly a reward model
Rafailov, R., Sharma, A., Mitchell, E., Manning, C. D., Ermon, S., and Finn, C · 2024
Later among the works it cites.
Standardizing the measurement of text diversity: A tool and a comparative analysis of scores
Shaib, C., Barrow, J., Sun, J., Siu, A. F., Wallace, B. C., and Nenkova, A · 2024
Later among the works it cites.
Ai models collapse when trained on recursively generated data
Shumailov, I., Shumaylov, Z., Zhao, Y., Papernot, N., Anderson, R., and Gal, Y · 2024
Later among the works it cites.
Tree of thoughts: Deliberate problem solving with large language models
Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T., Cao, Y., and Narasimhan, K · 2024
Later among the works it cites.
Diversify and conquer: Diversity-centric data selection with iterative refinement
Yu, S., Chen, L., Ahmadian, S., and Fadaee, M · 2024
Later among the works it cites.
Self-rewarding language models
Yuan, W., Pang, R. Y., Cho, K., Sukhbaatar, S., Xu, J., and Weston, J · 2024
Later among the works it cites.
Forcing diffuse distributions out of language models
Zhang, Y., Schwarzschild, A., Carlini, N., Kolter, Z., and Ippolito, D · 2024
Later among the works it cites.
Wildchat: 1m chatgpt interaction logs in the wild
Zhao, W., Ren, X., Hessel, J., Cardie, C., Choi, Y., and Deng, Y · 2024
Later among the works it cites.
Dive: Diversified iterative self-improvement
Qin, Y., Liu, Y., and Liu, P · 2025
Closest in time.
Generating surprising and diverse ideas using chatgpt
Tachibana, M., Shimizu, T., and Tomiura, Y · 2025
Closest in time.