Aligner: One Global Token is Worth Millions of Parameters When Aligning Large Language Models

J. Tenenbaum and W. Freeman, “Separating style and content,” in Advances in Neural Information Processing Systems , M. Mozer, M. Jordan, and T. Petsche, Eds., vol. 9. MIT Press, 1996

1996

Earlier work this paper cites.

M. A. O. Vasilescu and D. Terzopoulos, “Multilinear (tensor) image synthesis, analysis, and recognition [exploratory DSP],” IEEE Signal Processing Magazine , vol. 24, no. 6, pp. 118–123, 2007

2007

Earlier work this paper cites.

G. Alain and Y. Bengio, “Understanding intermediate layers using linear classifier probes,” arXiv preprint arXiv:1610.01644 , 2016

Original

2016

Earlier work this paper cites.

P. Christiano, J. Leike, and T. Lillicrap, “Deep reinforcement learning from human preferences,” arXiv preprint arXiv:1706.03741 , 2017

Original

2017

Earlier work this paper cites.

J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347 , 2017

Original

2017

Earlier work this paper cites.

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in Neural Information Processing Systems , vol. 30, 2017

2017

Earlier work this paper cites.

M. Gazzaniga, R. Ivry, and G. Mangun, Cognitive Neuroscience: The Biology of the Mind . W.W. Norton, 2019

2019

Earlier work this paper cites.

N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. De Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly, “Parameter-efficient transfer learning for NLP,” in International Conference on Machine Learning , 2019, pp. 2790–2799

2019

Earlier work this paper cites.

K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano et al. , “Training verifiers to solve math word problems,” arXiv preprint arXiv:2110.14168 , 2021

Original

2021

Earlier work this paper cites.

D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt, “Measuring massive multitask language understanding,” Proceedings of the International Conference on Learning Representations (ICLR) , 2021

2021

Earlier work this paper cites.

B. Lester, R. Al-Rfou, and N. Constant, “The power of scale for parameter-efficient prompt tuning,” arXiv preprint arXiv:2104.08691 , 2021

Original

2021

Earlier work this paper cites.

X. L. Li and P. Liang, “Prefix-Tuning: Optimizing continuous prompts for generation,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) . Association for Computational Linguistics, Aug. 2021, pp. 4582–4597

2021

Earlier work this paper cites.

P. Liu, W. Yuan, J. Fu, Z. Jiang, H. Hayashi, and G. Neubig, “Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing,” arXiv preprint arXiv:2107.13586 , 2021

Original

2021

Cited alongside, same era.

X. Liu, K. Ji, Y. Fu, W. L. Tam, Z. Du, Z. Yang, and J. Tang, “P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks,” arXiv preprint arXiv:2110.07602 , 2021

Original

2021

Cited alongside, same era.

X. Liu, Y. Zheng, Z. Du, M. Ding, Y. Qian, Z. Yang, and J. Tang, “GPT understands, too,” arXiv preprint arXiv:2103.10385 , 2021

Original

2021

Cited alongside, same era.

L. Yao, X. Wan, J. Xiao, B. Peng, and M. Zhang, “LoRA: Low-rank adaptation of large language models,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , 2021

2021

Cited alongside, same era.

M. Mosbach, T. Pimentel, S. Ravfogel, D. Klakow, and Y. Elazar, “Few-shot fine-tuning vs. in-context learning: A fair comparison and evaluation,” arXiv preprint arXiv:2305.16938 , 2023

Original

2023

Closest in time.

OpenAI Forum, “Fine-tuning for domain knowledge and questions,” 2023, accessed: 2023-09-27. [Online]. Available: https://community.openai.com/t/finetuning-for-domain-knowledge-and-questions/24817

2023

Closest in time.

C. Qian, H. Tang, Z. Yang, H. Liang, and Y. Liu, “Can large language models empower molecular property prediction?” 2023

2023

Closest in time.

R. Rafailov, A. Sharma, E. Mitchell, S. Ermon, C. D. Manning, and C. Finn, “Direct preference optimization: Your language model is secretly a reward model,” arXiv preprint arXiv:2305.18290 , 2023

Original

2023

Closest in time.