Fundamental Safety-Capability Trade-offs in Fine-tuning Large Language Models

Rasley, J., Rajbhandari, S., Ruwase, O., He, Y.: Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 3505–3506 (2020)

2020

Earlier work this paper cites.

Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al

2022

Earlier work this paper cites.

Ding, N., Qin, Y., Yang, G., Wei, F., Yang, Z., Su, Y., Hu, S., Chen, Y., Chan, C.-M., Chen, W., et al

2023

Earlier work this paper cites.

Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al.: Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 (2023)

Original

2023

Earlier work this paper cites.

Lian, W., Goodson, B., Pentland, E., Cook, A., Vong, C., ”Teknium”: OpenOrca: An Open Dataset of GPT Augmented FLAN Reasoning Traces. HuggingFace (2023)

2023

Earlier work this paper cites.

Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al.: GPT-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

Original

2023

Earlier work this paper cites.

Taori, R., Gulrajani, I., Zhang, T., Dubois, Y., Li, X., Guestrin, C., Liang, P., Hashimoto, T.B.: Stanford Alpaca: An Instruction-following LLaMA model. GitHub (2023)

2023

Earlier work this paper cites.

Muennighoff, N., Liu, Q., Zebaze, A., Zheng, Q., Hui, B., Zhuo, T.Y., Singh, S., Tang, X., Werra, L.V., Longpre, S.: OctoPack: Instruction tuning code large language models. In: NeurIPS 2023 Workshop on Instruction Tuning and Instruction Following (2023)

2023

Cited alongside, same era.

Lee, A.N., Hunter, C.J., Ruiz, N.: Platypus: Quick, cheap, and powerful refinement of LLMs. In: NeurIPS 2023 Workshop on Instruction Tuning and Instruction Following (2023)

2023

Cited alongside, same era.

Qi, X., Zeng, Y., Xie, T., Chen, P.-Y., Jia, R., Mittal, P., Henderson, P.: Fine-tuning aligned language models compromises safety, even when users do not intend to! In: International Conference on Learning Representations (2024)

2024

Cited alongside, same era.

Huang, T., Hu, S., Ilhan, F., Tekin, S.F., Liu, L.: Harmful fine-tuning attacks and defenses for large language models: A survey. arXiv preprint arXiv:2409.18169 (2024)

Original

2024

Cited alongside, same era.

Hsu, C.-Y., Tsai, Y.-L., Lin, C.-H., Chen, P.-Y., Yu, C.-M., Huang, C.-Y.: Safe LoRA: The silver lining of reducing safety risks when finetuning large language models. In: Neural Information Processing Systems (2024)

2024

Later among the works it cites.

Dao, T.: FlashAttention-2: Faster attention with better parallelism and work partitioning. In: International Conference on Learning Representations (2024)

2024

Later among the works it cites.

Chen, P.-Y.: Computational safety for generative AI: A signal processing perspective. arXiv preprint arXiv:2502.12445 (2025)

Original

2025

Closest in time.

Choudhury, M., Elyoseph, Z., Fast, N.J., Ong, D.C., Nsoesie, E.O., Pavlick, E.: The promise and pitfalls of generative ai. Nature Reviews Psychology, 1–6 (2025)

2025

Closest in time.