Fetching the paper…
Reading the bibliography…
Public release of the weights of pretrained foundation models, otherwise known as downloadable access \citep{solaiman_gradient_2023}, enables fine-tuning without the prohibitive expense of pretraining.
Parameter-Efficient Transfer Learning for NLP, June 2019
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly · 1902
Earlier work this paper cites.
Aviv Ovadya and Jess Whittlestone · 1907
Earlier work this paper cites.
Release Strategies and the Social Impacts of Language Models, November 2019
Irene Solaiman, Miles Brundage, Jack Clark, Amanda Askell, Ariel Herbert-Voss, Jeff Wu, Alec Radford, Gretchen Krueger, Jong Wook Kim, Sarah Kreps, Miles McCain, Alex Newhouse, Jason Blazakis, Kris McGuffie, and Jasmine Wang · 1908
Earlier work this paper cites.
The tension between openness and prudence in AI research, January 2020
Jess Whittlestone and Aviv Ovadya · 1910
Earlier work this paper cites.
Machine Unlearning, December 2020
Lucas Bourtoule, Varun Chandrasekaran, Christopher A. Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot · 1912
Earlier work this paper cites.
Aum Shinrikyo and weapons of mass destruction: A case study
Neal A Clinehens · 2000
Earlier work this paper cites.
Parameter Space Factorization for Zero-Shot Learning across Tasks and Languages, November 2020
Edoardo M. Ponti, Ivan Vulić, Ryan Cotterell, Marinela Parovic, Roi Reichart, and Anna Korhonen · 2001
Earlier work this paper cites.
Training Large Neural Networks with Constant Memory using a New Execution Algorithm, June 2020
Bharadwaj Pudipeddi, Maral Mesmakhosroshahi, Jinwen Xi, and Sujeeth Bharadwaj · 2002
Earlier work this paper cites.
A Model for When Disclosure Helps Security: What Is Different About Computer and Network Security?, 2004
Peter Swire · 2004
Earlier work this paper cites.
Open and Closed Systems are Equivalent (that is, in an ideal world)
Ross Anderson · 2007
Earlier work this paper cites.
The export of cryptography in the 20th and the 21st centuries
Whitfield Diffie and Susan Landau · 2007
Earlier work this paper cites.
Federated Learning for Internet of Things: Recent Advances, Taxonomy, and Open Challenges, June 2021
Latif U. Khan, Walid Saad, Zhu Han, Ekram Hossain, and Choong Seon Hong · 2009
Earlier work this paper cites.
Parameter-Efficient Transfer Learning with Diff Pruning, June 2021
Demi Guo, Alexander M. Rush, and Yoon Kim · 2012
Earlier work this paper cites.
8-Bit Approximations for Parallelism in Deep Learning, February 2016
Tim Dettmers · 2016
Earlier work this paper cites.
Deep Reinforcement Learning from Human Preferences
Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei · 2017
Earlier work this paper cites.
Learning multiple visual domains with residual adapters, November 2017
Sylvestre-Alvise Rebuffi, Hakan Bilen, and Andrea Vedaldi · 2017
Earlier work this paper cites.
How does the offense-defense balance scale?
Ben Garfinkel and Allan Dafoe · 2019
Earlier work this paper cites.
Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims, April 2020
Miles Brundage, Shahar Avin, Jasmine Wang, Haydn Belfield, Gretchen Krueger, Gillian Hadfield, Heidy Khlaaf, Jingying Yang, Helen Toner, Ruth Fong, Tegan Maharaj, Pang Wei Koh, Sara Hooker, Jade Leung, Andrew Trask, Emma Bluemke, Jonathan Lebensold, Cullen O’Keefe, Mark Koren, Théo Ryffel, J. B. Rubinovitz, Tamay Besiroglu, Federica Carugati, Jack Clark, Peter Eckersley, Sarah de Haas, Maritza Johnson, Ben Laurie, Alex Ingerman, Igor Krawczuk, Amanda Askell, Rosario Cammarota, Andrew Lohn, David Krueger, Charlotte Stix, Peter Henderson, Logan Graham, Carina Prunkl, Bianca Martin, Elizabeth Seger, Noa Zilberman, Seán Ó hÉigeartaigh, Frens Kroeger, Girish Sastry, Rebecca Kagan, Adrian Weller, Brian Tse, Elizabeth Barnes, Allan Dafoe, Paul Scharre, Ariel Herbert-Voss, Martijn Rasser, Shagun Sodhani, Carrick Flynn, Thomas Krendl Gilbert, Lisa Dyer, Saif Khan, Yoshua Bengio, and Markus Anderljung · 2020
Earlier work this paper cites.
The Offense-Defense Balance of Scientific Knowledge: Does Publishing AI Research Reduce Misuse?
Toby Shevlane and Allan Dafoe · 2020
Earlier work this paper cites.
LoRA: Low-Rank Adaptation of Large Language Models, October 2021
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen · 2021
Earlier work this paper cites.
The Power of Scale for Parameter-Efficient Prompt Tuning, September 2021
Brian Lester, Rami Al-Rfou, and Noah Constant · 2021
Earlier work this paper cites.
Prefix-Tuning: Optimizing Continuous Prompts for Generation, January 2021
Xiang Lisa Li and Percy Liang · 2021
Earlier work this paper cites.
Learning How to Ask: Querying LMs with Mixtures of Soft Prompts, April 2021
Guanghui Qin and Jason Eisner · 2021
Earlier work this paper cites.
ZeRO-Offload: Democratizing Billion-Scale Model Training, January 2021
Jie Ren, Samyam Rajbhandari, Reza Yazdani Aminabadi, Olatunji Ruwase, Shuangyan Yang, Minjia Zhang, Dong Li, and Yuxiong He · 2021
Earlier work this paper cites.
Beyond "Release" vs. "Not Release", October 2021
Girish Sastry · 2021
Earlier work this paper cites.
Runxin Xu, Fuli Luo, Zhiyuan Zhang, Chuanqi Tan, Baobao Chang, Songfang Huang, and Fei Huang · 2021
Earlier work this paper cites.
Composable Sparse Fine-Tuning for Cross-Lingual Transfer
Alan Ansell, Edoardo Ponti, Anna Korhonen, and Ivan Vulić · 2022
Cited alongside, same era.
Constitutional AI: Harmlessness from AI Feedback, December 2022
Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, Carol Chen, Catherine Olsson, Christopher Olah, Danny Hernandez, Dawn Drain, Deep Ganguli, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jamie Kerr, Jared Mueller, Jeffrey Ladish, Joshua Landau, Kamal Ndousse, Kamile Lukosuite, Liane Lovitt, Michael Sellitto, Nelson Elhage, Nicholas Schiefer, Noemi Mercado, Nova DasSarma, Robert Lasenby, Robin Larson, Sam Ringer, Scott Johnston, Shauna Kravec, Sheer El Showk, Stanislav Fort, Tamera Lanham, Timothy Telleen-Lawton, Tom Conerly, Tom Henighan, Tristan Hume, Samuel R. Bowman, Zac Hatfield-Dodds, Ben Mann, Dario Amodei, Nicholas Joseph, Sam McCandlish, Tom Brown, and Jared Kaplan · 2022
Cited alongside, same era.
Elad Ben-Zaken, Shauli Ravfogel, and Yoav Goldberg · 2022
Cited alongside, same era.
LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale, November 2022
BadLlama: cheaply removing safety fine-tuning from Llama 2-Chat 13B, October 2023
Pranav Gade, Simon Lermen, Charlie Rogers-Smith, and Jeffrey Ladish · 2023
Closest in time.
Anjali Gopal, Nathan Helm-Burger, Lennart Justen, Emily H. Soice, Tiffany Tzeng, Geetha Jeyapragasan, Simon Grimm, Benjamin Mueller, and Kevin M. Esvelt · 2023
Closest in time.
The False Promise of Imitating Proprietary LLMs, May 2023
Arnav Gudibande, Eric Wallace, Charlie Snell, Xinyang Geng, Hao Liu, Pieter Abbeel, Sergey Levine, and Dawn Song · 2023
Closest in time.
Scaling Expert Language Models with Unsupervised Domain Discovery, March 2023
Suchin Gururangan, Margaret Li, Mike Lewis, Weijia Shi, Tim Althoff, Noah A. Smith, and Luke Zettlemoyer · 2023
Closest in time.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer · 2022
Cited alongside, same era.
X-Risk Analysis for AI Research, September 2022
Dan Hendrycks and Mantas Mazeika · 2022
Cited alongside, same era.
Large Language Models Can Self-Improve, October 2022
Jiaxin Huang, Shixiang Shane Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han · 2022
Cited alongside, same era.
Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models, August 2022
Margaret Li, Suchin Gururangan, Tim Dettmers, Mike Lewis, Tim Althoff, Noah A. Smith, and Luke Zettlemoyer · 2022
Cited alongside, same era.
The Time Is Now to Develop Community Norms for the Release of Foundation Models, May 2022
Percy Liang, Rishi Bommasani, Kathleen Creel, and Rob Reich · 2022
Cited alongside, same era.
Discovering Language Model Behaviors with Model-Written Evaluations, December 2022
Ethan Perez, Sam Ringer, Kamilė Lukošiūtė, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, Andy Jones, Anna Chen, Ben Mann, Brian Israel, Bryan Seethor, Cameron McKinnon, Christopher Olah, Da Yan, Daniela Amodei, Dario Amodei, Dawn Drain, Dustin Li, Eli Tran-Johnson, Guro Khundadze, Jackson Kernion, James Landis, Jamie Kerr, Jared Mueller, Jeeyoon Hyun, Joshua Landau, Kamal Ndousse, Landon Goldberg, Liane Lovitt, Martin Lucas, Michael Sellitto, Miranda Zhang, Neerav Kingsland, Nelson Elhage, Nicholas Joseph, Noemí Mercado, Nova DasSarma, Oliver Rausch, Robin Larson, Sam McCandlish, Scott Johnston, Shauna Kravec, Sheer El Showk, Tamera Lanham, Timothy Telleen-Lawton, Tom Brown, Tom Henighan, Tristan Hume, Yuntao Bai, Zac Hatfield-Dodds, Jack Clark, Samuel R. Bowman, Amanda Askell, Roger Grosse, Danny Hernandez, Deep Ganguli, Evan Hubinger, Nicholas Schiefer, and Jared Kaplan · 2022
Cited alongside, same era.
EleutherAI: Going Beyond "Open Science" to "Science in the Open", October 2022
Jason Phang, Herbie Bradley, Leo Gao, Louis Castricato, and Stella Biderman · 2022
Cited alongside, same era.
Structured access: an emerging paradigm for safe AI deployment, April 2022
Toby Shevlane · 2022
Cited alongside, same era.
Transcending Scaling Laws with 0.1% Extra Compute, November 2022
Yi Tay, Jason Wei, Hyung Won Chung, Vinh Q. Tran, David R. So, Siamak Shakeri, Xavier Garcia, Huaixiu Steven Zheng, Jinfeng Rao, Aakanksha Chowdhery, Denny Zhou, Donald Metzler, Slav Petrov, Neil Houlsby, Quoc V. Le, and Mostafa Dehghani · 2022
Cited alongside, same era.
Julian Hazell · 2023
Closest in time.
Self-Destructing Models: Increasing the Costs of Harmful Dual Uses of Foundation Models, August 2023
Peter Henderson, Eric Mitchell, Christopher D. Manning, Dan Jurafsky, and Chelsea Finn · 2023
Closest in time.
AI Safety and the Age of Dislightenment, July 2023
Jeremy Howard · 2023
Closest in time.
Exploring the Benefits of Training Expert Language Models over Instruction Tuning, February 2023
Joel Jang, Seungone Kim, Seonghyeon Ye, Doyoung Kim, Lajanugen Logeswaran, Moontae Lee, Kyungjae Lee, and Minjoon Seo · 2023
Closest in time.
OpenAI’s CEO Says the Age of Giant AI Models Is Already Over
Will Knight · 2023
Closest in time.
Textbooks Are All You Need II: phi-1.5 technical report, September 2023
Yuanzhi Li, Sébastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, and Yin Tat Lee · 2023
Closest in time.
Fine-Tuning Language Models with Just Forward Passes, May 2023
Sadhika Malladi, Tianyu Gao, Eshaan Nichani, Alex Damian, Jason D. Lee, Danqi Chen, and Sanjeev Arora · 2023
Closest in time.
Towards a Grand Unified Threat Model of Biotechnology, September 2023
Michael Montague · 2023
Closest in time.
The Importance of Open Models for Transparency, Competition, and Resilience in AI: Considerations for AI Oversight in the United States, May 2023
Emad Mostaque · 2023
Closest in time.
Orca: Progressive Learning from Complex Explanation Traces of GPT-4, June 2023
Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar, Sahaj Agarwal, Hamid Palangi, and Ahmed Awadallah · 2023
Closest in time.
GPT-4 Technical Report
OpenAI · 2023
Closest in time.
The AI Brick Wall – A Practical Limit For Scaling Dense Transformer Models, and How GPT 4 Will Break Past It, January 2023
Dylan Patel · 2023
Closest in time.
Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson · 2023
Closest in time.
SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient, June 2023
Max Ryabinin, Tim Dettmers, Michael Diskin, and Alexander Borzunov · 2023
Closest in time.
Jonas B. Sandbrink · 2023
Closest in time.
Model evaluation for extreme risks, September 2023
Toby Shevlane, Sebastian Farquhar, Ben Garfinkel, Mary Phuong, Jess Whittlestone, Jade Leung, Daniel Kokotajlo, Nahema Marchal, Markus Anderljung, Noam Kolt, Lewis Ho, Divya Siddarth, Shahar Avin, Will Hawkins, Been Kim, Iason Gabriel, Vijay Bolina, Jack Clark, Yoshua Bengio, Paul Christiano, and Allan Dafoe · 2023
Closest in time.
The Curse of Recursion: Training on Generated Data Makes Models Forget, May 2023
Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot, and Ross Anderson · 2023
Closest in time.
Can large language models democratize access to dual-use biotechnology?, June 2023
Emily H. Soice, Rafael Rocha, Kimberlee Cordova, Michael Specter, and Kevin M. Esvelt · 2023
Closest in time.
The Gradient of Generative AI Release: Methods and Considerations, February 2023
Irene Solaiman · 2023
Closest in time.
Alpaca: A Strong, Replicable Instruction-Following Model, March 2023
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori Hashimoto · 2023
Closest in time.
Open (for Business): Big Tech, Concentrated Power, and the Political Economy of Open AI, August 2023
David Gray Widder, Meredith Whittaker, and Sarah Myers West · 2023
Closest in time.
Decentralized Training of Foundation Models in Heterogeneous Environments, June 2023
Binhang Yuan, Yongjun He, Jared Quincy Davis, Tianyi Zhang, Tri Dao, Beidi Chen, Percy Liang, Christopher Re, and Ce Zhang · 2023
Closest in time.