Fetching the paper…
Reading the bibliography…
Over the past two years, EleutherAI has established itself as a radically novel initiative aimed at both promoting open-source research and conducting research in a transparent, openly accessible and collaborative manner.
“The Cathedral and the Bazaar”
Eric. Raymond and Tim O’Reilly · 1999
Earlier work this paper cites.
“The Pile: An 800GB Dataset of Diverse Text for Language Modeling”
Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser and Connor Leahy · 2020
Earlier work this paper cites.
“GPT-NeoX: Large Scale Autoregressive Language Modeling in PyTorch”, 2021
Alex Andonian, Quentin Anthony, Stella Biderman, Sid Black, Preetham Gali, Leo Gao, Eric Hallahan, Josh Levy-Kramer, Connor Leahy, Lucas Nestler, Kip Parker, Michael Pieler, Shivanshu Purohit, Tri Songz, Wang Phil and Samuel Weinbach · 2021
Earlier work this paper cites.
“Gee Stella, #EleutherAI sure hypes rotary embeddings a lot. Are you sure that they’re that good?”
Stella Biderman · 2021
Earlier work this paper cites.
“GPT-Neo: Large Scale Autoregressive Language Modeling with Mesh-Tensorflow”
Sid Black, Leo Gao, Phil Wang, Connor Leahy and Stella Biderman · 2021
Cited alongside, same era.
“A framework for few-shot language model evaluation”
Leo Gao, Jonathan Tow, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Kyle McDonell, Niklas Muennighoff, Jason Phang, Laria Reynolds, Eric Tang, Anish Thite, Ben Wang, Kevin Wang and Andy Zou · 2021
Cited alongside, same era.
“GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model”, 2021
Ben Wang and Aran Komatsuzaki · 2021
Cited alongside, same era.
“Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback”
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Shauna Kravec, Liane Lovitt, Neel Nanda, Catherine Olsson, Dario Amodei, Tom Brown, Jack Clark, Sam McCandlish, Chris Olah, Ben Mann and Jared Kaplan · 2022
Cited alongside, same era.
“GPT-NeoX-20B: An Open-Source Autoregressive Language Model”
Sidney Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang, Michael Pieler, Usvsn Prashanth, Shivanshu Purohit, Laria Reynolds, Jonathan Tow, Ben Wang and Samuel Weinbach · 2022
Closest in time.
“VQGAN-CLIP: Open domain image generation and editing with natural language guidance”
Katherine Crowson, Stella Biderman, Daniel Kornis, Dashiell Stander, Eric Hallahan, Louis Castricato and Edward Raff · 2022
Closest in time.
“OSLO: Open Source framework for Large-scale transformer Optimization”, 2022
Hyunwoong Ko, Soohwan Kim and Kyubyong Park · 2022
Closest in time.
“Training language models to follow instructions with human feedback”
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike and Ryan Lowe · 2022
Closest in time.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…