Fetching the paper…
Reading the bibliography…
We study a class of optimization problems motivated by automating the design and update of AI systems like coding assistants, robots, and copilots.
Learning representations by back-propagating errors
David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams · 1986
Earlier work this paper cites.
An overview of evolutionary algorithms for parameter optimization
Thomas Bäck and Hans-Paul Schwefel · 1993
Earlier work this paper cites.
Response surface bandits
Josep Ginebra and Murray K Clayton · 1995
Earlier work this paper cites.
Particle swarm optimization
James Kennedy and Russell Eberhart · 1995
Earlier work this paper cites.
Artificial Intelligence in Games: A look at the smarts behind Lionhead Studio’s “Black and White” and where it can and will go in the future
James Wexler · 2002
Earlier work this paper cites.
On the Sample Complexity of Reinforcement Learning
Sham Machandranath Kakade · 2003
Earlier work this paper cites.
Enabling user-driven checkpointing strategies in reverse-mode automatic differentiation
Laurent Hascoet and Mauricio Araya-Polo · 2006
Earlier work this paper cites.
Introduction to derivative-free optimization
Andrew R Conn, Katya Scheinberg, and Luis N Vicente · 2009
Earlier work this paper cites.
Sequential model-based optimization for general algorithm configuration
Frank Hutter, Holger H Hoos, and Kevin Leyton-Brown · 2011
Earlier work this paper cites.
Structured comparative analysis of systems logs to diagnose performance problems
Karthik Nagaraj, Charles Killian, and Jennifer Neville · 2012
Earlier work this paper cites.
Statistical study of the impact of adaptive traffic signal control on traffic and transit performance
Courtney Slavin, Wei Feng, Miguel Figliozzi, and Peter Koonce · 2013
Earlier work this paper cites.
Adam: A method for stochastic optimization
Diederik P Kingma and Jimmy Ba · 2014
Earlier work this paper cites.
Partial monitoring—classification, regret bounds, and algorithms
Gábor Bartók, Dean P Foster, Dávid Pál, Alexander Rakhlin, and Csaba Szepesvári · 2014
Earlier work this paper cites.
Bayesian optimization
Peter I Frazier · 2018
Earlier work this paper cites.
Automatic differentiation in machine learning: a survey
Atilim Gunes Baydin, Barak A. Pearlmutter, Alexey Andreyevich Radul, and Jeffrey Mark Siskind · 2018
Earlier work this paper cites.
PyTorch: an imperative style, high-performance deep learning library
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al · 2019
Earlier work this paper cites.
Truncated back-propagation for bilevel optimization
Amirreza Shaban, Ching-An Cheng, Nathan Hatch, and Byron Boots · 2019
Earlier work this paper cites.
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al · 2020
Cited alongside, same era.
Online learning with continuous variations: Dynamic regret and reductions
Ching-An Cheng, Jonathan Lee, Ken Goldberg, and Byron Boots · 2020
Cited alongside, same era.
Productivity assessment of neural code completion
Albert Ziegler, Eirini Kalliamvakou, X. Alice Li, Andrew Rice, Devon Rifkin, Shawn Simister, Ganesh Sittampalam, and Edward Aftandilian · 2022
Cited alongside, same era.
Chain-of-thought prompting elicits reasoning in large language models
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H Chi, Quoc V Le, and Denny Zhou · 2022
Cited alongside, same era.
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al · 2023
Cited alongside, same era.
Joint prompt optimization of stacked LLMs using variational inference
Alessandro Sordoni, Xingdi Yuan, Marc-Alexandre Côté, Matheus Pereira, Adam Trischler, Ziang Xiao, Arian Hosseini, Friederike Niedtner, and Nicolas Le Roux · 2023
Later among the works it cites.
Using large language models for hyperparameter optimization
Michael R. Zhang, Nishkrit Desai, Juhan Bae, Jonathan Lorraine, and Jimmy Ba · 2023
Later among the works it cites.
The shift from models to compound AI systems
Matei Zaharia, Omar Khattab, Lingjiao Chen, Jared Quincy Davis, Heather Miller, Chris Potts, James Zou, Michael Carbin, Jonathan Frankle, Naveen Rao, and Ali Ghodsi · 2024
Closest in time.
Large language models as optimizers
Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V Le, Denny Zhou, and Xinyun Chen · 2024
Closest in time.
Prompts as programs: A structure-aware approach to efficient compile-time prompt optimization
Tobias Schnabel and Jennifer Neville · 2024
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Code as policies: Language model programs for embodied control
Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng · 2023
Cited alongside, same era.
Autogen: Enabling next-gen LLM applications via multi-agent conversation framework
Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Shaokun Zhang, Erkang Zhu, Beibin Li, Li Jiang, Xiaoyun Zhang, and Chi Wang · 2023
Cited alongside, same era.
Large language models are human-level prompt engineers
Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba · 2023
Cited alongside, same era.
Automatic prompt optimization with “gradient descent” and beam search
Reid Pryzant, Dan Iter, Jerry Li, Yin Lee, Chenguang Zhu, and Michael Zeng · 2023
Cited alongside, same era.
Self-taught optimizer (STOP): Recursively self-improving code generation
Eric Zelikman, Eliana Lorch, Lester Mackey, and Adam Tauman Kalai · 2023
Cited alongside, same era.
Challenging BIG-Bench tasks and whether chain-of-thought can solve them
Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc Le, Ed Chi, Denny Zhou, et al · 2023
Cited alongside, same era.
DSPy: Compiling declarative language model calls into self-improving pipelines
Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T Joshi, Hanna Moazam, et al · 2023
Cited alongside, same era.
Closest in time.
The importance of directional feedback for LLM-based optimizers
Allen Nie, Ching-An Cheng, Andrey Kolobov, and Adith Swaminathan · 2024
Closest in time.
TextGrad: Automatic “differentiation” via text
Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Zhi Huang, Carlos Guestrin, and James Zou · 2024
Closest in time.
AgentBench: Evaluating LLMs as agents
Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, Shudan Zhang, Xiang Deng, Aohan Zeng, Zhengxiao Du, Chenhui Zhang, Sheng Shen, Tianjun Zhang, Yu Su, Huan Sun, Minlie Huang, Yuxiao Dong, and Jie Tang · 2024
Closest in time.
Retroformer: Retrospective large language agents with policy gradient optimization
Weiran Yao, Shelby Heinecke, Juan Carlos Niebles, Zhiwei Liu, Yihao Feng, Le Xue, Rithesh R N, Zeyuan Chen, Jianguo Zhang, Devansh Arpit, Ran Xu, Phil L Mui, Huan Wang, Caiming Xiong, and Silvio Savarese · 2024
Closest in time.
Language agents as optimizable graphs
Mingchen Zhuge, Wenyi Wang, Louis Kirsch, Francesco Faccio, Dmitrii Khizbullin, and Jurgen Schmidhuber · 2024
Closest in time.
Are large language models good prompt optimizers?
Ruotian Ma, Xiaolei Wang, Xin Zhou, Jian Li, Nan Du, Tao Gui, Qi Zhang, and Xuanjing Huang · 2024
Closest in time.
PromptAgent: Strategic planning with language models enables expert-level prompt optimization
Xinyuan Wang, Chenxi Li, Zhen Wang, Fan Bai, Haotian Luo, Jiayou Zhang, Nebojsa Jojic, Eric Xing, and Zhiting Hu · 2024
Closest in time.
Xinyu Tang, Xiaolei Wang, Wayne Xin Zhao, Siyuan Lu, Yaliang Li, and Ji-Rong Wen · 2024
Closest in time.
Teaching large language models to self-debug
Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou · 2024
Closest in time.
LLaMoCo: Instruction tuning of large language models for optimization code generation
Zeyuan Ma, Hongshu Guo, Jiacheng Chen, Guojun Peng, Zhiguang Cao, Yining Ma, and Yue-Jiao Gong · 2024
Closest in time.
Large language models to enhance bayesian optimization
Tennison Liu, Nicolás Astorga, Nabeel Seedat, and Mihaela van der Schaar · 2024
Closest in time.