Fetching the paper…

Reading the bibliography…

2024

LLM Guided Evolution -- The Automation of Models Advancing Models

Morris, Clint, Jurado, Michael, Zutty, Jason

Original
PDF
Post on X

Understand

In the realm of machine learning, traditional model development and automated approaches like AutoML typically rely on layers of abstraction, such as tree-based or Cartesian genetic programming.

Our study introduces "Guided Evolution" (GE), a novel framework that diverges from these methods by utilizing Large Language Models (LLMs) to directly modify code.
GE leverages LLMs for a more intelligent, supervised evolutionary process, guiding mutations and crossovers.
Our unique "Evolution of Thought" (EoT) technique further enhances GE by enabling LLMs to reflect on and learn from the outcomes of previous mutations.

LLM Guided Evolution -- The Automation of Models Advancing Models · Around

Built on

SPEA2: Improving the strength pareto evolutionary algorithm for multiobjective optimization. In Evolutionary Methods for Design Optimization and Control with Applications to Industrial Problems (19–21 September 2001), K. C. Giannakoglou, D. T. Tsahalis, J. Périaux, K. D. Papailiou, and T. Fogarty (Eds.). International Center for Numerical Methods in Engineering, Athens, Greece, 95–100
E. Zitzler, M. Laumanns, and L. Thiele. 2001 · 2001
Earlier work this paper cites.
A fast and elitist multiobjective genetic algorithm: NSGA-II
K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. 2002 · 2002
Earlier work this paper cites.
Learning Multiple Layers of Features from Tiny Images
Alex Krizhevsky. 2009 · 2009
Earlier work this paper cites.
Multiple objective vector-based genetic programming using human-derived primitives. In Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation . 1127–1134
Original
Jason Zutty, Daniel Long, Heyward Adams, Gisele Bennett, and Christina Baxter. 2015 · 2015
Earlier work this paper cites.
Neural architecture search with reinforcement learning
Original
Barret Zoph and Quoc V Le. 2016 · 2016
Earlier work this paper cites.
Accelerating neural architecture search using performance prediction
Original
Bowen Baker, Otkrist Gupta, Ramesh Raskar, and Nikhil Naik. 2017 · 2017
Earlier work this paper cites.
Evolving Deep Neural Networks
Original
Risto Miikkulainen, Jason Liang, Elliot Meyerson, Aditya Rawal, Dan Fink, Olivier Francon, Bala Raju, Hormoz Shahrzad, Arshak Navruzyan, Nigel Duffy, and Babak Hodjat. 2017 · 2017
Earlier work this paper cites.
Searching for Activation Functions
Original
Prajit Ramachandran, Barret Zoph, and Quoc V. Le. 2017 · 2017
Earlier work this paper cites.

Similar

Path-level network transformation for efficient architecture search. In International Conference on Machine Learning . PMLR, 678–687
Han Cai, Jiacheng Yang, Weinan Zhang, Song Han, and Yong Yu. 2018 · 2018
Cited alongside, same era.
Darts: Differentiable architecture search
Original
Hanxiao Liu, Karen Simonyan, and Yiming Yang. 2018 · 2018
Cited alongside, same era.
Efficient neural architecture search via parameters sharing. In International conference on machine learning . PMLR, 4095–4104
Hieu Pham, Melody Guan, Barret Zoph, Quoc Le, and Jeff Dean. 2018 · 2018
Cited alongside, same era.
Nsga-net: neural architecture search using multi-objective genetic algorithm. In Proceedings of the genetic and evolutionary computation conference . 419–427
Zhichao Lu, Ian Whalen, Vishnu Boddeti, Yashesh Dhebar, Kalyanmoy Deb, Erik Goodman, and Wolfgang Banzhaf. 2019 · 2019
Cited alongside, same era.

Then

A Novel lightweight Convolutional Neural Network, ExquisiteNetV2
Original
Shi-Yao Zhou and Chung-Yen Su. 2022 · 2022
Later among the works it cites.
A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions
Original
Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. 2023 · 2023
Later among the works it cites.
Large Language Models are Zero-Shot Reasoners
Original
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2023 · 2023
Later among the works it cites.
Conceptual Design Generation Using Large Language Models
Original
Kevin Ma, Daniele Grandi, Christopher McComb, and Kosa Goucher-Lambert. 2023 · 2023
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Regularized evolution for image classifier architecture search. In Proceedings of the aaai conference on artificial intelligence , Vol. 33. 4780–4789

Esteban Real, Alok Aggarwal, Yanping Huang, and Quoc V Le. 2019 · 2019

Cited alongside, same era.

Improving the performance of multiobjective genetic algorithms: An elitism-based approach

Giorgio Guariso and Matteo Sangiorgio. 2020 · 2020

Cited alongside, same era.

An improved genetic algorithm and its application in neural network adversarial attack

Dingming Yang, Zeyu Yu, Hongqiang Yuan, and Yanrong Cui. 2022 · 2022

Cited alongside, same era.

Tree of Thoughts: Deliberate Problem Solving with Large Language Models

Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. 2023 · 2023

Later among the works it cites.

Automatic Chain of Thought Prompting in Large Language Models. In The Eleventh International Conference on Learning Representations

Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. 2023 · 2023

Later among the works it cites.

Mixtral of Experts

Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock, Sandeep Subramanian, Sophia Yang, Szymon Antoniak, Teven Le Scao, Théophile Gervet, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2024 · 2024

Closest in time.