Fetching the paper…
Reading the bibliography…
Recent research explores optimization using large language models (LLMs) by either iteratively seeking next-step solutions from LLMs or directly prompting LLMs for an optimizer.
Optimization by simulated annealing
Kirkpatrick, S., Gelatt Jr, C. D., and Vecchi, M. P · 1983
Earlier work this paper cites.
Simulated annealing
Van Laarhoven, P. J., Aarts, E. H., van Laarhoven, P. J., and Aarts, E. H · 1987
Earlier work this paper cites.
A software package for sequential quadratic programming
Kraft, D · 1988
Earlier work this paper cites.
Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence
Holland, J. H · 1992
Earlier work this paper cites.
Backpropagation and stochastic gradient descent method
Amari, S.-i · 1993
Earlier work this paper cites.
Particle swarm optimization
Kennedy, J. and Eberhart, R · 1995
Earlier work this paper cites.
Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces
Storn, R. and Price, K · 1997
Earlier work this paper cites.
Generalized simulated annealing algorithm and its application to the thomson model
Xiang, Y., Sun, D., Fan, W., and Gong, X · 1997
Earlier work this paper cites.
Trust region methods
Conn, A. R., Gould, N. I., and Toint, P. L · 2000
Earlier work this paper cites.
Completely derandomized self-adaptation in evolution strategies
Hansen, N. and Ostermeier, A · 2001
Earlier work this paper cites.
A study of the behavior of several methods for balancing machine learning training data
Batista, G. E., Prati, R. C., and Monard, M. C · 2004
Earlier work this paper cites.
Convex optimization
Boyd, S. P. and Vandenberghe, L · 2004
Earlier work this paper cites.
Dimensionality reduction by learning an invariant mapping
Hadsell, R., Chopra, S., and LeCun, Y · 2006
Earlier work this paper cites.
A view of algorithms for optimization without derivatives
Powell, M. J · 2007
Earlier work this paper cites.
Natural selection fails to optimize mutation rates for long-term adaptation on rugged fitness landscapes
Clune, J., Misevic, D., Ofria, C., Lenski, R. E., Elena, S. F., and Sanjuán, R · 2008
Earlier work this paper cites.
A simple modification in cma-es achieving linear time and space complexity
Ros, R. and Hansen, N · 2008
Earlier work this paper cites.
Benchmarking a bi-population cma-es on the bbob-2009 function testbed
Hansen, N · 2009
Earlier work this paper cites.
Remark on “algorithm 778: L-bfgs-b: Fortran subroutines for large-scale bound constrained optimization”
Morales, J. L. and Nocedal, J · 2011
Earlier work this paper cites.
Deap: Evolutionary algorithms made easy
Fortin, F.-A., De Rainville, F.-M., Gardner, M.-A. G., Parizeau, M., and Gagné, C · 2012
Earlier work this paper cites.
Practical bayesian optimization of machine learning algorithms
Snoek, J., Larochelle, H., and Adams, R. P · 2012
Earlier work this paper cites.
Genetic learning particle swarm optimization
Gong, Y.-J., Li, J.-J., Zhou, Y., Li, Y., Chung, H. S.-H., Shi, Y.-H., and Zhang, J · 2015
Earlier work this paper cites.
Scikit-optimize, 2016
Louppe, G. and Kumar, M · 2016
Earlier work this paper cites.
Deep reinforcement learning from human preferences
Christiano, P. F., Leike, J., Brown, T., Martic, M., Legg, S., and Amodei, D · 2017
Earlier work this paper cites.
Proximal policy optimization algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O · 2017
Earlier work this paper cites.
Attention is all you need
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I · 2017
Earlier work this paper cites.
Problem definitions and evaluation criteria for the cec 2017 competition on constrained real-parameter optimization
Wu, G., Mallipeddi, R., and Suganthan, P. N · 2017
Earlier work this paper cites.
A progressive batching l-bfgs method for machine learning
Bollapragada, R., Nocedal, J., Mudigere, D., Shi, H.-J., and Tang, P. T. P · 2018
Earlier work this paper cites.
Decoupled weight decay regularization
Loshchilov, I. and Hutter, F · 2019
Earlier work this paper cites.
Bayesmark: Benchmark framework to easily compare bayesian optimization methods on real machine learning tasks , 2019
R.Turner and D.Eriksson · 2019
Earlier work this paper cites.
Huggingface’s transformers: State-of-the-art natural language processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al · 2019
Cited alongside, same era.
Fine-tuning language models from human preferences
Ziegler, D. M., Stiennon, N., Wu, J., Brown, T. B., Radford, A., Amodei, D., Christiano, P., and Irving, G · 2019
Cited alongside, same era.
Language models are few-shot learners
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al · 2020
Cited alongside, same era.
Gpt-3: Its nature, scope, limits, and consequences
Floridi, L. and Chiriatti, M · 2020
Cited alongside, same era.
Mmes: Mixture model-based evolution strategy for large-scale optimization
He, X., Zheng, Z., and Zhou, Y · 2020
Cited alongside, same era.
Optimus: Optimization modeling using mip solvers and large language models
AhmadiTeshnizi, A., Gao, W., and Udell, M · 2023
Later among the works it cites.
Evoprompting: Language models for code-level neural architecture search
Chen, A., Dohan, D. M., and So, D. R · 2023
Later among the works it cites.
A hybrid particle swarm optimization and simulated annealing algorithm for the job shop scheduling problem with transport resources
Fontes, D. B., Homayouni, S. M., and Gonçalves, J. F · 2023
Later among the works it cites.
Incoder: A generative model for code infilling and synthesis
Fried, D., Aghajanyan, A., Lin, J., Wang, S., Wallace, E., Shi, F., Zhong, R., Yih, S., Zettlemoyer, L., and Lewis, M · 2023
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
A test-suite of non-convex constrained optimization problems from the real-world and some baseline results
Kumar, A., Wu, G., Ali, M. Z., Mallipeddi, R., Suganthan, P. N., and Das, S · 2020
Cited alongside, same era.
Exploring the limits of transfer learning with a unified text-to-text transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J · 2020
Cited alongside, same era.
Scipy 1.0: fundamental algorithms for scientific computing in python
Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., et al · 2020
Cited alongside, same era.
Learning search space partition for black-box optimization using monte carlo tree search
Wang, L., Fonseca, R., and Tian, Y · 2020
Cited alongside, same era.
Helper and equivalent objectives: Efficient approach for constrained optimization
Xu, T., He, J., and Shang, C · 2020
Cited alongside, same era.
Program synthesis with large language models
Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., Le, Q., et al · 2021
Cited alongside, same era.
Improving differential evolution through bayesian hyperparameter optimization
Biswas, S., Saha, D., De, S., Cobb, A. D., Das, S., and Jalaian, B. A · 2021
Cited alongside, same era.
Gupta, H., Sawant, S. A., Mishra, S., Nakamura, M., Mitra, A., Mashetty, S., and Baral, C · 2023
Later among the works it cites.
Huang, Q., Tao, M., An, Z., Zhang, C., Jiang, C., Chen, Z., Wu, Z., and Feng, Y · 2023
Later among the works it cites.
Phi-2: The surprising power of small language models, 2023
Javaheripi, M., Bubeck, S., Abdin, M., Aneja, J., Bubeck, S., Mendes, C. C. T., Chen, W., Del Giorno, A., Eldan, R., Gopi, S., et al · 2023
Later among the works it cites.
Structgpt: A general framework for large language model to reason over structured data
Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, W. X., and Wen, J.-R · 2023
Later among the works it cites.
Ds-1000: A natural and reliable benchmark for data science code generation
Lai, Y., Li, C., Wang, Y., Zhang, T., Zhong, R., Zettlemoyer, L., Yih, W.-t., Fried, D., Wang, S., and Yu, T · 2023
Later among the works it cites.
evosax: Jax-based evolution strategies
Lange, R. T · 2023
Later among the works it cites.
Evolution through large models
Lehman, J., Gordon, J., Jain, S., Ndousse, K., Yeh, C., and Stanley, K. O · 2023
Later among the works it cites.
Starcoder: may the source be with you!
Li, R., Allal, L. B., Zi, Y., Muennighoff, N., Kocetkov, D., Mou, C., Marone, M., Akiki, C., Li, J., Chim, J., et al · 2023
Later among the works it cites.
Double-track particle swarm optimizer for nonlinear constrained optimization problems
Lu, H.-C., Tseng, H.-Y., and Lin, S.-W · 2023
Later among the works it cites.
Chatting about chatgpt: how may ai and gpt impact academia and libraries?
Lund, B. D. and Wang, T · 2023
Later among the works it cites.
Codegen: An open large language model for code with multi-turn program synthesis
Nijkamp, E., Pang, B., Hayashi, H., Tu, L., Wang, H., Zhou, Y., Savarese, S., and Xiong, C · 2023
Later among the works it cites.
Leveraging large language models for the generation of novel metaheuristic optimization algorithms
Pluhacek, M., Kazikova, A., Kadavy, T., Viktorin, A., and Senkerik, R · 2023
Later among the works it cites.
Direct preference optimization: Your language model is secretly a reward model
Rafailov, R., Sharma, A., Mitchell, E., Ermon, S., Manning, C. D., and Finn, C · 2023
Later among the works it cites.
Code llama: Open foundation models for code
Roziere, B., Gehring, J., Gloeckle, F., Sootla, S., Gat, I., Tan, X. E., Adi, Y., Liu, J., Remez, T., Rapin, J., et al · 2023
Later among the works it cites.
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al · 2023
Later among the works it cites.
Llama 2: Open foundation and fine-tuned chat models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al · 2023
Later among the works it cites.
An improved genetic algorithm for constrained optimization problems
Wang, F., Xu, G., and Wang, M · 2023
Later among the works it cites.
Large language models as optimizers
Yang, C., Wang, X., Lu, Y., Liu, H., Le, Q. V., Zhou, D., and Chen, X · 2023
Later among the works it cites.
Differential evolution with alternation between steady monopoly and transient competition of mutation strategies
Ye, C., Li, C., Li, Y., Sun, Y., Yang, W., Bai, M., Zhu, X., Hu, J., Chi, T., Zhu, H., et al · 2023
Later among the works it cites.
Large language models meet nl2code: A survey
Zan, D., Chen, B., Zhang, F., Lu, D., Wu, B., Guan, B., Yongji, W., and Lou, J.-G · 2023
Later among the works it cites.
Recommendation as instruction following: A large language model empowered recommendation approach
Zhang, J., Xie, R., Hou, Y., Zhao, W. X., Lin, L., and Wen, J.-R · 2023
Later among the works it cites.
A survey of large language models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., et al · 2023
Later among the works it cites.
Codegeex: A pre-trained model for code generation with multilingual benchmarking on humaneval-x
Zheng, Q., Xia, X., Zou, X., Dong, Y., Wang, S., Xue, Y., Shen, L., Wang, Z., Wang, A., Li, Y., et al · 2023
Later among the works it cites.
Lima: Less is more for alignment
Zhou, C., Liu, P., Xu, P., Iyer, S., Sun, J., Mao, Y., Ma, X., Efrat, A., Yu, P., Yu, L., et al · 2023
Later among the works it cites.
Jiang, A. Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D. S., Casas, D. d. l., Hanna, E. B., Bressand, F., et al · 2024
Closest in time.