Collection on Large Language Models for Optimization (LLM4Opt) 🔥 Applying Large language models (LLMs ) for diverse optimization tasks (Opt ) is an emerging research area. This is a collection of references and papers of LLM4Opt . The Papers are sorted by time (first publicly available).
Any suggestions and pull requests are welcomed!
It is far from a comprehensive list. If you want to update the list:
The sharing principle of these references here is for research. If any authors do not want their paper to be listed here, please feel free to contact us.
Paper distribution from A Systematic Survey on Large Language Models for Algorithm Design, 2024
Event
Link
NeurIPS 2023 Workshop: Foundation Models for Decision Making
Link
AAAI 2024 Workshop: Public Sector LLMs: Algorithmic and Sociotechnical Design
Link
GECCO 2024 Workshop: Large Language Models for and with Evolutionary Computation (LLMfwEC)
Link
GECCO 2024 Workshop: EGML-EC — 3rd GECCO workshop on Enhancing Generative Machine Learning with Evolutionary Computation (EGML-EC) 2024
Link
GECCO 2024 Tutorial: Using Large Language Models for Evolutionary Search
Tutorial Link , Tutorial Report Link
PPSN 2024 Tutorial: Large Language Models as Tools for Metaheuristic Design: Exploring Challenges and Opportunities
Link
KDD 2024 Tutorial: NL2Code-Reasoning and Planning with LLM for Code Development
Link , Link
ICML 2024 Workshop: AI for Math
Link
NeurIPS 2024 Workshop: Multimodal Algorithmic Reasoning (MAR)
Link
NeurIPS 2024 Workshop: The 4th Workshop on Mathematical Reasoning and AI
Link
Event
Link
AAAI 2024 Global Competition on Math Problem Solving and Reasoning
Link
ICML 2024 Challenges on Automated Math Reasoning
Link
IEEE CIS 2024 Technical Challenge FLAME – ‘Fusing Large lAnguage Models with computational intElligence’
Link
Event
Link
IEEE TETCI, Special Issue on Neural Architecture Search and Large Machine Learning Models
Link
ACM TELO, Special Issue on Integrating Evolutionary Algorithms and Large Language Models
Link
IEEE TEVC, Special Issue on Evolutionary Computation Meets Large Language Models
link
Engineering, Special Issue on Applications of ChatGPT
link
Title
Publication with Date
Code
Paper
Evolutionary Computation in the Era of Large Language Model: Survey and Roadmap
Arxiv, Jan 2024
[code]
paper
A Survey of Neural Code Intelligence: Paradigms, Advances and Beyond
Arxiv, Mar. 2024
[code]
paper
When Large Language Model Meets Optimization
Arxiv, May 2024
[code]
paper
A Systematic Survey on Large Language Models for Algorithm Design
Arxiv, Oct 2024
[code]
paper
Title
Publication with Date
Code
Paper
The Era OF Semantic Decoding
Arxiv, Mar. 2024
[code]
paper
Leveraging Foundational Models for Black-Box Optimization: Benefits, Challenges, and Future Directions
ICML 2024, May 2024
[code]
paper
Algorithm/Heuristic/Function Search
Title
Publication with Date
Code
Paper
Hypothesis Search: Inductive Reasoning with Language Models
Arxiv Sep 2023, ICLR 2024
[code]
paper
ToolChain*: Efficient Action Space Navigation in Large Language Models with A* Search
Arxiv Oct 2023, ICLR 2024
[code]
paper
Algorithm Evolution using Large Language Model
Arxiv Nov 2023
code paper
Mathematical discoveries from program search with large language models
Nature code paper
Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Model
ICML 2024 (Oral) code paper
ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolution
Arxiv Feb 2024, NeurIPS 2024
code paper
Discovering More Effective Tensor Network Structure Search Algorithms via Large Language Models (LLMs)
[code]
paper
AutoSAT: Automatically Optimize SAT Solvers via Large Language Models
Arxiv Feb 2024
[code]
paper
Large Language Models tO Enhance Bayesian Optimization
Arxiv Feb 2024, ICLR 2024
[code]
paper
On the Self-Verification Limitations of Large Language Models on Reasoning and Planning Tasks
Arxiv Feb 2024
[code]
paper
How Can LLM Guide RL? A Value-Based Approach
Arxiv Feb 2024
code paper
LLaMoCo: Instruction Tuning of Large Language Models for Optimization Code Generation
Arxiv Mar 2024
code paper
Evolve Cost-aware Acquisition Functions Using Large Language Models
PPSN 2024 [code]
paper
Benchmarking ChatGPT on Algorithmic Reasoning
Arxiv April 2024
[code]
paper
How Multimodal Integration Boost the Performance of LLM for Optimization: Case Study on Capacitated Vehicle Routing Problems
Arxiv March 2024
[code]
paper
LLM-ABR: Designing Adaptive Bitrate Algorithms via Large Language Models
Arxiv April 2024
[code]
paper
Constrained Neural Networks for Interpretable Heuristic Creation to Optimise Computer Algebra Systems
Arxiv April 2024
[code]
paper
GLHF: General Learned Evolutionary Algorithm Via Hyper Functions
Arxiv
[code]
paper
tnGPS: Discovering Unknown Tensor Network Structure Search Algorithms via Large Language Models (LLMs)
ICML 2024
[code]
paper
LLaMEA: A Large Language Model Evolutionary Algorithm for Automatically Generating Metaheuristics
Arxiv May 2024
code paper
Understanding the Importance of Evolutionary Search in Automated Heuristic Design with Large Language Models
PPSN 2024
code
paper
Discovering Preference Optimization Algorithms with and for Large Language Models
Arxiv June 2024
code
paper
OMNI-EPIC: Open-endedness via Models of human Notions of Interestingness with Environments Programmed in Code
Arxiv May 2024
code paper
In-the-loop Hyper-Parameter Optimization for LLM-Based Automated Design of Heuristics
Arxiv Oct 2024
code paper
Title
Publication with Date
Code
Paper
Large Language Models as Optimizers
Arxiv, Sep 2023
code paper
Large language model for multi-objective evolutionary optimization
Arxiv, Oct. 2023
code paper
Large Language Models as Evolutionary Optimizers
Arxiv, Oct. 2023
[code]
paper
Using Large Language Models for Hyperparameter Optimization
NeurIPS 2023
[code]
paper
Large Language Models As Evolution Strategies
Arxiv, Feb. 2024
[code]
paper
Large Language Model-based Evolutionary Optimizer: Reasoning with Elitism
Arxiv, Mar. 2024
[code]
paper
Large Language Model-Aided Evolutionary Search for Constrained Multiobjective Optimization
Arxiv, May 9, 2024
[code]
paper
Towards Optimizing with Large Language Model
KDD 2024
[code]
paper
LLM as/for surrogate model
Title
Publication with Date
Code
Paper
Large Language Models as Surrogate Models in Evolutionary Algorithms: A Preliminary Study
Arxiv, June 2024
code paper
Large Language Model-assisted Surrogate Modelling for Engineering Optimization
CAI, 2024
[code]
paper
LLM Performance Predictors are good initializers for Architecture Search
Arxiv, Aug 2024
[code]
paper
Title
Publication with Date
Code
Paper
L2MAC: LARGE LANGUAGE MODEL AUTOMATIC COMPUTER FOR EXTENSIVE CODE GENERATION
Arxiv Oct 2023, ICLR 2024
[code]
paper
Title
Publication with Date
Code
Paper
Connecting Large Language Models with Evolutionary Algorithms Yields Powerful Prompt Optimizers
Arxiv Sep 2023, ICLR 2024
[code]
paper
PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimization
Arxiv Oct 2023, ICLR 2024
[code]
paper
Title
Publication with Date
Code
Paper
Evolution through Large Models
Arxiv, June 2022
code paper
Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph
Arxiv, July 2023, ICLR 2024
[code]
paper
Evoprompting: Language models for code-level neural architecture search
NeuIPS 2023
[code]
paper
Eureka: Human-Level Reward Design via Coding Large Language Models
ICLR 2024
code paper
Language Model Decoding as Direct Metrics Optimization
Arxiv, Oct 2023, ICLR 2024
[code]
paper
Label-free Node Classification on Graphs with Large Language Models (LLMS)
Arxiv, Oct 2023, ICLR 2024
[code]
paper
L-AutoDA: Leveraging Large Language Models for Automated Decision-based Adversarial Attacks
GECCO 2024, Jan 2024
[code]
paper
Data-driven Discovery with Large Generative Models
Arxiv, Feb. 2024
[code]
paper
Large Language Model-driven Meta-structure Discovery in Heterogeneous Information Network
Arxiv, Feb. 2024
[code]
paper
LLM Guided Evolution-The Automation of Models Advancing Models
Arxiv, Mar. 2024
[code]
paper
Identify Critical Nodes in Complex Network with Large Language Models
Arxiv, Mar. 2024
[code]
paper
Evolving Interpretable Visual Classifiers with Large Language Models
Arxiv, April 2024
[code]
paper
Title
Publication with Date
Code
Paper
Exploring evolution-aware & -free protein language models as protein function predictors
NeurIPS 2022
[code]
paper
A Prompt-Engineered Large Language Model
Arxiv, Jan 2024
[code]
paper
LLM-SR: Scientific Equation Discovery via Programming with Large Language Models
Arxiv, April 2024
[code]
paper
Large Language Model Agent as a Mechanical Designer
Arxiv, April 2024
[code]
paper
LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discovery
Arxiv, May 2024
code paper
Taming Large Language Model For Conversational Protein Design
Openreview, 2024
[code]
paper
AutoTurb: Using Large Language Models for Automatic Algebraic Model Discovery of Turbulence Closure
Arxiv, Oct. 2024
[code]
paper
Title
Publication with Date
Code
Paper
Large Language Models for Supply Chain Optimization
Arxiv, July 2023
[code]
paper
How Can Large Language Models Help Humans in Design and Manufacturing
Arxiv, July 2023
[code]
paper
LLM4EDA: Emerging Progress in Large Language Models for Electronic Design Automation
Arxiv, Dec. 2023
[code]
paper
Collection
Link
Awesome LLM
Link
Foundation Models for Combinatorial Optimization
Link
LLM for Planning
Link
MOO-ML-Papers
Link
