Fetching the paper…

Reading the bibliography…

2023

Meta Prompting for AI Systems

Zhang, Yifan, Yuan, Yang, Yao, Andrew Chi-Chih

Original
PDF
Post on X

Understand

We introduce Meta Prompting (MP), a framework that emphasizes the formal structure of a task rather than content-specific worked examples.

We give a categorical formalization in which a functor maps typed task transformations to typed prompt transformations.
Functoriality encodes preservation of identities and composition, but it does not by itself guarantee semantic correctness.
We extend MP to Recursive Meta Prompting (RMP), in which a proposer model generates candidate prompt edits, a validator enforces the edit schema, and an executor model uses the resulting prompt.

Meta Prompting for AI Systems · Around

Built on

Using “annotator rationales” to improve machine learning for text categorization
Omar Zaidan, Jason Eisner, and Christine Piatko · 2007
Earlier work this paper cites.
Thinking, fast and slow
Daniel Kahneman · 2011
Earlier work this paper cites.
Sympy: symbolic computing in python
Aaron Meurer, Christopher P. Smith, Mateusz Paprocki, Ondřej Čertík, Sergey B. Kirpichev, Matthew Rocklin, AMiT Kumar, Sergiu Ivanov, Jason K. Moore, Sartaj Singh, Thilina Rathnayake, Sean Vig, Brian E. Granger, Richard P. Muller, Francesco Bonazzi, Harsh Gupta, Shivam Vats, Fredrik Johansson, Fabian Pedregosa, Matthew J. Curry, Andy R. Terrel, Štěpán Roučka, Ashutosh Saboo, Isuru Fernando, Sumith Kulal, Robert Cimrman, and Anthony Scopatz · 2017
Earlier work this paper cites.
Commonsense for generative multi-hop question answering tasks
Original
Lisa Bauer, Yicheng Wang, and Mohit Bansal · 2018
Earlier work this paper cites.
Bert: Pre-training of deep bidirectional transformers for language understanding
Original
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2018
Earlier work this paper cites.
Exploiting explicit paths for multi-hop reading comprehension
Original
Souvik Kundu, Tushar Khot, Ashish Sabharwal, and Peter Clark · 2018
Earlier work this paper cites.
Knowledgeable reader: Enhancing cloze-style reading comprehension with external commonsense knowledge
Original
Todor Mihaylov and Anette Frank · 2018
Earlier work this paper cites.
Improving language understanding by generative pre-training
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al · 2018
Earlier work this paper cites.
Cognitive graph for multi-hop reading comprehension at scale
Original
Ming Ding, Chang Zhou, Qibin Chen, Hongxia Yang, and Jie Tang · 2019
Earlier work this paper cites.
Kagnet: Knowledge-aware graph networks for commonsense reasoning
Original
Bill Yuchen Lin, Xinyue Chen, Jamin Chen, and Xiang Ren · 2019
Earlier work this paper cites.
Language models are unsupervised multitask learners
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al · 2019
Earlier work this paper cites.
Improving natural language inference using external knowledge in the science questions domain
Xiaoyan Wang, Pavan Kapanipathi, Ryan Musa, Mo Yu, Kartik Talamadupula, Ibrahim Abdelaziz, Maria Chang, Achille Fokoue, Bassem Makni, Nicholas Mattei, et al · 2019
Earlier work this paper cites.
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al · 2020
Earlier work this paper cites.
Scalable multi-hop relational reasoning for knowledge-aware question answering
Original
Yanlin Feng, Xinyue Chen, Bill Yuchen Lin, Peifeng Wang, Jun Yan, and Xiang Ren · 2020
Earlier work this paper cites.
Multi-step inference for reasoning over paragraphs
Original
Jiangming Liu, Matt Gardner, Shay B Cohen, and Mirella Lapata · 2020
Earlier work this paper cites.
Exploring the limits of transfer learning with a unified text-to-text transformer
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu · 2020
Earlier work this paper cites.
Training verifiers to solve math word problems
Original
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al · 2021
Earlier work this paper cites.

Similar

When can models learn from explanations? a formal framework for understanding the roles of explanation data
Original
Peter Hase and Mohit Bansal · 2021
Cited alongside, same era.
Measuring mathematical problem solving with the math dataset
Original
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt · 2021
Cited alongside, same era.
Prompt programming for large language models: Beyond the few-shot paradigm
Laria Reynolds and Kyle McDonell · 2021
Cited alongside, same era.
Refining language models with compositional explanations
Huihan Yao, Ying Chen, Qinyuan Ye, Xisen Jin, and Xiang Ren · 2021
Cited alongside, same era.
Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks
Original
Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W Cohen · 2022

Then

Tora: A tool-integrated reasoning agent for mathematical problem solving
Original
Zhibin Gou, Zhihong Shao, Yeyun Gong, Yujiu Yang, Minlie Huang, Nan Duan, Weizhu Chen, et al · 2023
Closest in time.
Look before you leap: Unveiling the power of gpt-4v in robotic vision-language planning
Original
Yingdong Hu, Fanqi Lin, Tong Zhang, Li Yi, and Yang Gao · 2023
Closest in time.
Making language models better reasoners with step-aware verifier
Yifei Li, Zeqi Lin, Shizhuo Zhang, Qiang Fu, Bei Chen, Jian-Guang Lou, and Weizhu Chen · 2023
Closest in time.
Let’s verify step by step
Original
Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe · 2023
Closest in time.
Large language model guided tree-of-thought
Original

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Cited alongside, same era.

Metaprompting: Learning to learn better prompts

Yutai Hou, Hongyuan Dong, Xinghao Wang, Bohan Li, and Wanxiang Che · 2022

Cited alongside, same era.

Draft, sketch, and prove: Guiding formal theorem provers with informal proofs

Original

Albert Qiaochu Jiang, Sean Welleck, Jin Peng Zhou, Wenda Li, Jiacheng Liu, Mateja Jamnik, Timothée Lacroix, Yuhuai Wu, and Guillaume Lample · 2022

Cited alongside, same era.

Solving quantitative reasoning problems with language models

Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, et al · 2022

Cited alongside, same era.

Chain of thought prompting elicits reasoning in large language models

Original

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou · 2022

Cited alongside, same era.

Ai chains: Transparent and controllable human-ai interaction by chaining large language model prompts

Tongshuang Wu, Michael Terry, and Carrie Jun Cai · 2022

Cited alongside, same era.

Seqzero: Few-shot compositional semantic parsing with sequential prompts and zero-shot models

Jingfeng Yang, Haoming Jiang, Qingyu Yin, Danqing Zhang, Bing Yin, and Diyi Yang · 2022

Cited alongside, same era.

Least-to-most prompting enables complex reasoning in large language models

Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Olivier Bousquet, Quoc Le, and Ed Chi · 2022

Cited alongside, same era.

Jieyi Long · 2023

Closest in time.

Microsoft guidance library

Scott Lundberg, Marco Tulio Correia Ribeiro, David Viggiano, Joao Rafael, Riya Amemiya, and et. al · 2023

Closest in time.

Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct

Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang · 2023

Closest in time.

Faithful chain-of-thought reasoning

Original

Qing Lyu, Shreya Havaldar, Adam Stein, Li Zhang, Delip Rao, Eric Wong, Marianna Apidianaki, and Chris Callison-Burch · 2023

Closest in time.

Gpt-4 technical report

Original

OpenAI · 2023

Closest in time.

Reflexion: Language agents with verbal reinforcement learning

Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik R Narasimhan, and Shunyu Yao · 2023

Closest in time.

Llama: Open and efficient foundation language models

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al · 2023

Closest in time.

Leandojo: Theorem proving with retrieval-augmented language models

Kaiyu Yang, Aidan M Swope, Alex Gu, Rahul Chalamala, Peiyang Song, Shixing Yu, Saad Godil, Ryan Prenger, and Anima Anandkumar · 2023

Closest in time.

Tree of thoughts: Deliberate problem solving with large language models

Original

Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L Griffiths, Yuan Cao, and Karthik Narasimhan · 2023

Closest in time.

Metamath: Bootstrap your own mathematical questions for large language models

Original

Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu · 2023

Closest in time.

Cumulative reasoning with large language models

Yifan Zhang, Jingqin Yang, Yang Yuan, and Andrew Chi-Chih Yao · 2023

Closest in time.

Langchain: Building applications with llms through composability

LangChain Contributors · 2024

Closest in time.

Multi-level recommendation reasoning over knowledge graphs with reinforcement learning

Xiting Wang, Kunpeng Liu, Dongjie Wang, Le Wu, Yanjie Fu, and Xing Xie · 2098

Closest in time.