2023

Reprompting: Automated Chain-of-Thought Prompt Inference Through Gibbs Sampling

Xu, Weijia, Banburski-Fahey, Andrzej, Jojic, Nebojsa

Understand

We introduce Reprompting, an iterative sampling algorithm that automatically learns the Chain-of-Thought (CoT) recipes for a given task without human intervention.

Through Gibbs sampling, Reprompting infers the CoT recipes that work consistently well for a set of training samples by iteratively sampling new recipes using previously sampled recipes as parent prompts to solve other training problems.
We conduct extensive experiments on 20 challenging reasoning tasks.
Results show that Reprompting outperforms human-written CoT prompts substantially by +9.4 points on average.

Built on

Stochastic relaxation, gibbs distributions, and the bayesian restoration of images
Geman, S. and Geman, D · 1984
Earlier work this paper cites.
Explaining the gibbs sampler
Casella, G. and George, E. I · 1992
Earlier work this paper cites.
Simple conditions for the convergence of the gibbs sampler and metropolis-hastings algorithms
Roberts, G. O. and Smith, A. F · 1994
Earlier work this paper cites.
Language models are few-shot learners
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., and Amodei, D · 2020
Earlier work this paper cites.
Exploiting cloze questions for few shot text classification and natural language inference
Original
Schick, T. and Schütze, H · 2020
Earlier work this paper cites.
Unsupervised commonsense question answering with self-talk
Shwartz, V., West, P., Le Bras, R., Bhagavatula, C., and Choi, Y · 2020
Earlier work this paper cites.
Program synthesis with large language models
Original
Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., Le, Q., et al · 2021
Earlier work this paper cites.
Evaluating large language models trained on code, 2021
Original
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. d. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., Ray, A., Puri, R., Krueger, G., Petrov, M., Khlaaf, H., Sastry, G., Mishkin, P., Chan, B., Gray, S., Ryder, N., Pavlov, M., Power, A., Kaiser, L., Bavarian, M., Winter, C., Tillet, P., Such, F. P., Cummings, D., Plappert, M., Chantzis, F., Barnes, E., Herbert-Voss, A., Guss, W. H., Nichol, A., Paino, A., Tezak, N., Tang, J., Babuschkin, I., Balaji, S., Jain, S., Saunders, W., Hesse, C., Carr, A. N., Leike, J., Achiam, J., Misra, V., Morikawa, E., Radford, A., Knight, M., Brundage, M., Murati, M., Mayer, K., Welinder, P., McGrew, B., Amodei, D., McCandlish, S., Sutskever, I., and Zaremba, W · 2021
Earlier work this paper cites.
Training verifiers to solve math word problems
Original
Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C., and Schulman, J · 2021
Earlier work this paper cites.
Measuring mathematical problem solving with the MATH dataset
Original
Hendrycks, D., Burns, C., Kadavath, S., Arora, A., Basart, S., Tang, E., Song, D., and Steinhardt, J · 2021
Earlier work this paper cites.
Show your work: Scratchpads for intermediate computation with language models
Original
Nye, M., Andreassen, A. J., Gur-Ari, G., Michalewski, H., Austin, J., Bieber, D., Dohan, D., Lewkowycz, A., Bosma, M., Luan, D., et al · 2021
Earlier work this paper cites.
True few-shot learning with language models
Perez, E., Kiela, D., and Cho, K · 2021
Earlier work this paper cites.
Ask me anything: A simple strategy for prompting language models
Original
Arora, S., Narayan, A., Chen, M. F., Orr, L. J., Guha, N., Bhatia, K., Chami, I., Sala, F., and Ré, C · 2022
Earlier work this paper cites.

Similar

PaLM: Scaling language modeling with pathways
Original
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al · 2022
Cited alongside, same era.
Selection-inference: Exploiting large language models for interpretable logical reasoning
Original
Creswell, A., Shanahan, M., and Higgins, I · 2022
Cited alongside, same era.
Rlprompt: Optimizing discrete text prompts with reinforcement learning
Original
Deng, M., Wang, J., Hsieh, C.-P., Wang, Y., Guo, H., Shu, T., Song, M., Xing, E. P., and Hu, Z · 2022
Cited alongside, same era.
Complexity-based prompting for multi-step reasoning
Original
Fu, Y., Peng, H., Sabharwal, A., Clark, P., and Khot, T · 2022
Cited alongside, same era.
Large language models are zero-shot reasoners
Original
Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., and Iwasawa, Y · 2022
Cited alongside, same era.
On the advance of making language models better reasoners
Original
Li, Y., Lin, Z., Zhang, S., Fu, Q., Chen, B., Lou, J.-G., and Chen, W · 2022
Cited alongside, same era.
Multi-stage prompting for knowledgeable dialogue generation
Liu, Z., Patwary, M., Prenger, R., Prabhumoye, S., Ping, W., Shoeybi, M., and Catanzaro, B · 2022
Cited alongside, same era.
Text and patterns: For effective chain of thought, it takes two to tango
Original
Madaan, A. and Yazdanbakhsh, A · 2022
Cited alongside, same era.
Training language models to follow instructions with human feedback
Original
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al · 2022
Cited alongside, same era.
Measuring and narrowing the compositionality gap in language models
Original
Press, O., Zhang, M., Min, S., Schmidt, L., Smith, N. A., and Lewis, M · 2022
Cited alongside, same era.
Beyond the imitation game: Quantifying and extrapolating the capabilities of language models
Original
Srivastava, A., Rastogi, A., Rao, A., Shoeb, A. A. M., Abid, A., Fisch, A., Brown, A. R., Santoro, A., Gupta, A., Garriga-Alonso, A., et al · 2022
Cited alongside, same era.
Challenging big-bench tasks and whether chain-of-thought can solve them
Original
Suzgun, M., Scales, N., Schärli, N., Gehrmann, S., Tay, Y., Chung, H. W., Chowdhery, A., Le, Q. V., Chi, E. H., Zhou, D., et al · 2022
Cited alongside, same era.

Then

STaR: Bootstrapping reasoning with reasoning
Original
Zelikman, E., Wu, Y., and Goodman, N. D · 2022
Later among the works it cites.
Automatic chain of thought prompting in large language models
Original
Zhang, Z., Zhang, A., Li, M., and Smola, A · 2022
Later among the works it cites.
Least-to-most prompting enables complex reasoning in large language models
Original
Zhou, D., Schärli, N., Hou, L., Wei, J., Scales, N., Wang, X., Schuurmans, D., Bousquet, O., Le, Q., and Chi, E · 2022
Later among the works it cites.
Gpt is becoming a turing machine: Here are some ways to program it, 2023
Jojic, A., Wang, Z., and Jojic, N · 2023
Closest in time.
Gpt-4 technical report, 2023
OpenAI · 2023
Closest in time.
Art: Automatic multi-step reasoning and tool-use for large language models
Original
Paranjape, B., Lundberg, S., Singh, S., Hajishirzi, H., Zettlemoyer, L., and Ribeiro, M. T · 2023
Closest in time.
Automatic prompt optimization with “gradient descent” and beam search
Pryzant, R., Iter, D., Li, J., Lee, Y., Zhu, C., and Zeng, M · 2023
Closest in time.
Answering questions by meta-reasoning over multiple chains of thought
Original
Yoran, O., Wolfson, T., Bogin, B., Katz, U., Deutch, D., and Berant, J · 2023
Closest in time.
Why johnny can’t prompt: How non-ai experts try (and fail) to design llm prompts
Zamfirescu-Pereira, J., Wong, R. Y., Hartmann, B., and Yang, Q · 2023
Closest in time.
TEMPERA: Test-time prompt editing via reinforcement learning
Zhang, T., Wang, X., Zhou, D., Schuurmans, D., and Gonzalez, J. E · 2023
Closest in time.
Progressive-hint prompting improves reasoning in large language models
Original
Zheng, C., Liu, Z., Xie, E., Li, Z., and Li, Y · 2023
Closest in time.
Large language models are human-level prompt engineers
Zhou, Y., Muresanu, A. I., Han, Z., Paster, K., Pitis, S., Chan, H., and Ba, J · 2023
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…