Fetching the paper…

Reading the bibliography…

2023

Guiding Large Language Models via Directional Stimulus Prompting

Li, Zekun, Peng, Baolin, He, Pengcheng et al.

Original
PDF
Post on X

Understand

We introduce Directional Stimulus Prompting, a novel framework for guiding black-box large language models (LLMs) toward specific desired outputs.

Instead of directly adjusting LLMs, our method employs a small tunable policy model (e.g., T5) to generate an auxiliary directional stimulus prompt for each input instance.
These directional stimulus prompts act as nuanced, instance-specific hints and clues to guide LLMs in generating desired outcomes, such as including specific keywords in the generated summary.
Our approach sidesteps the challenges of direct LLM tuning by optimizing the policy model to explore directional stimulus prompts that align LLMs with desired behaviors.

Guiding Large Language Models via Directional Stimulus Prompting · Around

Built on

BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., and Zhu, W.-J · 2002
Earlier work this paper cites.
Rouge: A package for automatic evaluation of summaries
Lin, C.-Y · 2004
Earlier work this paper cites.
Textrank: Bringing order into text
Mihalcea, R. and Tarau, P · 2004
Earlier work this paper cites.
METEOR: An automatic metric for mt evaluation with improved correlation with human judgments
Banerjee, S. and Lavie, A · 2005
Earlier work this paper cites.
Training parsers by inverse reinforcement learning
Neu, G. and Szepesvári, C · 2009
Earlier work this paper cites.
Variations of the similarity function of textrank for automated summarization
Original
Barrios, F., López, F., Argerich, L., and Wachenchauzer, R · 2016
Earlier work this paper cites.
Deep reinforcement learning for dialogue generation
Original
Li, J., Monroe, W., Ritter, A., Galley, M., Gao, J., and Jurafsky, D · 2016
Earlier work this paper cites.
Abstractive text summarization using sequence-to-sequence rnns and beyond
Original
Nallapati, R., Zhou, B., Gulcehre, C., Xiang, B., et al · 2016
Earlier work this paper cites.
Solving general arithmetic word problems
Original
Roy, S. and Roth, D · 2016
Earlier work this paper cites.
Google’s neural machine translation system: Bridging the gap between human and machine translation
Original
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al · 2016
Earlier work this paper cites.
spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing
Honnibal, M. and Montani, I · 2017
Earlier work this paper cites.
Tackling error propagation through reinforcement learning: A case of greedy dependency parsing
Original
Le, M. and Fokkens, A · 2017
Earlier work this paper cites.
Program induction by rationale generation: Learning to solve and explain algebraic word problems
Original
Ling, W., Yogatama, D., Dyer, C., and Blunsom, P · 2017
Earlier work this paper cites.
A deep reinforced model for abstractive summarization
Original
Paulus, R., Xiong, C., and Socher, R · 2017
Earlier work this paper cites.
Proximal policy optimization algorithms
Original
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O · 2017
Earlier work this paper cites.
MultiWOZ – a large-scale multi-domain Wizard-of-Oz dataset for task-oriented dialogue modelling
Original
Budzianowski, P., Wen, T.-H., Tseng, B.-H., Casanueva, I., Ultes, S., Ramadan, O., and Gašić, M · 2018
Earlier work this paper cites.
BERT: Pre-training of deep bidirectional transformers for language understanding
Original
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K · 2018
Earlier work this paper cites.
A call for clarity in reporting bleu scores
Original
Post, M · 2018
Earlier work this paper cites.
Plug and play language models: A simple approach to controlled text generation
Original
Dathathri, S., Madotto, A., Lan, J., Hung, J., Frank, E., Molino, P., Yosinski, J., and Liu, R · 2019
Earlier work this paper cites.
MultiWOZ 2.1: A consolidated multi-domain dialogue dataset with state corrections and state tracking baselines
Original
Eric, M., Goel, R., Paul, S., Kumar, A., Sethi, A., Ku, P., Goyal, A. K., Agarwal, S., Gao, S., and Hakkani-Tur, D · 2019
Earlier work this paper cites.
CTRL: A conditional transformer language model for controllable generation
Original
Keskar, N. S., McCann, B., Varshney, L. R., Xiong, C., and Socher, R · 2019
Earlier work this paper cites.
Reinforcement learning based curriculum optimization for neural machine translation
Original
Kumar, G., Foster, G., Cherry, C., and Krikun, M · 2019
Earlier work this paper cites.
Roberta: A robustly optimized bert pretraining approach
Original
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V · 2019
Earlier work this paper cites.
Language models as knowledge bases?
Original
Petroni, F., Rocktäschel, T., Lewis, P., Bakhtin, A., Wu, Y., Miller, A. H., and Riedel, S · 2019
Earlier work this paper cites.
Language models are unsupervised multitask learners
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al · 2019
Earlier work this paper cites.

Similar

Bertscore: Evaluating text generation with bert
Original
Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., and Artzi, Y · 2019
Cited alongside, same era.
Fine-tuning language models from human preferences
Original
Ziegler, D. M., Stiennon, N., Wu, J., Brown, T. B., Radford, A., Amodei, D., Christiano, P., and Irving, G · 2019
Cited alongside, same era.
Language models are few-shot learners
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al · 2020
Cited alongside, same era.
With little power comes great responsibility
Original
Card, D., Henderson, P., Khandelwal, U., Jia, R., Mahowald, K., and Jurafsky, D · 2020
Cited alongside, same era.
Don’t stop pretraining: Adapt language models to domains and tasks
Original

Then

Do as i can, not as i say: Grounding language in robotic affordances
Original
Ahn, M., Brohan, A., Brown, N., Chebotar, Y., Cortes, O., David, B., Finn, C., Fu, C., Gopalakrishnan, K., Hausman, K., et al · 2022
Later among the works it cites.
Input-Tuning: Adapting unfamiliar inputs to frozen pretrained models
Original
An, S., Li, Y., Lin, Z., Liu, Q., Chen, B., Fu, Q., Chen, W., Zheng, N., and Lou, J.-G · 2022
Later among the works it cites.
PaLM: Scaling language modeling with pathways
Original
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al · 2022
Later among the works it cites.
Scaling instruction-finetuned language models
Original
Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, E., Wang, X., Dehghani, M., Brahma, S., et al · 2022
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Gururangan, S., Marasović, A., Swayamdipta, S., Lo, K., Beltagy, I., Downey, D., and Smith, N. A · 2020

Cited alongside, same era.

A simple language model for task-oriented dialogue

Hosseini-Asl, E., McCann, B., Wu, C.-S., Yavuz, S., and Socher, R · 2020

Cited alongside, same era.

GeDi: Generative discriminator guided sequence generation

Krause, B., Gotmare, A. D., McCann, B., Keskar, N. S., Joty, S., Socher, R., and Rajani, N. F · 2020

Cited alongside, same era.

Mintl: Minimalist transfer learning for task-oriented dialogue systems

Lin, Z., Madotto, A., Winata, G. I., and Fung, P · 2020

Cited alongside, same era.

Exploring the limits of transfer learning with a unified text-to-text transformer

Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J · 2020

Cited alongside, same era.

Autoprompt: Eliciting knowledge from language models with automatically generated prompts

Shin, T., Razeghi, Y., Logan IV, R. L., Wallace, E., and Singh, S · 2020

Cited alongside, same era.

Learning to summarize with human feedback

Stiennon, N., Ouyang, L., Wu, J., Ziegler, D., Lowe, R., Voss, C., Radford, A., Amodei, D., and Christiano, P. F · 2020

Cited alongside, same era.

Task-oriented dialog systems that consider multiple appropriate responses under the same context

Zhang, Y., Ou, Z., and Yu, Z · 2020

Cited alongside, same era.

RLPrompt: Optimizing discrete text prompts with reinforcement learning

Deng, M., Wang, J., Hsieh, C.-P., Wang, Y., Guo, H., Shu, T., Song, M., Xing, E. P., and Hu, Z · 2022

Later among the works it cites.

News summarization and evaluation in the era of GPT-3

Original

Goyal, T., Li, J. J., and Durrett, G · 2022

Later among the works it cites.

Thinking about GPT-3 in-context learning for biomedical IE? Think again

Gutiérrez, B. J., McNeal, N., Washington, C., Chen, Y., Li, L., Sun, H., and Su, Y · 2022

Later among the works it cites.

Galaxy: A generative pre-trained model for task-oriented dialog with semi-supervised learning and explicit policy injection

He, W., Dai, Y., Zheng, Y., Wu, Y., Cao, Z., Liu, D., Jiang, P., Yang, M., Huang, F., Si, L., et al · 2022

Later among the works it cites.

Demonstrate-search-predict: Composing retrieval and language models for knowledge-intensive nlp

Khattab, O., Santhanam, K., Li, X. L., Hall, D., Liang, P., Potts, C., and Zaharia, M · 2022

Later among the works it cites.

Large language models are zero-shot reasoners

Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., and Iwasawa, Y · 2022

Later among the works it cites.

Training language models to follow instructions with human feedback

Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al · 2022

Later among the works it cites.

Is reinforcement learning (not) for natural language processing?: Benchmarks, baselines, and building blocks for natural language policy optimization

Ramamurthy, R., Ammanabrolu, P., Brantley, K., Hessel, J., Sifa, R., Bauckhage, C., Hajishirzi, H., and Choi, Y · 2022

Later among the works it cites.

BLOOM: A 176b-parameter open-access multilingual language model

Scao, T. L., Fan, A., Akiki, C., Pavlick, E., Ilić, S., Hesslow, D., Castagné, R., Luccioni, A. S., Yvon, F., Gallé, M., et al · 2022

Later among the works it cites.

Black-box tuning for language-model-as-a-service

Sun, T., Shao, Y., Qian, H., Huang, X., and Qiu, X · 2022

Later among the works it cites.

Follow the wisdom of the crowd: Effective text generation via minimum bayes risk decoding

Suzgun, M., Melas-Kyriazi, L., and Jurafsky, D · 2022

Later among the works it cites.

LaMDA: Language models for dialog applications

Thoppilan, R., De Freitas, D., Hall, J., Shazeer, N., Kulshreshtha, A., Cheng, H.-T., Jin, A., Bos, T., Baker, L., Du, Y., et al · 2022

Later among the works it cites.

Democratizing access to large-scale language models with opt-175b

Zhang, S., Diab, M., and Zettlemoyer, L · 2022

Later among the works it cites.

Large language models are human-level prompt engineers

Zhou, Y., Muresanu, A. I., Han, Z., Paster, K., Pitis, S., Chan, H., and Ba, J · 2022

Later among the works it cites.

A multitask, multilingual, multimodal evaluation of ChatGPT on reasoning, hallucination, and interactivity

Bang, Y., Cahyawijaya, S., Lee, N., Dai, W., Su, D., Wilie, B., Lovenia, H., Ji, Z., Yu, T., Chung, W., et al · 2023

Closest in time.

Are LLMs all you need for task-oriented dialogue?

Hudeček, V. and Dušek, O · 2023

Closest in time.

Gpteval: Nlg evaluation using gpt-4 with better human alignment

Liu, Y., Iter, D., Xu, Y., Wang, S., Xu, R., and Zhu, C · 2023

Closest in time.

Gpt-4 technical report, 2023

OpenAI · 2023

Closest in time.

Replug: Retrieval-augmented black-box language models

Shi, W., Min, S., Yasunaga, M., Seo, M., James, R., Lewis, M., Zettlemoyer, L., and Yih, W.-t · 2023

Closest in time.

Judging llm-as-a-judge with mt-bench and chatbot arena

Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E., et al · 2023

Closest in time.