Guiding LLMs The Right Way: Fast, Non-Invasive Constrained Generation

Regular expressions and state graphs for automata

McNaughton, R. and Yamada, H · 1960

Earlier work this paper cites.

Regular expression search algorithm

Thompson, K · 1968

Earlier work this paper cites.

Introduction to the CoNLL-2002 shared task: Language-independent named entity recognition

Tjong Kim Sang, E. F · 2002

Earlier work this paper cites.

Neural machine translation of rare words with subword units

Sennrich, R., Haddow, B., and Birch, A · 2016

Earlier work this paper cites.

Attention is all you need

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I · 2017

Earlier work this paper cites.

SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing

Kudo, T. and Richardson, J · 2018

Earlier work this paper cites.

Huggingface’s transformers: State-of-the-art natural language processing

Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al · 2019

Earlier work this paper cites.

Language models are few-shot learners

Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., and Amodei, D · 2020

Cited alongside, same era.

Evaluating large language models trained on code

Chen, M., Tworek, J., Jun, H., Yuan, Q., de Oliveira Pinto, H. P., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., Ray, A., Puri, R., Krueger, G., Petrov, M., Khlaaf, H., Sastry, G., Mishkin, P., Chan, B., Gray, S., Ryder, N., Pavlov, M., Power, A., Kaiser, L., Bavarian, M., Winter, C., Tillet, P., Such, F. P., Cummings, D., Plappert, M., Chantzis, F., Barnes, E., Herbert-Voss, A., Guss, W. H., Nichol, A., Paino, A., Tezak, N., Tang, J., Babuschkin, I., Balaji, S., Jain, S., Saunders, W., Hesse, C., Carr, A. N., Leike, J., Achiam, J., Misra, V., Morikawa, E., Radford, A., Knight, M., Brundage, M., Murati, M., Mayer, K., Welinder, P., McGrew, B., Amodei, D., McCandlish, S., Sutskever, I., and Zaremba, W · 2021

Cited alongside, same era.

Training verifiers to solve math word problems

Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al · 2021

Cited alongside, same era.

PICARD: Parsing incrementally for constrained auto-regressive decoding from language models

Scholak, T., Schucher, N., and Bahdanau, D · 2021

Prompting is programming: A query language for large language models

Beurer-Kellner, L., Fischer, M., and Vechev, M. T · 2023

Later among the works it cites.

Accelerating large language model decoding with speculative sampling

Chen, C., Borgeaud, S., Irving, G., Lespiau, J., Sifre, L., and Jumper, J · 2023

Later among the works it cites.

Mistral 7b

Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., Casas, D. d. l., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., et al · 2023

Later among the works it cites.

GPT-4 technical report

OpenAI · 2023

Later among the works it cites.

Efficient guided generation for large language models

Willard, B. T. and Louf, R · 2023

Later among the works it cites.

Mixtral of experts

Jiang, A. Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D. S., de Las Casas, D., Hanna, E. B., Bressand, F., Lengyel, G., Bour, G., Lample, G., Lavaud, L. R., Saulnier, L., Lachaux, M., Stock, P., Subramanian, S., Yang, S., Antoniak, S., Scao, T. L., Gervet, T., Lavril, T., Wang, T., Lacroix, T., and Sayed, W. E · 2024