Fetching the paper…
Reading the bibliography…
The dominant approach to generating from language models subject to some constraint is locally constrained decoding (LCD), incrementally sampling tokens at each time step such that the constraint is never violated.
Unsupervised part of speech inference with particle filters
Gregory Dubbin and Phil Blunsom · 1907
Earlier work this paper cites.
Fine-tuning language models from human preferences
Daniel M. Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B. Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving · 1909
Earlier work this paper cites.
SMILES, a chemical language and information system. 1. introduction to methodology and encoding rules
David Weininger · 1988
Earlier work this paper cites.
Adaptive rejection sampling for Gibbs sampling
Walter R. Gilks and Pascal Wild · 1992
Earlier work this paper cites.
Whole-sentence exponential language models: A vehicle for linguistic-statistical integration
Ronald Rosenfeld, Stanley Chen, and Xiaojin Zhu · 2001
Earlier work this paper cites.
Hierarchical probabilistic neural network language model
Frederic Morin and Yoshua Bengio · 2005
Earlier work this paper cites.
Expectation-based syntactic comprehension
Roger Levy · 2007
Earlier work this paper cites.
Exact and approximate sampling by systematic stochastic search
Vikash Mansinghka, Daniel Roy, Eric Jonas, and Joshua Tenenbaum · 2009
Earlier work this paper cites.
When are probabilistic programs probably computationally tractable
Cameron E Freer, Vikash K Mansinghka, and Daniel M Roy · 2010
Earlier work this paper cites.
A particle filter algorithm for Bayesian wordsegmentation
Benjamin Börschinger and Mark Johnson · 2011
Earlier work this paper cites.
Statistical Distributions
Catherine Forbes, Merran Evans, Nicholas Hastings, and Brian Peacock · 2011
Earlier work this paper cites.
Quantifying the chemical beauty of drugs
G Richard Bickerton, Gaia V Paolini, Jérémy Besnard, Sorel Muresan, and Andrew L Hopkins · 2012
Earlier work this paper cites.
Enumeration of 166 billion organic small molecules in the chemical universe database GDB-17
Lars Ruddigkeit, Ruud Van Deursen, Lorenz C. Blum, and Jean-Louis Reymond · 2012
Earlier work this paper cites.
A log-linear model for unsupervised text normalization
Yi Yang and Jacob Eisenstein · 2013
Earlier work this paper cites.
regex, 2014
Matthew Barnett · 2014
Earlier work this paper cites.
Gumbel-max trick and weighted reservoir sampling, 2014
Tim Vieira · 2014
Earlier work this paper cites.
A Bayesian model for generative transition-based dependency parsing
Jan Buys and Phil Blunsom · 2015
Earlier work this paper cites.
Nested sequential Monte Carlo methods
Christian Naesseth, Fredrik Lindsten, and Thomas Schon · 2015
Earlier work this paper cites.
Six challenges for neural machine translation
Philipp Koehn and Rebecca Knowles · 2017
Earlier work this paper cites.
Neural particle smoothing for sampling from conditional sequence models
Chu-Cheng Lin and Jason Eisner · 2018
Earlier work this paper cites.
Correcting length bias in neural machine translation
Kenton Murray and David Chiang · 2018
Earlier work this paper cites.
Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-SQL task
Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang, and Dragomir Radev · 2018
Earlier work this paper cites.
Empirical analysis of beam search performance degradation in neural sequence models
Eldan Cohen and Christopher Beck · 2019
Earlier work this paper cites.
A general-purpose algorithm for constrained sequential inference
Daniel Deutsch, Shyam Upadhyay, and Dan Roth · 2019
Earlier work this paper cites.
interegular, 2019
MegaIng · 2019
Earlier work this paper cites.
Elements of sequential Monte Carlo
Christian A. Naesseth, Fredrik Lindsten, and Thomas B. Schön · 2019
Earlier work this paper cites.
On NMT search errors and model errors: Cat got your tongue?
Felix Stahlberg and Bill Byrne · 2019
Earlier work this paper cites.
Do you have the right scissors? Tailoring pre-trained language models via Monte-Carlo methods
Ning Miao, Yuxuan Song, Hao Zhou, and Lei Li · 2020
Earlier work this paper cites.
Learning to summarize with human feedback
Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano · 2020
Earlier work this paper cites.
Training verifiers to solve math word problems
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman · 2021
Earlier work this paper cites.
Measuring coding challenge competence with APPS
Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, and Jacob Steinhardt · 2021
Earlier work this paper cites.
GeDi: Generative discriminator guided sequence generation
Ben Krause, Akhilesh Deepak Gotmare, Bryan McCann, Nitish Shirish Keskar, Shafiq Joty, Richard Socher, and Nazneen Fatema Rajani · 2021
Cited alongside, same era.
NeuroLogic Decoding: (un)supervised neural text generation with predicate logic constraints
Ximing Lu, Peter West, Rowan Zellers, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi · 2021
Cited alongside, same era.
WebGPT: Browser-assisted question-answering with human feedback
Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, and John Schulman · 2021
Cited alongside, same era.
Constrained language models yield few-shot semantic parsers
Richard Shin, Christopher Lin, Sam Thomson, Charles Chen, Subhro Roy, Emmanouil Antonios Platanios, Adam Pauls, Dan Klein, Jason Eisner, and Benjamin Van Durme · 2021
Cited alongside, same era.
FUDGE: Controlled text generation with future discriminators
Guiding LLMs the right way: Fast, non-invasive constrained generation
Luca Beurer-Kellner, Marc Fischer, and Martin Vechev · 2024
Later among the works it cites.
XGrammar: Flexible and efficient structured generation engine for large language models
Yixin Dong, Charlie F Ruan, Yaxing Cai, Ruihang Lai, Ziyi Xu, Yilong Zhao, and Tianqi Chen · 2024
Later among the works it cites.
Principled gradient-based MCMC for conditional sampling of text
Li Du, Afra Amini, Lucas Torroba Hennigen, Xinyan Velocity Yu, Holden Lee, Jason Eisner, and Ryan Cotterell · 2024
Later among the works it cites.
Stream of Search (SoS): Learning to search in language
Kanishk Gandhi, Denise HJ Lee, Gabriel Grand, Muxin Liu, Winson Cheng, Archit Sharma, and Noah Goodman · 2024
Later among the works it cites.
Local and global decoding in text generation
Daniel Gareev, Thomas Hofmann, Ezhilmathi Krishnasamy, and Tiago Pimentel · 2024
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Kevin Yang and Dan Klein · 2021
Cited alongside, same era.
Do as I can, not as I say: Grounding language in robotic affordances
Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Daniel Ho, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Eric Jang, Rosario Jauregui Ruano, Kyle Jeffrey, Sally Jesmonth, Nikhil J Joshi, Ryan Julian, Dmitry Kalashnikov, Yuheng Kuang, Kuang-Huei Lee, Sergey Levine, Yao Lu, Linda Luu, Carolina Parada, Peter Pastor, Jornell Quiambao, Kanishka Rao, Jarek Rettinghouse, Diego Reyes, Pierre Sermanet, Nicolas Sievers, Clayton Tan, Alexander Toshev, Vincent Vanhoucke, Fei Xia, Ted Xiao, Peng Xu, Sichun Xu, Mengyuan Yan, and Andy Zeng · 2022
Cited alongside, same era.
Constitutional AI: Harmlessness from AI feedback
Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, Carol Chen, Catherine Olsson, Christopher Olah, Danny Hernandez, Dawn Drain, Deep Ganguli, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jamie Kerr, Jared Mueller, Jeffrey Ladish, Joshua Landau, Kamal Ndousse, Kamile Lukosuite, Liane Lovitt, Michael Sellitto, Nelson Elhage, Nicholas Schiefer, Noemi Mercado, Nova DasSarma, Robert Lasenby, Robin Larson, Sam Ringer, Scott Johnston, Shauna Kravec, Sheer El Showk, Stanislav Fort, Tamera Lanham, Timothy Telleen-Lawton, Tom Conerly, Tom Henighan, Tristan Hume, Samuel R. Bowman, Zac Hatfield-Dodds, Ben Mann, Dario Amodei, Nicholas Joseph, Sam McCandlish, Tom Brown, and Jared Kaplan · 2022
Cited alongside, same era.
SI 𝒳 \mathcal{X} O: Smoothing inference with twisted objectives
Dieterich Lawson, Allan Raventós, Andrew Warrington, and Scott Linderman · 2022
Cited alongside, same era.
Recursive Monte Carlo and variational inference with auxiliary variables
Alexander K. Lew, Marco Cusumano-Towner, and Vikash K. Mansinghka · 2022
Cited alongside, same era.
NeuroLogic A ⋆ esque decoding: Constrained text generation with lookahead heuristics
Ximing Lu, Sean Welleck, Peter West, Liwei Jiang, Jungo Kasai, Daniel Khashabi, Ronan Le Bras, Lianhui Qin, Youngjae Yu, Rowan Zellers, Noah Smith, and Yejin Choi · 2022
Cited alongside, same era.
Controllable text generation with neurally-decomposed oracle
Tao Meng, Sidi Lu, Nanyun Peng, and Kai-Wei Chang · 2022
Cited alongside, same era.
Training language models to follow instructions with human feedback
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe · 2022
Cited alongside, same era.
Amortizing intractable inference in large language models
Edward J. Hu, Moksh Jain, Eric Elmoznino, Younesse Kaddar, Guillaume Lajoie, Yoshua Bengio, and Nikolay Malkin · 2024
Later among the works it cites.
Automata-based constraints for language model decoding
Terry Koo, Frederick Liu, and Luheng He · 2024
Later among the works it cites.
Guidance: a guidance language for controlling large language models
Scott Lundberg, Marco Ribeiro, Richard Edgar, and Harsha Nori · 2024
Later among the works it cites.
AI Controller Interface
Michal Moskal, Madan Musuvathi, and Emre Kıcıman · 2024
Later among the works it cites.
partialsmiles: A validating SMILES parser, with support for incomplete SMILES, 2024
Noel O’Boyle · 2024
Later among the works it cites.
Grammar-aligned decoding
Kanghee Park, Jiayu Wang, Taylor Berg-Kirkpatrick, Nadia Polikarpova, and Loris D’Antoni · 2024
Later among the works it cites.
SynCode: Improving LLM code generation with grammar augmentation
Shubham Ugare, Tarun Suresh, Hangoo Kang, Sasa Misailovic, and Gagandeep Singh · 2024
Later among the works it cites.
Math-Shepherd: Verify and reinforce LLMs step-by-step without human annotations
Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui · 2024
Later among the works it cites.
Deepseek-prover: Advancing theorem proving in LLMs through large-scale synthetic data
Huajian Xin, Daya Guo, Zhihong Shao, Zhizhou Ren, Qihao Zhu, Bo Liu, Chong Ruan, Wenda Li, and Xiaodan Liang · 2024
Later among the works it cites.
Adaptable logical control for large language models
Honghua Zhang, Po-Nien Kung, Masahiro Yoshida, Guy Van den Broeck, and Nanyun Peng · 2024
Later among the works it cites.
Probabilistic inference in language models via twisted sequential Monte Carlo
Stephen Zhao, Rob Brekelmans, Alireza Makhzani, and Roger Baker Grosse · 2024
Later among the works it cites.
SGLang: Efficient execution of structured language model programs
Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark Barrett, and Ying Sheng · 2024
Later among the works it cites.
Planetarium: A rigorous benchmark for translating text to structured planning languages
Max Zuo, Francisco Piedrahita Velez, Xiaochen Li, Michael L. Littman, and Stephen H. Bach · 2024
Later among the works it cites.
Controllable generation via locally constrained resampling
Kareem Ahmed, Kai-Wei Chang, and Guy Van den Broeck · 2025
Closest in time.
CRANE: Reasoning with constrained LLM generation
Debangshu Banerjee, Tarun Suresh, Shubham Ugare, Sasa Misailovic, and Gagandeep Singh · 2025
Closest in time.
On the query complexity of verifier-assisted language generation
Edoardo Botta, Yuchen Li, Aashay Mehta, Jordan T Ash, Cyril Zhang, and Andrej Risteski · 2025
Closest in time.
Pitfalls, subtleties, and techniques in automata-based subword-level constrained generation
Marco Cognetta, David Pohl, Junyoung Lee, and Naoaki Okazaki · 2025
Closest in time.
Sample, don’t search: Rethinking test-time alignment for language models
Gonçalo Faria and Noah A Smith · 2025
Closest in time.
Generating structured outputs from language models: Benchmark and studies, 2025
Saibo Geng, Hudson Cooper, Micha i · 2025
Closest in time.
Tom Kempton and Stuart Burrell · 2025
Closest in time.
Syntactic and semantic control of large language models via sequential Monte Carlo
João Loula, Benjamin LeBrun, Li Du, Ben Lipkin, Clemente Pasti, Gabriel Grand, Tianyu Liu, Yahya Emara, Marjorie Freedman, Jason Eisner, Ryan Cotterell, Vikash Mansinghka, Alex Lew, Tim Vieira, and Tim O’Donnell · 2025
Closest in time.
Type-constrained code generation with language models
Niels Mündler, Jingxuan He, Hao Wang, Koushik Sen, Dawn Song, and Martin Vechev · 2025
Closest in time.
Flexible and efficient grammar-constrained decoding
Kanghee Park, Timothy Zhou, and Loris D’Antoni · 2025
Closest in time.
Isha Puri, Shivchander Sudalairaj, Guangxuan Xu, Kai Xu, and Akash Srivastava · 2025
Closest in time.
IterGen: Iterative structured LLM generation
Shubham Ugare, Rohan Gumaste, Tarun Suresh, Gagandeep Singh, and Sasa Misailovic · 2025
Closest in time.
Efficient and asymptotically unbiased constrained decoding for large language models
Haotian Ye, Himanshu Jain, Chong You, Ananda Theertha Suresh, Haowei Lin, James Zou, and Felix Yu · 2025
Closest in time.