Built on

“The Basic AI Drives”
Stephen. Omohundro · 2008
Earlier work this paper cites.
“Superintelligence: Paths, Dangers, Strategies”
Nick Bostrom · 2014
Earlier work this paper cites.
“Deep Reinforcement Learning from Human Preferences”
Paul Christiano et al · 2017
Earlier work this paper cites.
“Human compatible: Artificial intelligence and the problem of control”
Stuart Russell · 2019
Earlier work this paper cites.
“A General Language Assistant as a Laboratory for Alignment”
Original
Amanda Askell et al · 2021
Earlier work this paper cites.
“BBQ: A Hand-Built Bias Benchmark for Question Answering”
Original
Alicia Parrish et al · 2021
Earlier work this paper cites.
“Training language models to follow instructions with human feedback”
Long Ouyang et al · 2022
Earlier work this paper cites.
“Solving math word problems with process-and outcome-based feedback”
Original
Jonathan Uesato et al · 2022
Earlier work this paper cites.
“Learning by Distilling Context”
Original
Charlie Snell, Dan Klein and Ruiqi Zhong · 2022
Earlier work this paper cites.
“Constitutional AI: Harmlessness from AI Feedback”
Original
Yuntao Bai et al · 2022
Earlier work this paper cites.
“GPT-4 technical report”
Original
Josh Achiam et al · 2023
Earlier work this paper cites.
“Universal and transferable adversarial attacks on aligned language models”
Original
Andy Zou et al · 2023
Earlier work this paper cites.
“Self-Refine: Iterative Refinement with Self-Feedback”
Original
Aman Madaan et al · 2023
Earlier work this paper cites.

Similar

“Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies”
Original
Liangming Pan et al · 2023
Cited alongside, same era.
“Generating sequences by learning to self-correct”
Sean Welleck et al · 2023
Cited alongside, same era.
“Large Language Model Programs”
Original
Imanol Schlag et al · 2023
Cited alongside, same era.
“Learning to Reason with LLMs”, 2024
OpenAI · 2024
Cited alongside, same era.
“The LLaMA 3 herd of models”
Original
Abhimanyu Dubey et al · 2024
Cited alongside, same era.
“Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context”

Then

“Jailbreaking Black Box Large Language Models in Twenty Queries”
Original
Patrick Chao et al · 2024
Closest in time.
“JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models”
Original
Patrick Chao et al · 2024
Closest in time.
“Refusal-Trained LLMs Are Easily Jailbroken As Browser Agents”
Original
Priyanshu Kumar et al · 2024
Closest in time.
“O1 System Card”, 2024
OpenAI · 2024
Closest in time.
“GPT-4o System Card”, 2024
OpenAI · 2024
Closest in time.
“Model Card Addendum: Claude 3.5 Haiku and Upgraded Claude 3.5 Sonnet”, 2024

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2024

Deliberative Alignment: Reasoning Enables Safer Language Models

Guan, Melody Y., Joglekar, Manas, Wallace, Eric et al.

Understand

As large-scale language models increasingly impact safety-critical domains, ensuring their reliable adherence to well-defined principles remains a fundamental challenge.

We introduce Deliberative Alignment, a new paradigm that directly teaches the model safety specifications and trains it to explicitly recall and accurately reason over the specifications before answering.
We used this approach to align OpenAI's o-series models, and achieved highly precise adherence to OpenAI's safety policies, without requiring human-written chain-of-thoughts or answers.
Deliberative Alignment pushes the Pareto frontier by simultaneously increasing robustness to jailbreaks while decreasing overrefusal rates, and also improves out-of-distribution generalization.

Built on

“The Basic AI Drives”
Stephen. Omohundro · 2008
Earlier work this paper cites.
“Superintelligence: Paths, Dangers, Strategies”
Nick Bostrom · 2014
Earlier work this paper cites.
“Deep Reinforcement Learning from Human Preferences”
Paul Christiano et al · 2017
Earlier work this paper cites.
“Human compatible: Artificial intelligence and the problem of control”
Stuart Russell · 2019
Earlier work this paper cites.
“A General Language Assistant as a Laboratory for Alignment”
Original
Amanda Askell et al · 2021
Earlier work this paper cites.
“BBQ: A Hand-Built Bias Benchmark for Question Answering”
Original
Alicia Parrish et al · 2021
Earlier work this paper cites.
“Training language models to follow instructions with human feedback”
Long Ouyang et al · 2022
Earlier work this paper cites.
“Solving math word problems with process-and outcome-based feedback”
Original
Jonathan Uesato et al · 2022
Earlier work this paper cites.
“Learning by Distilling Context”
Original
Charlie Snell, Dan Klein and Ruiqi Zhong · 2022
Earlier work this paper cites.
“Constitutional AI: Harmlessness from AI Feedback”
Original
Yuntao Bai et al · 2022
Earlier work this paper cites.
“GPT-4 technical report”
Original
Josh Achiam et al · 2023
Earlier work this paper cites.
“Universal and transferable adversarial attacks on aligned language models”
Original
Andy Zou et al · 2023
Earlier work this paper cites.
“Self-Refine: Iterative Refinement with Self-Feedback”
Original
Aman Madaan et al · 2023
Earlier work this paper cites.