Regression Transformer: Concurrent sequence regression and generation for molecular language modeling

Understand

Despite significant progress of generative models in the natural sciences, their controllability remains challenging.

• One fundamentally missing aspect of molecular or protein generative models is an inductive bias that can reflect continuous properties of interest.
• To that end, we propose the Regression Transformer (RT), a novel method that abstracts regression as a conditional sequence modeling problem.
• This introduces a new paradigm of multitask language models which seamlessly bridge sequence regression and conditional sequence generation.