2024

Text-Guided Molecule Generation with Diffusion Language Model

Gong, Haisong, Liu, Qiang, Wu, Shu et al.

Understand

Text-guided molecule generation is a task where molecules are generated to match specific textual descriptions.

Recently, most existing SMILES-based molecule generation methods rely on an autoregressive architecture.
In this work, we propose the Text-Guided Molecule Generation with Diffusion Language Model (TGM-DLM), a novel approach that leverages diffusion models to address the limitations of autoregressive methods.
TGM-DLM updates token embeddings within the SMILES string collectively and iteratively, using a two-phase diffusion generation process.

Built on

SMILES, a chemical language and information system. 1. Introduction to methodology and encoding rules
Weininger, D. 1988 · 1988
Earlier work this paper cites.
SMILES. 2. Algorithm for generation of unique SMILES notation
Weininger, D.; Weininger, A.; and Weininger, J. L. 1989 · 1989
Earlier work this paper cites.
A new algorithm for data compression
Gage, P. 1994 · 1994
Earlier work this paper cites.
Molecular docking and structure-based drug design strategies
Ferreira, L. G.; Dos Santos, R. N.; Oliva, G.; and Andricopulo, A. D. 2015 · 2015
Earlier work this paper cites.
Adam: A Method for Stochastic Optimization
Kingma, D. P.; and Ba, J. 2015 · 2015
Earlier work this paper cites.
ZINC 15–ligand discovery for everyone
Sterling, T.; and Irwin, J. J. 2015 · 2015
Earlier work this paper cites.
Molecular design of benzodithiophene-based organic photovoltaic materials
Yao, H.; Ye, L.; Zhang, H.; Li, S.; Zhang, S.; and Hou, J. 2016 · 2016
Earlier work this paper cites.
A high-throughput framework for determining adsorption energies on solid surfaces
Montoya, J. H.; and Persson, K. A. 2017 · 2017
Earlier work this paper cites.
Attention is All you Need
Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, L.; and Polosukhin, I. 2017 · 2017
Earlier work this paper cites.
Syntax-directed variational autoencoder for molecule generation
Dai, H.; Tian, Y.; Dai, B.; Skiena, S.; and Song, L. 2018 · 2018
Earlier work this paper cites.
Automatic chemical design using a data-driven continuous representation of molecules
Gómez-Bombarelli, R.; Wei, J. N.; Duvenaud, D.; Hernández-Lobato, J. M.; Sánchez-Lengeling, B.; Sheberla, D.; Aguilera-Iparraguirre, J.; Hirzel, T. D.; Adams, R. P.; and Aspuru-Guzik, A. 2018 · 2018
Earlier work this paper cites.
Generating focused molecule libraries for drug discovery with recurrent neural networks
Segler, M. H.; Kogej, T.; Tyrchan, C.; and Waller, M. P. 2018 · 2018
Earlier work this paper cites.
Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation
You, J.; Liu, B.; Ying, Z.; Pande, V. S.; and Leskovec, J. 2018 · 2018
Earlier work this paper cites.

Similar

SciBERT: A Pretrained Language Model for Scientific Text
Beltagy, I.; Lo, K.; and Cohan, A. 2019 · 2019
Cited alongside, same era.
GPT-3: Its nature, scope, limits, and consequences
Floridi, L.; and Chiriatti, M. 2020 · 2020
Cited alongside, same era.
Bidirectional molecule generation with recurrent neural networks
Grisoni, F.; Moret, M.; Lingwood, R.; and Schneider, G. 2020 · 2020
Cited alongside, same era.
Denoising Diffusion Probabilistic Models
Ho, J.; Jain, A.; and Abbeel, P. 2020 · 2020
Cited alongside, same era.
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
Lewis, M.; Liu, Y.; Goyal, N.; Ghazvininejad, M.; Mohamed, A.; Levy, O.; Stoyanov, V.; and Zettlemoyer, L. 2020 · 2020
Cited alongside, same era.
Exploring the limits of transfer learning with a unified text-to-text transformer
Raffel, C.; Shazeer, N.; Roberts, A.; Lee, K.; Narang, S.; Matena, M.; Zhou, Y.; Li, W.; and Liu, P. J. 2020 · 2020
Cited alongside, same era.
Text2Mol: Cross-Modal Molecule Retrieval with Natural Language Queries
Edwards, C.; Zhai, C.; and Ji, H. 2021 · 2021
Cited alongside, same era.
DiffWave: A Versatile Diffusion Model for Audio Synthesis
Kong, Z.; Ping, W.; Huang, J.; Zhao, K.; and Catanzaro, B. 2021 · 2021
Cited alongside, same era.
Artificial intelligence in drug discovery and development
Paul, D.; Sanap, G.; Shenoy, S.; Kalyane, D.; Kalia, K.; and Tekade, R. K. 2021 · 2021
Cited alongside, same era.
Translation between molecules and natural language
Original
Edwards, C.; Lai, T.; Ros, K.; Honke, G.; Cho, K.; and Ji, H. 2022 · 2022
Cited alongside, same era.
Neural scaling of deep chemical models
Frey, N.; Soklaski, R.; Axelrod, S.; Samsi, S.; Gomez-Bombarelli, R.; Coley, C.; and Gadepally, V. 2022 · 2022
Cited alongside, same era.
Diffuseq: Sequence to sequence text generation with diffusion models
Original
Gong, S.; Li, M.; Feng, J.; Wu, Z.; and Kong, L. 2022 · 2022
Cited alongside, same era.

Then

Chemformer: a pre-trained transformer for computational chemistry
Irwin, R.; Dimitriadis, S.; He, J.; and Bjerrum, E. J. 2022 · 2022
Later among the works it cites.
Diffusion-lm improves controllable text generation
Li, X.; Thickstun, J.; Gulrajani, I.; Liang, P. S.; and Hashimoto, T. B. 2022 · 2022
Later among the works it cites.
Multi-modal molecule structure-text model for text-based retrieval and editing
Original
Liu, S.; Nie, W.; Wang, C.; Lu, J.; Qiao, Z.; Liu, L.; Tang, J.; Xiao, C.; and Anandkumar, A. 2022 · 2022
Later among the works it cites.
Graph neural networks for materials science and chemistry
Reiser, P.; Neubert, M.; Eberhard, A.; Torresi, L.; Zhou, C.; Shao, C.; Metni, H.; van Hoesel, C.; Schopmans, H.; Sommer, T.; et al. 2022 · 2022
Later among the works it cites.
High-resolution image synthesis with latent diffusion models
Rombach, R.; Blattmann, A.; Lorenz, D.; Esser, P.; and Ommer, B. 2022 · 2022
Later among the works it cites.
Diffusion models: A comprehensive survey of methods and applications
Original
Yang, L.; Zhang, Z.; Song, Y.; Hong, S.; Xu, R.; Zhao, Y.; Shao, Y.; Zhang, W.; Cui, B.; and Yang, M.-H. 2022 · 2022
Later among the works it cites.
A deep-learning system bridging molecule structure and biomedical text with comprehension comparable to human professionals
Zeng, Z.; Yao, Y.; Liu, Z.; and Sun, M. 2022 · 2022
Later among the works it cites.
Featurizations matter: a multiview contrastive learning approach to molecular pretraining
Original
Zhu, Y.; Chen, D.; Du, Y.; Wang, Y.; Liu, Q.; and Wu, S. 2022 · 2022
Later among the works it cites.
Sparks of artificial general intelligence: Early experiments with gpt-4
Original
Bubeck, S.; Chandrasekaran, V.; Eldan, R.; Gehrke, J.; Horvitz, E.; Kamar, E.; Lee, P.; Lee, Y. T.; Li, Y.; Lundberg, S.; et al. 2023 · 2023
Later among the works it cites.
Uncovering Neural Scaling Laws in Molecular Representation Learning
Chen, D.; Zhu, Y.; Zhang, J.; Du, Y.; Li, Z.; Liu, Q.; Wu, S.; and Wang, L. 2023 · 2023
Later among the works it cites.
DecompDiff: Diffusion Models with Decomposed Priors for Structure-Based Drug Design
Guan, J.; Zhou, X.; Yang, Y.; Bao, Y.; Peng, J.; Ma, J.; Liu, Q.; Wang, L.; and Gu, Q. 2023 · 2023
Later among the works it cites.
MolGPT: molecular generation using a transformer-decoder model
Bagal, V.; Aggarwal, R.; Vinod, P.; and Priyakumar, U. D. 2021 · 2076
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…