2020

Advanced Graph and Sequence Neural Networks for Molecular Property Prediction and Drug Discovery

Wang, Zhengyang, Liu, Meng, Luo, Youzhi et al.

Understand

Properties of molecules are indicative of their functions and thus are useful in many applications.

With the advances of deep learning methods, computational approaches for predicting molecular properties are gaining increasing momentum.
However, there lacks customized and advanced methods and comprehensive tools for this task currently.
Here we develop a suite of comprehensive machine learning methods and tools spanning different computational models, molecular representations, and loss functions for molecular property prediction and drug discovery.

Built on

SMILES, a chemical language and information system. 1. introduction to methodology and encoding rules
Weininger, D · 1988
Earlier work this paper cites.
Support-vector networks
Cortes, C. & Vapnik, V · 1995
Earlier work this paper cites.
The Nature of Statistical Learning Theory (Springer-Verlag, Berlin, Heidelberg, 1995)
Vapnik, V. N · 1995
Earlier work this paper cites.
Long short-term memory
Hochreiter, S. & Schmidhuber, J · 1997
Earlier work this paper cites.
Support vector regression machines
Drucker, H., Burges, C. J., Kaufman, L., Smola, A. J. & Vapnik, V · 1997
Earlier work this paper cites.
Artificial neural networks for computer-based molecular design
Schneider, G. & Wrede, P · 1998
Earlier work this paper cites.
Feature trees: a new molecular similarity measure based on tree matching
Rarey, M. & Dixon, J. S · 1998
Earlier work this paper cites.
Text classification using string kernels
Lodhi, H., Saunders, C., Shawe-Taylor, J., Cristianini, N. & Watkins, C · 2002
Earlier work this paper cites.
Assigning unique keys to chemical compounds for data integration: some interesting counter examples
Neglur, G., Grossman, R. L. & Liu, B · 2005
Earlier work this paper cites.
Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance
Willmott, C. J. & Matsuura, K · 2005
Earlier work this paper cites.
Dimensionality reduction by learning an invariant mapping
Hadsell, R., Chopra, S. & LeCun, Y · 2006
Earlier work this paper cites.
An introduction to ROC analysis
Fawcett, T · 2006
Earlier work this paper cites.
The relationship between Precision-Recall and ROC curves
Davis, J. & Goadrich, M · 2006
Earlier work this paper cites.
Generalized neural-network representation of high-dimensional potential-energy surfaces
Behler, J. & Parrinello, M · 2007
Earlier work this paper cites.
Gaussian approximation potentials: the accuracy of quantum mechanics, without the electrons
Bartók, A. P., Payne, M. C., Kondor, R. & Csányi, G · 2010
Earlier work this paper cites.
Weisfeiler-Lehman graph kernels
Shervashidze, N., Schweitzer, P., van Leeuwen, E. J., Mehlhorn, K. & Borgwardt, K. M · 2011
Earlier work this paper cites.
LIBSVM: A library for support vector machines
Chang, C.-C. & Lin, C.-J · 2011
Earlier work this paper cites.
Machine learning methods for property prediction in chemoinformatics: quo vadis?
Varnek, A. & Baskin, I · 2012
Earlier work this paper cites.
In silico toxicity prediction by support vector machine and smiles representation-based string kernel
Cao, D.-S. et al · 2012
Earlier work this paper cites.
ZINC: a free tool to discover chemistry for biology
Irwin, J. J., Sterling, T., Mysinger, M. M., Bolstad, E. S. & Coleman, R. G · 2012
Earlier work this paper cites.
On representing chemical environments
Bartók, A. P., Kondor, R. & Csányi, G · 2013
Earlier work this paper cites.

Similar

Deep learning as an opportunity in virtual screening
Unterthiner, T. et al · 2014
Cited alongside, same era.
Learning phrase representations using RNN encoder–decoder for statistical machine translation
Cho, K. et al · 2014
Cited alongside, same era.
Convolutional networks on graphs for learning molecular fingerprints
Duvenaud, D. K. et al · 2015
Cited alongside, same era.
Deep neural nets as a method for quantitative structure–activity relationships
Ma, J., Sheridan, R. P., Liaw, A., Dahl, G. E. & Svetnik, V · 2015
Cited alongside, same era.
Deep learning
LeCun, Y., Bengio, Y. & Hinton, G · 2015
Cited alongside, same era.
Batch normalization: accelerating deep network training by reducing internal covariate shift
Ioffe, S. & Szegedy, C · 2015
Cited alongside, same era.
Adam: A method for stochastic optimization
Kingma, D. P. & Ba, J · 2015
Cited alongside, same era.
Molecular graph convolutions: moving beyond fingerprints
Kearnes, S., McCloskey, K., Berndl, M., Pande, V. & Riley, P · 2016
Cited alongside, same era.
Gaussian error linear units (GELUs)
Original
Hendrycks, D. & Gimpel, K · 2016
Cited alongside, same era.
Deep residual learning for image recognition
He, K., Zhang, X., Ren, S. & Sun, J · 2016
Cited alongside, same era.
Layer normalization
Original
Ba, J. L., Kiros, J. R. & Hinton, G. E · 2016
Cited alongside, same era.
Neural message passing for quantum chemistry
Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O. & Dahl, G. E · 2017
Cited alongside, same era.

Then

Junction tree variational autoencoder for molecular graph generation
Jin, W., Barzilay, R. & Jaakkola, T · 2018
Later among the works it cites.
PhysNet: A neural network for predicting energies, forces, dipole moments, and partial charges
Unke, O. T. & Meuwly, M · 2019
Later among the works it cites.
Analyzing learned molecular representations for property prediction
Yang, K. et al · 2019
Later among the works it cites.
SMILES-BERT: large scale unsupervised pre-training for molecular property prediction
Wang, S., Guo, Y., Wang, Y., Sun, H. & Huang, J · 2019
Later among the works it cites.
SMILES Transformer: pre-trained molecular fingerprint for low data drug discovery
Original
Honda, S., Shi, S. & Ueda, H. R · 2019
Later among the works it cites.
BERT: Pre-training of deep bidirectional transformers for language understanding
Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K · 2019
Later among the works it cites.
Directional message passing for molecular graphs
Klicpera, J., Groß, J. & Günnemann, S · 2019
Later among the works it cites.
PyTorch: An imperative style, high-performance deep learning library
Paszke, A. et al · 2019
Later among the works it cites.
Fast graph representation learning with PyTorch Geometric
Fey, M. & Lenssen, J. E · 2019
Later among the works it cites.
Strategies for pre-training graph neural networks
Hu, W. et al · 2019
Later among the works it cites.
A deep learning approach to antibiotic discovery
Stokes, J. M. et al · 2020
Closest in time.
https://www.aicures.mit.edu/tasks
AI Cures · 2020
Closest in time.
Hierarchical inter-message passing for learning on molecular graphs
Fey, M., Yuen, J. G. & Weichert, F · 2020
Closest in time.
RDKit: Open-source cheminformatics
Landrum, G · 2020
Closest in time.
Benchmarking graph neural networks
Original
Dwivedi, V. P., Joshi, C. K., Laurent, T., Bengio, Y. & Bresson, X · 2020
Closest in time.
wav2vec 2.0: a framework for self-supervised learning of speech representations
Baevski, A., Zhou, Y., Mohamed, A.-r. & Auli, M · 2020
Closest in time.
Momentum contrast for unsupervised visual representation learning
He, K., Fan, H., Wu, Y., Xie, S. & Girshick, R · 2020
Closest in time.
A simple framework for contrastive learning of visual representations
Chen, T., Kornblith, S., Norouzi, M. & Hinton, G · 2020
Closest in time.
Robust Deep AUC Maximization: A New Surrogate Loss and Empirical Studies on Medical Image Classification
Original
Yuan, Z., Yan, Y., Sonka, M. & Yang, T · 2020
Closest in time.
Stochastic Optimization of Areas Under Precision-Recall Curves with Provable Convergence
Original
Qi, Q., Luo, Y., Xu, Z.,Ji, S. & Yang, T · 2021
Closest in time.
An End-to-end Machine Learning Library for Deep AUC Optimization
Yuan, Z. & Yang, T · 2021
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…