Fetching the paper…
Reading the bibliography…
In this work we introduce RITA: a suite of autoregressive generative models for protein sequences, with up to 1.2 billion parameters, trained on over 280 million protein sequences belonging to the UniRef-100 database.
Evaluating protein transfer learning with tape, 2019
Rao, R., Bhattacharya, N., Thomas, N., Duan, Y., Chen, X., Canny, J., Abbeel, P., and Song, Y. S · 1906
Earlier work this paper cites.
Design by directed evolution
Arnold, F. H · 1998
Earlier work this paper cites.
The swiss-prot protein sequence database and its supplement trembl in 2000
Bairoch, A. and Apweiler, R · 2000
Earlier work this paper cites.
Adam: A method for stochastic optimization
Kingma, D. P. and Ba, J · 2015
Earlier work this paper cites.
The coming of age of de novo protein design
Huang, P.-S., Boyken, S. E., and Baker, D · 2016
Earlier work this paper cites.
Deep generative models of genetic variation capture mutation effects
Riesselman, A. J., Ingraham, J. B., and Marks, D. S · 2017
Earlier work this paper cites.
Attention is all you need
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L. u., and Polosukhin, I · 2017
Earlier work this paper cites.
Clustering huge protein sequence sets in linear time
Steinegger, M. and Söding, J · 2018
Earlier work this paper cites.
Unified rational protein engineering with sequence-only deep representation learning
Alley, E. C., Khimulya, G., Biswas, S., AlQuraishi, M., and Church, G. M · 2019
Earlier work this paper cites.
Generative models for graph-based protein design
Ingraham, J., Garg, V., Barzilay, R., and Jaakkola, T · 2019
Earlier work this paper cites.
Low-n protein engineering with data-efficient deep learning
Biswas, S., Khimulya, G., Alley, E. C., Esvelt, K. M., and Church, G. M · 2020
Earlier work this paper cites.
Language models are few-shot learners
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al · 2020
Earlier work this paper cites.
Scaling laws for neural language models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D · 2020
Earlier work this paper cites.
ProGen: Language Modeling for Protein Generation
Madani, A., McCann, B., Naik, N., Keskar, N. S., Anand, N., Eguchi, R. R., Huang, P.-S., and Socher, R · 2020
Earlier work this paper cites.
Mgnify: the microbiome analysis resource in 2020
Mitchell, A. L., Almeida, A., Beracochea, M., Boland, M. A., Burgin, J., Cochrane, G., Crusoe, M. R., Kale, V., Potter, S. C., Richardson, L. J., Sakharova, E. A., Scheremetjew, M., Korobeynikov, A. I., Shlemov, A., Kunyavskaya, O., Lapidus, A. L., and Finn, R. D · 2020
Cited alongside, same era.
Transformer protein language models are unsupervised structure learners, December 2020
Rao, R., Meier, J., Sercu, T., Ovchinnikov, S., and Rives, A · 2020
Cited alongside, same era.
UniProt: the universal protein knowledgebase in 2021
The UniProt Consortium · 2020
Cited alongside, same era.
Twitter post: Adding a big enough number for residue index feature is enough to model hetero-complex using alphafold (green&cyan crystal structure magenta: predicted model w/ residue_index modification), July 2021
Baek, M · 2021
Cited alongside, same era.
ProteinBERT: A universal deep-learning model of protein sequence and function, May 2021
Brandes, N., Ofer, D., Peleg, Y., Rappoport, N., and Linial, M · 2021
MSA Transformer, August 2021
Rao, R., Liu, J., Verkuil, R., Meier, J., Canny, J. F., Abbeel, P., Sercu, T., and Rives, A · 2021
Later among the works it cites.
Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences
Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., and Fergus, R · 2021
Later among the works it cites.
Proteinfer: deep networks for protein functional inference
Sanderson, T., Bileschi, M. L., Belanger, D., and Colwell, L · 2021
Later among the works it cites.
Roformer: Enhanced transformer with rotary position embedding
Su, J., Lu, Y., Pan, S., Wen, B., and Liu, Y · 2021
Later among the works it cites.
Controllable neural text generation
Weng, L · 2021
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Single-sequence protein structure prediction using language models from deep learning, August 2021
Chowdhury, R., Bouatta, N., Biswas, S., Rochereau, C., Church, G. M., Sorger, P. K., and AlQuraishi, M · 2021
Cited alongside, same era.
ProtTrans: Towards Cracking the Language of Lifes Code Through Self-Supervised Deep Learning and High Performance Computing
Elnaggar, A., Heinzinger, M., Dallago, C., Rehawi, G., Wang, Y., Jones, L., Gibbs, T., Feher, T., Angerer, C., Steinegger, M., Bhowmik, D., and Rost, B · 2021
Cited alongside, same era.
Disease variant prediction with deep generative models of evolutionary data
Frazer, J., Notin, P., Dias, M., Gomez, A. N., Min, J. K., Brock, K. P., Gal, Y., and Marks, D. S · 2021
Cited alongside, same era.
Highly accurate protein structure prediction with alphafold
Jumper, J. M., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Zídek, A., Potapenko, A., Bridgland, A., Meyer, C., Kohl, S. A. A., Ballard, A., Cowie, A., Romera-Paredes, B., Nikolov, S., Jain, R., Adler, J., Back, T., Petersen, S., Reiman, D. A., Clancy, E., Zielinski, M., Steinegger, M., Pacholska, M., Berghammer, T., Bodenstein, S., Silver, D., Vinyals, O., Senior, A. W., Kavukcuoglu, K., Kohli, P., and Hassabis, D · 2021
Cited alongside, same era.
The power of scale for parameter-efficient prompt tuning
Lester, B., Al-Rfou, R., and Constant, N · 2021
Cited alongside, same era.
Deep neural language modeling enables functional protein generation across families
Madani, A., Krause, B., Greene, E. R., Subramanian, S., Mohr, B. P., Holton, J. M., Olmos, J. L., Xiong, C., Sun, Z. Z., Socher, R., Fraser, J. S., and Naik, N · 2021
Cited alongside, same era.
Language models enable zero-shot prediction of the effects of mutations on protein function
Meier, J., Rao, R., Verkuil, R., Liu, J., Sercu, T., and Rives, A · 2021
Cited alongside, same era.
Twitter post: Alphafold2 can also predict heterocomplexes. all you have to do is input the two sequences you want to predict and connect them with a long linker., July 2021
Yoshitaka, M · 2021
Later among the works it cites.
Decoding methods in neural language generation: A survey
Zarrieß, S., Voigt, H., and Schüz, S · 2021
Later among the works it cites.
A deep unsupervised language model for protein design
Ferruz, N., Schmidt, S., and Höcker, B · 2022
Closest in time.
Predictability and surprise in large generative models
Ganguli, D., Hernandez, D., Lovitt, L., DasSarma, N., Henighan, T. J., Jones, A., Joseph, N., Kernion, J., Mann, B., Askell, A., Bai, Y., Chen, A., Conerly, T., Drain, D., Elhage, N., Showk, S. E., Fort, S., Hatfield-Dodds, Z., Johnston, S., Kravec, S., Nanda, N., Ndousse, K., Olsson, C., Amodei, D., Amodei, D., Brown, T. B., Kaplan, J., McCandlish, S., Olah, C., and Clark, J · 2022
Closest in time.
Training compute-optimal large language models
Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., de Las Casas, D., Hendricks, L. A., Welbl, J., Clark, A., Hennigan, T., Noland, E., Millican, K., van den Driessche, G., Damoc, B., Guy, A., Osindero, S., Simonyan, K., Elsen, E., Rae, J. W., Vinyals, O., and Sifre, L · 2022
Closest in time.
Design in the dark: Learning deep generative models for de novo protein design
Moffat, L., Kandathil, S. M., and Jones, D. T · 2022
Closest in time.
Notin, P., Dias, M., Frazer, J., Marchena-Hurtado, J., Gomez, A. N., Marks, D. S., and Gal, Y · 2022
Closest in time.
Protein design and variant prediction using autoregressive generative models
Shin, J.-E., Riesselman, A. J., Kollasch, A. W., McMahon, C., Simon, E., Sander, C., Manglik, A., Kruse, A. C., and Marks, D. S · 2041
Closest in time.