Fetching the paper…
Reading the bibliography…
Large-scale sequence modeling has sparked rapid advances that now extend into biology and genomics.
Prefix sums and their applications
Blelloch, G. E · 1990
Earlier work this paper cites.
Long short-term memory
Hochreiter, S. and Schmidhuber, J · 1997
Earlier work this paper cites.
Qualitatively predicting acetylation and methylation areas in dna sequences
Phaml, T. H., Tran, D. H., Ho, T. B., Satou, K., and Valiente, G · 2005
Earlier work this paper cites.
Matplotlib: A 2d graphics environment
Hunter, J. D · 2007
Earlier work this paper cites.
Genome reference consortium human build 37 (grch37)
Consortium, G. R. et al · 2009
Earlier work this paper cites.
Scikit-learn: Machine learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E · 2011
Earlier work this paper cites.
Adam: A method for stochastic optimization
Kingma, D. P. and Ba, J · 2014
Earlier work this paper cites.
Predicting the sequence specificities of dna-and rna-binding proteins by deep learning
Alipanahi, B., Delong, A., Weirauch, M. T., and Frey, B. J · 2015
Earlier work this paper cites.
Predicting effects of noncoding variants with deep learning–based sequence model
Zhou, J. and Troyanskaya, O. G · 2015
Earlier work this paper cites.
Parallelizing linear recurrent neural nets over sequence length
Martin, E. and Cundy, C · 2017
Earlier work this paper cites.
Semi-supervised sequence tagging with bidirectional language models
Peters, M. E., Ammar, W., Bhagavatula, C., and Power, R · 2017
Earlier work this paper cites.
Searching for activation functions
Ramachandran, P., Zoph, B., and Le, Q. V · 2017
Earlier work this paper cites.
Reverse-complement parameter sharing improves deep learning models for genomics
Shrikumar, A., Greenside, P., and Kundaje, A · 2017
Earlier work this paper cites.
Attention is all you need
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I · 2017
Earlier work this paper cites.
Bert: Pre-training of deep bidirectional transformers for language understanding
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K · 2018
Earlier work this paper cites.
Developmental enhancers and chromosome topology
Furlong, E. E. M. and Levine, M · 2018
Earlier work this paper cites.
Simple tricks of convolutional neural network architectures improve dna–protein binding prediction
Cao, Z. and Zhang, S · 2019
Earlier work this paper cites.
PyTorch Lightning, March 2019
Falcon, W. and The PyTorch Lightning team · 2019
Earlier work this paper cites.
Deepromoter: robust promoter predictor using deep learning
Oubounyt, M., Louadi, Z., Tayara, H., and Chong, K. T · 2019
Earlier work this paper cites.
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S · 2019
Earlier work this paper cites.
Factornet: a deep learning framework for predicting cell type specific transcription factor binding from nucleotide-resolution sequential data
Quang, D. and Xie, X · 2019
Cited alongside, same era.
Triton: an intermediate language and compiler for tiled neural network computations
Tillet, P., Kung, H.-T., and Cox, D · 2019
Cited alongside, same era.
Splicefinder: ab initio prediction of splice sites using convolutional neural network
Wang, R., Wang, Z., Wang, J., and Li, S · 2019
Cited alongside, same era.
Huggingface’s transformers: State-of-the-art natural language processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al · 2019
Cited alongside, same era.
Hydra - a framework for elegantly configuring complex applications
Yadan, O · 2019
Cited alongside, same era.
A deep learning framework for enhancer prediction using word embedding and sequence generation
Geng, Q., Yang, R., and Zhang, L · 2022
Later among the works it cites.
On the parameterization and initialization of diagonal state space models
Gu, A., Goel, K., Gupta, A., and Ré, C · 2022
Later among the works it cites.
Diagonal state spaces are as effective as structured state spaces
Gupta, A., Gu, A., and Berant, J · 2022
Later among the works it cites.
Simplified state space layers for sequence modeling
Smith, J. T., Warrington, A., and Linderman, S. W · 2022
Later among the works it cites.
Wang, J., Yan, J. N., Gu, A., and Rush, A. M · 2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Array programming with NumPy
Harris, C. R., Millman, K. J., van der Walt, S. J., Gommers, R., Virtanen, P., Cournapeau, D., Wieser, E., Taylor, J., Berg, S., Smith, N. J., Kern, R., Picus, M., Hoyer, S., van Kerkwijk, M. H., Brett, M., Haldane, A., del Río, J. F., Wiebe, M., Peterson, P., Gérard-Marchant, P., Sheppard, K., Reddy, T., Weckesser, W., Abbasi, H., Gohlke, C., and Oliphant, T. E · 2020
Cited alongside, same era.
pandas-dev/pandas: Pandas, February 2020
pandas development team, T · 2020
Cited alongside, same era.
Transformer protein language models are unsupervised structure learners
Rao, R., Meier, J., Sercu, T., Ovchinnikov, S., and Rives, A · 2020
Cited alongside, same era.
A simple new approach to variable selection in regression, with application to genetic fine mapping
Wang, G., Sarkar, A., Carbonetto, P., and Stephens, M · 2020
Cited alongside, same era.
Big bird: Transformers for longer sequences
Zaheer, M., Guruganesh, G., Dubey, K. A., Ainslie, J., Alberti, C., Ontanon, S., Pham, P., Ravula, A., Wang, Q., Yang, L., et al · 2020
Cited alongside, same era.
Effective gene expression prediction from sequence by integrating long-range interactions
Avsec, Ž., Agarwal, V., Visentin, D., Ledsam, J. R., Grabska-Barwinska, A., Taylor, K. R., Assael, Y., Jumper, J., Kohli, P., and Kelley, D. R · 2021
Cited alongside, same era.
Dnabert: pre-trained bidirectional encoder representations from transformers model for dna-language in genome
Ji, Y., Zhou, Z., Liu, H., and Davuluri, R. V · 2021
Cited alongside, same era.
Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al · 2023
Later among the works it cites.
The nucleotide transformer: Building and evaluating robust foundation models for human genomics
Dalla-Torre, H., Gonzalez, L., Mendoza-Revilla, J., Carranza, N. L., Grzywaczewski, A. H., Oteri, F., Dallago, C., Trop, E., de Almeida, B. P., Sirelkhatim, H., et al · 2023
Later among the works it cites.
Gena-lm: A family of open-source foundational models for long dna sequences
Fishman, V., Kuratov, Y., Petrov, M., Shmelev, A., Shepelin, D., Chekanov, N., Kardymon, O., and Burtsev, M · 2023
Later among the works it cites.
Genomic benchmarks: a collection of datasets for genomic sequence classification
Grešová, K., Martinek, V., Čechák, D., Šimeček, P., and Alexiou, P · 2023
Later among the works it cites.
Mamba: Linear-time sequence modeling with selective state spaces
Gu, A. and Dao, T · 2023
Later among the works it cites.
A self-supervised deep learning method for data-efficient training in genomics
Gündüz, H. A., Binder, M., To, X.-Y., Mreches, R., Bischl, B., McHardy, A. C., Münch, P. C., and Rezaei, M · 2023
Later among the works it cites.
Evolutionary-scale prediction of atomic-level protein structure with a language model
Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., Verkuil, R., Kabeli, O., Shmueli, Y., et al · 2023
Later among the works it cites.
Large language models generate functional protein sequences across diverse families
Madani, A., Krause, B., Greene, E. R., Subramanian, S., Mohr, B. P., Holton, J. M., Olmos Jr, J. L., Xiong, C., Sun, Z. Z., Socher, R., et al · 2023
Later among the works it cites.
Hyenadna: Long-range genomic sequence modeling at single nucleotide resolution
Nguyen, E., Poli, M., Faizi, M., Thomas, A., Birch-Sykes, C., Wornow, M., Patel, A., Rabideau, C., Massaroli, S., Bengio, Y., et al · 2023
Later among the works it cites.
Hyena hierarchy: Towards larger convolutional language models
Poli, M., Massaroli, S., Nguyen, E., Fu, D. Y., Dao, T., Baccus, S., Bengio, Y., Ermon, S., and Ré, C · 2023
Later among the works it cites.
Gemini: a family of highly capable multimodal models
Team, G., Anil, R., Borgeaud, S., Wu, Y., Alayrac, J.-B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A. M., Hauth, A., et al · 2023
Later among the works it cites.
Advancing dna language models: The genomics long-range benchmark
Trop, E., Kao, C.-H., Polen, M., Schiff, Y., de Almeida, B. P., Gokaslan, A., Pierrot, T., and Kuleshov, V · 2023
Later among the works it cites.
Dnabert-2: Efficient foundation model and benchmark for multi-species genome
Zhou, Z., Ji, Y., Li, W., Dutta, P., Davuluri, R., and Liu, H · 2023
Later among the works it cites.
Vision mamba: Efficient visual representation learning with bidirectional state space model
Zhu, L., Liao, B., Zhang, Q., Wang, X., Liu, W., and Wang, X · 2024
Closest in time.