Epigenomic language models powered by Cerebras

An integrated encyclopedia of DNA elements in the human genome

I. Dunham et al · 2012

Earlier work this paper cites.

Absence of a simple code: how transcription factors read the genome

M. Slattery, T. Zhou, L. Yang, A. Dantas Machado, R. Gordân, and R. Rohs · 2014

Earlier work this paper cites.

Integrative analysis of 111 reference human epigenomes

A. Kundaje et al · 2015

Earlier work this paper cites.

Jointly characterizing epigenetic dynamics across multiple human cell types

Original

Y. Zhang, L. An, F. Yue, and R. C. Hardison · 2016

Earlier work this paper cites.

The Human Transcription Factors

S. Lambert, A. Jolma, L. Campitelli, P. Das, Y. Yin, M. Albu, X. Chen, J. Taipale, T. Hughes, and M. Weirauch · 2018

Earlier work this paper cites.

Prediction of enhancer-promoter interactions via natural language processing

W. Zeng, M. Wu, and R. Jiang · 2018

Earlier work this paper cites.

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Original

J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova · 2019

Cited alongside, same era.

Accurate prediction of cell type-specific transcription factor binding

J. Keilwagen, S. Posch, and J. Grau · 2019

Cited alongside, same era.

Anchor: trans-cell type prediction of transcription factor binding sites

H. Li, D. Quang, and Y. Guan · 2019

Cited alongside, same era.

RoBERTa: A Robustly Optimized BERT Pretraining Approach

Original

Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov · 2019

Cited alongside, same era.

Factornet: A deep learning framework for predicting cell type specific transcription factor binding from nucleotide-resolution sequential data

X. X. Quang D · 2019

Cited alongside, same era.

Bringing BERT to the field: Transformer models for gene expression prediction in maize., 2020

B. Levy, Z. Xu, L. Zhao, and S. Ni · 2020

Later among the works it cites.

Big Bird: Transformers for Longer Sequences

M. Zaheer, G. Guruganesh, K. A. Dubey, J. Ainslie, C. Alberti, S. Ontanon, P. Pham, A. Ravula, Q. Wang, L. Yang, and A. Ahmed · 2020

Later among the works it cites.

Learning the protein language: Evolution, structure, and function

T. Bepler and B. Berger · 2021

Closest in time.

DeepGRN: prediction of transcription factor binding site across cell-types using attention-based deep neural networks

C. Chen, J. Hou, X. Shi, H. Yang, J. A. Birchler, and J. Cheng · 2021

Closest in time.

DNABERT: pre-trained Bidirectional Encoder Representations from Transformers model for DNA-language in genome

Y. Ji, Z. Zhou, H. Liu, and R. V. Davuluri · 2021

Closest in time.