Kformer: Knowledge Injection in Transformer Feed-Forward Layers

Robertson, S., Zaragoza, H.: The probabilistic relevance framework: BM25 and beyond. Now Publishers Inc (2009)

2009

Earlier work this paper cites.

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. In: Proc. of NeurIPS (2017)

2017

Earlier work this paper cites.

Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proc. of NAACL (2019)

2019

Earlier work this paper cites.

Lin, B.Y., Chen, X., Chen, J., Ren, X.: Kagnet: Knowledge-aware graph networks for commonsense reasoning. In: Inui, K., Jiang, J., Ng, V., Wan, X. (eds.) Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019. pp. 2829–2839. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/D19-1282, https://doi.org/10.18653/v1/D19-1282

2019

Earlier work this paper cites.

Ott, M., Edunov, S., Baevski, A., Fan, A., Gross, S., Ng, N., Grangier, D., Auli, M.: fairseq: A fast, extensible toolkit for sequence modeling. In: Proc. of NAACL: Demonstrations (2019)

2019

Earlier work this paper cites.

Sap, M., Le Bras, R., Allaway, E., Bhagavatula, C., Lourie, N., Rashkin, H., Roof, B., Smith, N.A., Choi, Y.: Atomic: An atlas of machine commonsense for if-then reasoning. In: Proc. of AAAI (2019)

2019

Earlier work this paper cites.

Sap, M., Rashkin, H., Chen, D., Le Bras, R., Choi, Y.: Social IQa: Commonsense reasoning about social interactions. In: Proc. of EMNLP (2019)

2019

Earlier work this paper cites.

Wang, X., Kapanipathi, P., Musa, R., Yu, M., Talamadupula, K., Abdelaziz, I., Chang, M., Fokoue, A., Makni, B., Mattei, N., Witbrock, M.: Improving natural language inference using external knowledge in the science questions domain. In: Proc. of AAAI (2019)

2019

Earlier work this paper cites.

Chang, T.Y., Liu, Y., Gopalakrishnan, K., Hedayatnia, B., Zhou, P., Hakkani-Tur, D.: Incorporating commonsense knowledge graph in pretrained models for social commonsense tasks. In: Proc. DeeLIO: The First Workshop on Knowledge Extraction and Integration for Deep Learning Architectures (2020)

2020

Cited alongside, same era.

Jin, D., Pan, E., Oufattole, N., Weng, W.H., Fang, H., Szolovits, P.: What disease does this patient have? a large-scale open domain question answering dataset from medical exams. arXiv preprint arXiv:2009.13081 (2020)

Original

2020

Cited alongside, same era.

Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.t., Rocktäschel, T., et al.: Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems 33

2020

Cited alongside, same era.

Liu, W., Zhou, P., Zhao, Z., Wang, Z., Ju, Q., Deng, H., Wang, P.: K-bert: Enabling language representation with knowledge graph. In: Proc. of AAAI (2020)

Wang, W., Tu, Z.: Rethinking the value of transformer components. In: Proc. of COLING (2020)

2020

Later among the works it cites.

Xu, Y., Zhu, C., Xu, R., Liu, Y., Zeng, M., Huang, X.: Fusing context into knowledge graph for commonsense reasoning. arXiv preprint arXiv:2012.04808 (2020)

Original

2020

Later among the works it cites.

Geva, M., Schuster, R., Berant, J., Levy, O.: Transformer feed-forward layers are key-value memories. In: Proc. of EMNLP (2021)

2021

Later among the works it cites.

Han, X., Zhang, Z., Ding, N., Gu, Y., Liu, X., Huo, Y., Qiu, J., Yao, Y., Zhang, A., Zhang, L., et al.: Pre-trained models: Past, present and future. AI Open 2

2021

Later among the works it cites.

Hao, Y., Dong, L., Wei, F., Xu, K.: Self-attention attribution: Interpreting information interactions inside transformer. In: Proc. of AAAI (2021)

2021