Fetching the paper…
Reading the bibliography…
How can we best encode structured data into sequential form for use in large language models (LLMs)? In this work, we introduce a parameter-efficient method to explicitly represent structured data for LLMs.
Practical graph isomorphism
McKay, B. D. et al · 1981
Earlier work this paper cites.
A neural probabilistic language model
Bengio, Y., Ducharme, R., and Vincent, P · 2000
Earlier work this paper cites.
Two decades of statistical language modeling: Where do we go from here?
Rosenfeld, R · 2000
Earlier work this paper cites.
Efficient estimation of word representations in vector space
Mikolov, T., Chen, K., Corrado, G., and Dean, J · 2013
Earlier work this paper cites.
Deepwalk: online learning of social representations
Perozzi, B., Al-Rfou, R., and Skiena, S · 2014
Earlier work this paper cites.
TensorFlow: Large-scale machine learning on heterogeneous systems, 2015
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Mané, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Viégas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., and Zheng, X · 2015
Earlier work this paper cites.
Neural message passing for quantum chemistry, 2017
Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O., and Dahl, G. E · 2017
Earlier work this paper cites.
In-datacenter performance analysis of a tensor processing unit
Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., Boyle, R., Cantin, P.-l., Chao, C., Clark, C., Coriell, J., Daley, M., Dau, M., Dean, J., Gelb, B., Ghaemmaghami, T. V., Gottipati, R., Gulland, W., Hagmann, R., Ho, C. R., Hogberg, D., Hu, J., Hundt, R., Hurt, D., Ibarz, J., Jaffey, A., Jaworski, A., Kaplan, A., Khaitan, H., Killebrew, D., Koch, A., Kumar, N., Lacy, S., Laudon, J., Law, J., Le, D., Leary, C., Liu, Z., Lucke, K., Lundin, A., MacKean, G., Maggiore, A., Mahony, M., Miller, K., Nagarajan, R., Narayanaswami, R., Ni, R., Nix, K., Norrie, T., Omernick, M., Penukonda, N., Phelps, A., Ross, J., Ross, M., Salek, A., Samadiani, E., Severn, C., Sizikov, G., Snelham, M., Souter, J., Steinberg, D., Swing, A., Tan, M., Thorson, G., Tian, B., Toma, H., Tuttle, E., Vasudevan, V., Walter, R., Wang, W., Wilcox, E., and Yoon, D. H · 2017
Earlier work this paper cites.
Semi-supervised classification with graph convolutional networks, 2017
Kipf, T. N. and Welling, M · 2017
Earlier work this paper cites.
Attention is all you need
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I · 2017
Earlier work this paper cites.
Relational inductive biases, deep learning, and graph networks, 2018
Battaglia, P. W., Hamrick, J. B., Bapst, V., Sanchez-Gonzalez, A., Zambaldi, V., Malinowski, M., Tacchetti, A., Raposo, D., Santoro, A., Faulkner, R., Gulcehre, C., Song, F., Ballard, A., Gilmer, J., Dahl, G., Vaswani, A., Allen, K., Nash, C., Langston, V., Dyer, C., Heess, N., Wierstra, D., Kohli, P., Botvinick, M., Vinyals, O., Li, Y., and Pascanu, R · 2018
Earlier work this paper cites.
Bert: Pre-training of deep bidirectional transformers for language understanding
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K · 2018
Earlier work this paper cites.
Umap: Uniform manifold approximation and projection for dimension reduction
McInnes, L., Healy, J., and Melville, J · 2018
Earlier work this paper cites.
Improving language understanding by generative pre-training
Radford, A., Narasimhan, K., Salimans, T., Sutskever, I., et al · 2018
Earlier work this paper cites.
Sgr: Self-supervised spectral graph representation learning
Tsitsulin, A., Mottin, D., Karras, P., Bronstein, A., and Müller, E · 2018
Earlier work this paper cites.
How powerful are graph neural networks?
Xu, K., Hu, W., Leskovec, J., and Jegelka, S · 2018
Earlier work this paper cites.
Parameter-efficient transfer learning for nlp
Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., De Laroussilhe, Q., Gesmundo, A., Attariyan, M., and Gelly, S · 2019
Earlier work this paper cites.
Generalization through memorization: Nearest neighbor language models
Khandelwal, U., Levy, O., Jurafsky, D., Zettlemoyer, L., and Lewis, M · 2019
Earlier work this paper cites.
Language models are unsupervised multitask learners
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al · 2019
Earlier work this paper cites.
Knowledge graph based synthetic corpus generation for knowledge-enhanced language model pre-training
Agarwal, O., Ge, H., Shakeri, S., and Al-Rfou, R · 2020
Earlier work this paper cites.
Scaling graph neural networks with approximate pagerank
Bojchevski, A., Gasteiger, J., Perozzi, B., Kapoor, A., Blais, M., Rózemberczki, B., Lukasik, M., and Günnemann, S · 2020
Earlier work this paper cites.
Language models are few-shot learners
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al · 2020
Cited alongside, same era.
Retrieval augmented language model pre-training
Guu, K., Lee, K., Tung, Z., Pasupat, P., and Chang, M · 2020
Cited alongside, same era.
Heterogeneous graph transformer, 2020
Hu, Z., Dong, Y., Wang, K., and Sun, Y · 2020
Cited alongside, same era.
Exploring the limits of transfer learning with a unified text-to-text transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J · 2020
Cited alongside, same era.
Masking as an efficient alternative to finetuning for pretrained language models
Zhao, M., Lin, T., Mi, F., Jaggi, M., and Schütze, H · 2020
Cited alongside, same era.
Vivit: A video vision transformer
Standing on the shoulders of giant frozen language models, 2022
Levine, Y., Dalmedigos, I., Ram, O., Zeldes, Y., Jannai, D., Muhlgay, D., Osin, Y., Lieber, O., Lenz, B., Shalev-Shwartz, S., Shashua, A., Leyton-Brown, K., and Shoham, Y · 2022
Later among the works it cites.
Valipour, M., Rezagholizadeh, M., Kobyzev, I., and Ghodsi, A · 2022
Later among the works it cites.
Equivariant and stable positional encoding for more powerful graph neural networks
Wang, H., Yin, H., Zhang, M., and Li, P · 2022
Later among the works it cites.
Chain-of-thought prompting elicits reasoning in large language models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al · 2022
Later among the works it cites.
Self-supervised learning of graph neural networks: A unified review
Xie, Y., Xu, Z., Zhang, J., Wang, Z., and Ji, S · 2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lučić, M., and Schmid, C · 2021
Cited alongside, same era.
Breaking the limits of message passing graph neural networks
Balcilar, M., Héroux, P., Gauzere, B., Vasseur, P., Adam, S., and Honeine, P · 2021
Cited alongside, same era.
A generalization of transformer networks to graphs, 2021
Dwivedi, V. P. and Bresson, X · 2021
Cited alongside, same era.
On the effectiveness of adapter-based tuning for pretrained language model adaptation
He, R., Liu, L., Ye, H., Tan, Q., Ding, B., Cheng, L., Low, J.-W., Bing, L., and Si, L · 2021
Cited alongside, same era.
Lora: Low-rank adaptation of large language models
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W · 2021
Cited alongside, same era.
N-gram language models
Jurafsky, Dan; Martin, J. H · 2021
Cited alongside, same era.
The power of scale for parameter-efficient prompt tuning, 2021
Lester, B., Al-Rfou, R., and Constant, N · 2021
Cited alongside, same era.
Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al · 2023
Later among the works it cites.
Anil, R., Dai, A. M., Firat, O., Johnson, M., Lepikhin, D., Passos, A., Shakeri, S., Taropa, E., Bailey, P., Chen, Z., et al · 2023
Later among the works it cites.
Benchmarking graph neural networks
Dwivedi, V. P., Joshi, C. K., Luu, A. T., Laurent, T., Bengio, Y., and Bresson, X · 2023
Later among the works it cites.
Universal prompt tuning for graph neural networks, 2023
Fang, T., Zhang, Y., Yang, Y., Wang, C., and Chen, L · 2023
Later among the works it cites.
TF-GNN: Graph neural networks in tensorflow, 2023
Ferludin, O., Eigenwillig, A., Blais, M., Zelle, D., Pfeifer, J., Sanchez-Gonzalez, A., Li, W. L. S., Abu-El-Haija, S., Battaglia, P., Bulut, N., Halcrow, J., de Almeida, F. M. G., Gonnet, P., Jiang, L., Kothari, P., Lattanzi, S., Linhares, A., Mayer, B., Mirrokni, V., Palowitch, J., Paradkar, M., She, J., Tsitsulin, A., Villela, K., Wang, L., Wong, D., and Perozzi, B · 2023
Later among the works it cites.
Guo, J., Du, L., and Liu, H · 2023
Later among the works it cites.
Sign and basis invariant networks for spectral graph representation learning
Lim, D., Robinson, J., Zhao, L., Smidt, T., Sra, S., Maron, H., and Jegelka, S · 2023
Later among the works it cites.
Graphprompt: Unifying pre-training and downstream tasks for graph neural networks
Liu, Z., Yu, X., Fang, Y., and Zhang, X · 2023
Later among the works it cites.
Gpt-4 doesn’t know it’s wrong: An analysis of iterative prompting for reasoning problems
Stechly, K., Marquez, M., and Kambhampati, S · 2023
Later among the works it cites.
Gemini: a family of highly capable multimodal models
Team, G., Anil, R., Borgeaud, S., Wu, Y., Alayrac, J.-B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A. M., Hauth, A., et al · 2023
Later among the works it cites.
Llama 2: Open foundation and fine-tuned chat models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al · 2023
Later among the works it cites.
Freshllms: Refreshing large language models with search engine augmentation, 2023
Vu, T., Iyyer, M., Wang, X., Constant, N., Wei, J., Wei, J., Tar, C., Sung, Y.-H., Zhou, D., Le, Q., and Luong, T · 2023
Later among the works it cites.
Xu, L., Xie, H., Qin, S.-Z. J., Tao, X., and Wang, F. L · 2023
Later among the works it cites.
Natural language is all a graph needs
Ye, R., Zhang, C., Wang, R., Xu, S., and Zhang, Y · 2023
Later among the works it cites.
A survey of large language models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., et al · 2023
Later among the works it cites.
Talk like a graph: Encoding graphs for large language models
Fatemi, B., Halcrow, J., and Perozzi, B · 2024
Closest in time.