Fetching the paper…
Reading the bibliography…
Prompt-tuning is an emerging strategy to adapt large language models (LLM) to downstream tasks by learning a (soft-)prompt parameter from data.
Empirical processes: theory and applications
Pollard, D · 1990
Earlier work this paper cites.
Probability in Banach Spaces: isoperimetry and processes , volume 23
Ledoux, M. and Talagrand, M · 1991
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Krizhevsky, A., Hinton, G., et al · 2009
Earlier work this paper cites.
Neural machine translation by jointly learning to align and translate
Bahdanau, D., Cho, K., and Bengio, Y · 2014
Earlier work this paper cites.
Adam: A method for stochastic optimization
Kingma, D. P. and Ba, J · 2014
Earlier work this paper cites.
Chan, W., Jaitly, N., Le, Q. V., and Vinyals, O · 2015
Earlier work this paper cites.
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A. C., and Fei-Fei, L · 2015
Earlier work this paper cites.
Show, attend and tell: Neural image caption generation with visual attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhudinov, R., Zemel, R., and Bengio, Y · 2015
Earlier work this paper cites.
Attention is all you need
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I · 2017
Earlier work this paper cites.
Mohammadi, H., Zare, A., Soltanolkotabi, M., and Jovanović, M. R · 2019
Earlier work this paper cites.
Stochastic gradient descent learns state equations with nonlinear activations
Oymak, S · 2019
Earlier work this paper cites.
Language models are few-shot learners
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al · 2020
Cited alongside, same era.
Are transformers universal approximators of sequence-to-sequence functions?
Yun, C., Bhojanapalli, S., Rawat, A. S., Reddi, S., and Kumar, S · 2020
Cited alongside, same era.
Attention is not all you need: Pure attention loses rank doubly exponentially with depth
Dong, Y., Cordonnier, J.-B., and Loukas, A · 2021
Cited alongside, same era.
An image is worth 16x16 words: Transformers for image recognition at scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N · 2021
Cited alongside, same era.
Inductive biases and variable creation in self-attention mechanisms
Edelman, B. L., Goel, S., Kakade, S., and Zhang, C · 2021
Scenic: A jax library for computer vision research and beyond
Dehghani, M., Gritsenko, A., Arnab, A., Minderer, M., and Tay, Y · 2022
Later among the works it cites.
Convexifying transformers: Improving optimization and understanding of transformer networks
Ergen, T., Neyshabur, B., and Mehta, H · 2022
Later among the works it cites.
Random feature amplification: Feature learning and generalization in neural networks
Frei, S., Chatterji, N. S., and Bartlett, P. L · 2022
Later among the works it cites.
Vision transformers provably learn spatial structure
Jelassi, S., Sander, M. E., and Li, Y · 2022
Later among the works it cites.
Hierarchical text-conditional image generation with clip latents
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., and Chen, M · 2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Local signal adaptivity: Provable feature learning in neural networks beyond kernels
Karp, S., Winston, E., Li, Y., and Singh, A · 2021
Cited alongside, same era.
The power of scale for parameter-efficient prompt tuning
Lester, B., Al-Rfou, R., and Constant, N · 2021
Cited alongside, same era.
Prefix-tuning: Optimizing continuous prompts for generation
Li, X. L. and Liang, P · 2021
Cited alongside, same era.
Efficient large-scale language model training on gpu clusters using megatron-lm
Narayanan, D., Shoeybi, M., Casper, J., LeGresley, P., Patwary, M., Korthikanti, V., Vainbrand, D., Kashinkunti, P., Bernauer, J., Catanzaro, B., et al · 2021
Cited alongside, same era.
Baldi, P. and Vershynin, R · 2022
Cited alongside, same era.
Benign overfitting in two-layer convolutional neural networks
Cao, Y., Chen, Z., Belkin, M., and Gu, Q · 2022
Cited alongside, same era.
https://openai.com/blog/chatgpt/
Cited in the paper.
Reed, S., Zolna, K., Parisotto, E., Colmenarejo, S. G., Novikov, A., Barth-Maron, G., Gimenez, M., Sulsky, Y., Kay, J., Springenberg, J. T., et al · 2022
Later among the works it cites.
Photorealistic text-to-image diffusion models with deep language understanding
Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E., Ghasemipour, S. K. S., Ayan, B. K., Mahdavi, S. S., Lopes, R. G., et al · 2022
Later among the works it cites.
Unraveling attention via convex duality: Analysis and interpretations of vision transformers
Sahiner, A., Ergen, T., Ozturkler, B., Pauly, J., Mardani, M., and Pilanci, M · 2022
Later among the works it cites.
A theoretical understanding of shallow vision transformers: Learning, generalization, and sample complexity
Li, H., Wang, M., Liu, S., and Chen, P.-Y · 2023
Closest in time.
Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., and Neubig, G · 2023
Closest in time.