Fetching the paper…
Reading the bibliography…
Attention-based vision models, such as Vision Transformer (ViT) and its variants, have shown promising performance in various computer vision tasks.
Imagenet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L · 2009
Earlier work this paper cites.
Microsoft coco: Common objects in context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C. L · 2014
Earlier work this paper cites.
Han, S., Mao, H., and Dally, W. J · 2015
Earlier work this paper cites.
Distilling the knowledge in a neural network
Hinton, G., Vinyals, O., and Dean, J · 2015
Earlier work this paper cites.
Compression of deep convolutional neural networks for fast and low power mobile applications
Kim, Y.-D., Park, E., Yoo, S., Choi, T., Yang, L., and Shin, D · 2015
Earlier work this paper cites.
Eyeriss: A spatial architecture for energy-efficient dataflow for convolutional neural networks
Chen, Y.-H., Emer, J., and Sze, V · 2016
Earlier work this paper cites.
Categorical reparameterization with gumbel-softmax
Jang, E., Gu, S., and Poole, B · 2016
Earlier work this paper cites.
Gans trained by a two time-scale update rule converge to a local nash equilibrium
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S · 2017
Earlier work this paper cites.
Darts: Differentiable architecture search
Liu, H., Simonyan, K., and Yang, Y · 2018
Earlier work this paper cites.
Timeloop: A systematic approach to dnn accelerator evaluation
Parashar, A., Raina, P., Shao, Y. S., Chen, Y.-H., Ying, V. A., Mukkara, A., Venkatesan, R., Khailany, B., Keckler, S. W., and Emer, J · 2019
Earlier work this paper cites.
Efficientnet: Rethinking model scaling for convolutional neural networks
Tan, M. and Le, Q · 2019
Earlier work this paper cites.
Huggingface’s transformers: State-of-the-art natural language processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al · 2019
Earlier work this paper cites.
Fbnet: Hardware-aware efficient convnet design via differentiable neural architecture search
Wu, B., Dai, X., Zhang, P., Wang, Y., Sun, F., Wu, Y., Tian, Y., Vajda, P., Jia, Y., and Keutzer, K · 2019
Earlier work this paper cites.
Intrinsic dimensionality explains the effectiveness of language model fine-tuning
Aghajanyan, A., Zettlemoyer, L., and Gupta, S · 2020
Earlier work this paper cites.
End-to-end object detection with transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., and Zagoruyko, S · 2020
Earlier work this paper cites.
Rethinking attention with performers
Choromanski, K., Likhosherstov, V., Dohan, D., Song, X., Gane, A., Sarlos, T., Hawkins, P., Davis, J., Mohiuddin, A., Kaiser, L., et al · 2020
Cited alongside, same era.
Power-bert: Accelerating bert inference via progressive word-vector elimination
Goyal, S., Choudhury, A. R., Raje, S., Chakaravarthy, V., Sabharwal, Y., and Verma, A · 2020
Cited alongside, same era.
Compressing pre-trained language models by matrix decomposition
Noach, M. B. and Goldberg, Y · 2020
Cited alongside, same era.
Scop: Scientific control for reliable neural network pruning
Tang, Y., Wang, Y., Xu, Y., Tao, D., Xu, C., Xu, C., and Xu, C · 2020
Cited alongside, same era.
Linformer: Self-attention with linear complexity
Wang, S., Li, B. Z., Khabsa, M., Fang, H., and Ma, H · 2020
Cited alongside, same era.
Language model compression with weighted low-rank factorization
Hsu, Y.-C., Hua, T., Chang, S., Lou, Q., Shen, Y., and Jin, H · 2022
Later among the works it cites.
Multi-concept customization of text-to-image diffusion
Kumari, N., Zhang, B., Zhang, R., Shechtman, E., and Zhu, J.-Y · 2022
Later among the works it cites.
Efficientformer: Vision transformers at mobilenet speed
Li, Y., Yuan, G., Wen, Y., Hu, J., Evangelidis, G., Tulyakov, S., Wang, Y., and Ren, J · 2022
Later among the works it cites.
Pseudo numerical methods for diffusion models on manifolds
Liu, L., Ren, Y., Lin, Z., and Zhao, Z · 2022
Later among the works it cites.
Hierarchical text-conditional image generation with clip latents
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Emerging properties in self-supervised vision transformers
Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., and Joulin, A · 2021
Cited alongside, same era.
Compressing neural networks: Towards determining the optimal layer-wise decomposition
Liebenwein, L., Maalouf, A., Feldman, D., and Rus, D · 2021
Cited alongside, same era.
On buggy resizing libraries and surprising subtleties in fid calculation
Parmar, G., Zhang, R., and Zhu, J.-Y · 2021
Cited alongside, same era.
Training data-efficient image transformers & distillation through attention
Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., and Jégou, H · 2021
Cited alongside, same era.
Segformer: Simple and efficient design for semantic segmentation with transformers
Xie, E., Wang, W., Yu, Z., Anandkumar, A., Alvarez, J. M., and Luo, P · 2021
Cited alongside, same era.
Towards efficient tensor decomposition-based dnn model compression with optimization framework
Yin, M., Sui, Y., Liao, S., and Yuan, B · 2021
Cited alongside, same era.
Zhu, M., Tang, Y., and Han, K · 2021
Cited alongside, same era.
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., and Chen, M · 2022
Later among the works it cites.
Exploring extreme parameter compression for pre-trained language models
Ren, Y., Wang, B., Shang, L., Jiang, X., and Liu, Q · 2022
Later among the works it cites.
High-resolution image synthesis with latent diffusion models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B · 2022
Later among the works it cites.
Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation
Ruiz, N., Li, Y., Jampani, V., Pritch, Y., Rubinstein, M., and Aberman, K · 2022
Later among the works it cites.
Photorealistic text-to-image diffusion models with deep language understanding
Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E., Ghasemipour, S. K. S., Ayan, B. K., Mahdavi, S. S., Lopes, R. G., et al · 2022
Later among the works it cites.
Patch slimming for efficient vision transformers
Tang, Y., Han, K., Wang, Y., Xu, C., Guo, J., Xu, C., and Tao, D · 2022
Later among the works it cites.
Upop: Unified and progressive pruning for compressing vision-language transformers
Shi, D., Tao, C., Jin, Y., Yang, Z., Yuan, C., and Wang, J · 2023
Closest in time.
Tdc: Towards extremely efficient cnns on gpus via hardware-aware tucker decomposition
Xiang, L., Yin, M., Zhang, C., Sukumaran-Rajam, A., Sadayappan, P., Yuan, B., and Tao, D · 2023
Closest in time.
Haloc: Hardware-aware automatic low-rank compression for compact neural networks
Xiao, J., Zhang, C., Gong, Y., Yin, M., Sui, Y., Xiang, L., Tao, D., and Yuan, B · 2023
Closest in time.
Yin, M., Uzkent, B., Shen, Y., Jin, H., and Yuan, B · 2023
Closest in time.
Compressing transformers: Features are low-rank, but weights are not!
Yu, H. and Wu, J · 2023
Closest in time.