Fetching the paper…
Reading the bibliography…
Mainstream 3D representation learning approaches are built upon contrastive or generative modeling pretext tasks, where great improvements in performance on various downstream tasks have been achieved.
A training algorithm for optimal margin classifiers
Boser, B. E., Guyon, I., and Vapnik, V · 1992
Earlier work this paper cites.
Statistical learning theory
Vapnik, V · 1998
Earlier work this paper cites.
Dimensionality reduction by learning an invariant mapping
Hadsell, R., Chopra, S., and LeCun, Y · 2006
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L., Li, K., and Fei-Fei, L · 2009
Earlier work this paper cites.
The Noether Theorems , pp. 55–64
Kosmann-Schwarzbach, Y · 2011
Earlier work this paper cites.
Shapenet: An information-rich 3d model repository
Chang, A. X., Funkhouser, T. A., Guibas, L. J., Hanrahan, P., Huang, Q., Li, Z., Savarese, S., Savva, M., Song, S., Su, H., Xiao, J., Yi, L., and Yu, F · 2015
Earlier work this paper cites.
Distilling the knowledge in a neural network
Hinton, G. E., Vinyals, O., and Dean, J · 2015
Earlier work this paper cites.
3d shapenets: A deep representation for volumetric shapes
Wu, Z., Song, S., Khosla, A., Yu, F., Zhang, L., Tang, X., and Xiao, J · 2015
Earlier work this paper cites.
Deep residual learning for image recognition
He, K., Zhang, X., Ren, S., and Sun, J · 2016
Earlier work this paper cites.
A scalable active framework for region annotation in 3d shape collections
Yi, L., Kim, V. G., Ceylan, D., Shen, I.-C., Yan, M., Su, H., Lu, C., Huang, Q., Sheffer, A., and Guibas, L · 2016
Earlier work this paper cites.
A point set generation network for 3d object reconstruction from a single image
Fan, H., Su, H., and Guibas, L. J · 2017
Earlier work this paper cites.
SGDR: stochastic gradient descent with warm restarts
Loshchilov, I. and Hutter, F · 2017
Earlier work this paper cites.
An overview of multi-task learning in deep neural networks
Ruder, S · 2017
Earlier work this paper cites.
Attention is all you need
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I · 2017
Earlier work this paper cites.
Mutual information neural estimation
Belghazi, M. I., Baratin, A., Rajeswar, S., Ozair, S., Bengio, Y., Hjelm, R. D., and Courville, A. C · 2018
Earlier work this paper cites.
VSE++: improving visual-semantic embeddings with hard negatives
Faghri, F., Fleet, D. J., Kiros, J. R., and Fidler, S · 2018
Earlier work this paper cites.
Pointcnn: Convolution on x-transformed points
Li, Y., Bu, R., Sun, M., Wu, W., Di, X., and Chen, B · 2018
Earlier work this paper cites.
Improving language understanding by generative pre-training
Radford, A., Narasimhan, K., Salimans, T., Sutskever, I., et al · 2018
Earlier work this paper cites.
Representation learning with contrastive predictive coding
van den Oord, A., Li, Y., and Vinyals, O · 2018
Earlier work this paper cites.
Unsupervised feature learning via non-parametric instance discrimination
Wu, Z., Xiong, Y., Yu, S. X., and Lin, D · 2018
Earlier work this paper cites.
Learning representations by maximizing mutual information across views
Bachman, P., Hjelm, R. D., and Buchwalter, W · 2019
Earlier work this paper cites.
BERT: pre-training of deep bidirectional transformers for language understanding
Devlin, J., Chang, M., Lee, K., and Toutanova, K · 2019
Earlier work this paper cites.
Scaling and benchmarking self-supervised visual representation learning
Goyal, P., Mahajan, D., Gupta, A., and Misra, I · 2019
Earlier work this paper cites.
Learning deep representations by mutual information estimation and maximization
Hjelm, R. D., Fedorov, A., Lavoie-Marchildon, S., Grewal, K., Bachman, P., Trischler, A., and Bengio, Y · 2019
Earlier work this paper cites.
Relation-shape convolutional neural network for point cloud analysis
Liu, Y., Fan, B., Xiang, S., and Pan, C · 2019
Earlier work this paper cites.
Decoupled weight decay regularization
Loshchilov, I. and Hutter, F · 2019
Earlier work this paper cites.
Revisiting point cloud classification: A new benchmark dataset and classification model on real-world data
Uy, M. A., Pham, Q.-H., Hua, B.-S., Nguyen, T., and Yeung, S.-K · 2019
Cited alongside, same era.
Dynamic graph CNN for learning on point clouds
Wang, Y., Sun, Y., Liu, Z., Sarma, S. E., Bronstein, M. M., and Solomon, J. M · 2019
Cited alongside, same era.
Language models are few-shot learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., and Amodei, D · 2020
Cited alongside, same era.
A simple framework for contrastive learning of visual representations
Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. E · 2020
Cited alongside, same era.
Bootstrap your own latent - A new approach to self-supervised learning
Grill, J., Strub, F., Altché, F., Tallec, C., Richemond, P. H., Buchatskaya, E., Doersch, C., Pires, B. Á., Guo, Z., Azar, M. G., Piot, B., Kavukcuoglu, K., Munos, R., and Valko, M · 2020
Barlow twins: Self-supervised learning via redundancy reduction
Zbontar, J., Jing, L., Misra, I., LeCun, Y., and Deny, S · 2021
Later among the works it cites.
Self-supervised pretraining of 3d features on any point-cloud
Zhang, Z., Girdhar, R., Joulin, A., and Misra, I · 2021
Later among the works it cites.
Crosspoint: Self-supervised cross-modal contrastive learning for 3d point cloud understanding
Afham, M., Dissanayake, I., Dissanayake, D., Dharmasiri, A., Thilakarathna, K., and Rodrigo, R · 2022
Later among the works it cites.
Flamingo: a visual language model for few-shot learning
Alayrac, J., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., Ring, R., Rutherford, E., Cabi, S., Han, T., Gong, Z., Samangooei, S., Monteiro, M., Menick, J., Borgeaud, S., Brock, A., Nematzadeh, A., Sharifzadeh, S., Binkowski, M., Barreira, R., Vinyals, O., Zisserman, A., and Simonyan, K · 2022
Later among the works it cites.
Beit: BERT pre-training of image transformers
Bao, H., Dong, L., Piao, S., and Wei, F · 2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Momentum contrast for unsupervised visual representation learning
He, K., Fan, H., Wu, Y., Xie, S., and Girshick, R. B · 2020
Cited alongside, same era.
Supervised contrastive learning
Khosla, P., Teterwak, P., Wang, C., Sarna, A., Tian, Y., Isola, P., Maschinot, A., Liu, C., and Krishnan, D · 2020
Cited alongside, same era.
Feature-level ensemble knowledge distillation for aggregating knowledge from multiple networks
Park, S. and Kwak, N · 2020
Cited alongside, same era.
Learning from multiple experts: Self-paced knowledge distillation for long-tailed classification
Xiang, L., Ding, G., and Han, J · 2020
Cited alongside, same era.
Pointcontrast: Unsupervised pre-training for 3d point cloud understanding
Xie, S., Gu, J., Guo, D., Qi, C. R., Guibas, L. J., and Litany, O · 2020
Cited alongside, same era.
On the opportunities and risks of foundation models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N. S., Chen, A. S., Creel, K., Davis, J. Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N. D., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D. E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P. W., Krass, M. S., Krishna, R., Kuditipudi, R., and et al · 2021
Cited alongside, same era.
Emerging properties in self-supervised vision transformers
Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., and Joulin, A · 2021
Cited alongside, same era.
4dcontrast: Contrastive learning with dynamic correspondences for 3d scene understanding
Chen, Y., Nießner, M., and Dai, A · 2022
Later among the works it cites.
Equivariant self-supervised learning: Encouraging equivariance in representations
Dangovski, R., Jing, L., Loh, C., Han, S., Srivastava, A., Cheung, B., Agrawal, P., and Soljacic, M · 2022
Later among the works it cites.
Dong, X., Bao, J., Zhang, T., Chen, D., Gu, S., Zhang, W., Yuan, L., Chen, D., Wen, F., and Yu, N · 2022
Later among the works it cites.
Masked autoencoders are scalable vision learners
He, K., Chen, X., Xie, S., Li, Y., Dollár, P., and Girshick, R. B · 2022
Later among the works it cites.
Clip2point: Transfer CLIP to point cloud classification with image-depth pre-training
Huang, T., Dong, B., Yang, Y., Huang, X., Lau, R. W. H., Ouyang, W., and Zuo, W · 2022
Later among the works it cites.
Masked discrimination for self-supervised learning on point clouds
Liu, H., Cai, M., and Lee, Y. J · 2022
Later among the works it cites.
Rethinking network design and local geometry in point cloud: A simple residual MLP framework
Ma, X., Qin, C., You, H., Ran, H., and Fu, Y · 2022
Later among the works it cites.
Introducing chatgpt
OpenAI · 2022
Later among the works it cites.
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P. F., Leike, J., and Lowe, R · 2022
Later among the works it cites.
Masked autoencoders for point cloud self-supervised learning
Pang, Y., Wang, W., Tay, F. E. H., Liu, W., Tian, Y., and Yuan, L · 2022
Later among the works it cites.
Pointnext: Revisiting pointnet++ with improved training and scaling strategies
Qian, G., Li, Y., Peng, H., Mai, J., Hammoud, H. A. A. K., Elhoseiny, M., and Ghanem, B · 2022
Later among the works it cites.
High-resolution image synthesis with latent diffusion models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B · 2022
Later among the works it cites.
LAION-5B: an open large-scale dataset for training next generation image-text models
Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., Schramowski, P., Kundurthy, S., Crowson, K., Schmidt, L., Kaczmarczyk, R., and Jitsev, J · 2022
Later among the works it cites.
Siamese image modeling for self-supervised vision representation learning
Tao, C., Zhu, X., Huang, G., Qiao, Y., Wang, X., and Dai, J · 2022
Later among the works it cites.
Knowledge distillation and student-teacher learning for visual intelligence: A review and new outlooks
Wang, L. and Yoon, K · 2022
Later among the works it cites.
P2P: tuning pre-trained image models for point cloud analysis with point-to-pixel prompting
Wang, Z., Yu, X., Rao, Y., Zhou, J., and Lu, J · 2022
Later among the works it cites.
ibot: Image BERT pre-training with online tokenizer
Zhou, J., Wei, C., Wang, H., Shen, W., Xie, C., Yuille, A. L., and Kong, T · 2022
Later among the works it cites.
Autoencoders as cross-modal teachers: Can pretrained 2d image transformers help 3d representation learning?
Dong, R., Qi, Z., Zhang, L., Zhang, J., Sun, J., Ge, Z., Yi, L., and Ma, K · 2023
Closest in time.
Understanding masked image modeling via learning occlusion invariant feature
Kong, X. and Zhang, X · 2023
Closest in time.
Li, J., Li, D., Savarese, S., and Hoi, S. C. H · 2023
Closest in time.