Fetching the paper…
Reading the bibliography…
This study investigates the loss of generalization ability in neural networks, revisiting warm-starting experiments from Ash & Adams.
A simple weight decay can improve generalization
Krogh, A. and Hertz, J · 1991
Earlier work this paper cites.
Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory
McClelland, J. L., McNaughton, B. L., and O’Reilly, R. C · 1995
Earlier work this paper cites.
Temporal difference learning and td-gammon
Tesauro, G. et al · 1995
Earlier work this paper cites.
The mnist database of handwritten digits
LeCun, Y., Cortes, C., and Burges, C · 1998
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Krizhevsky, A · 2009
Earlier work this paper cites.
Understanding the difficulty of training deep feedforward neural networks
Glorot, X. and Bengio, Y · 2010
Earlier work this paper cites.
Hippocampal neurogenesis and forgetting
Frankland, P. W., Köhler, S., and Josselyn, S. A · 2013
Earlier work this paper cites.
Adam: A method for stochastic optimization
Kingma, D. P. and Ba, J · 2014
Earlier work this paper cites.
Very deep convolutional networks for large-scale image recognition
Simonyan, K. and Zisserman, A · 2014
Earlier work this paper cites.
Delving deep into rectifiers: Surpassing human-level performance on imagenet classification
He, K., Zhang, X., Ren, S., and Sun, J · 2015
Earlier work this paper cites.
Tiny imagenet visual recognition challenge
Le, Y. and Yang, X · 2015
Earlier work this paper cites.
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al · 2015
Earlier work this paper cites.
Imagenet large scale visual recognition challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al · 2015
Earlier work this paper cites.
Deep learning with differential privacy
Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., and Zhang, L · 2016
Earlier work this paper cites.
Rl 2 : Fast reinforcement learning via slow reinforcement learning
Duan, Y., Schulman, J., Chen, X., Bartlett, P. L., Sutskever, I., and Abbeel, P · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
He, K., Zhang, X., Ren, S., and Sun, J · 2016
Earlier work this paper cites.
What learning systems do intelligent agents need? complementary learning systems theory updated
Kumaran, D., Hassabis, D., and McClelland, J. L · 2016
Earlier work this paper cites.
Understanding and improving convolutional neural networks via concatenated rectified linear units
Shang, W., Sohn, K., Almeida, D., and Lee, H · 2016
Earlier work this paper cites.
Decoupled weight decay regularization
Loshchilov, I. and Hutter, F · 2017
Earlier work this paper cites.
Communication-efficient learning of deep networks from decentralized data
McMahan, B., Moore, E., Ramage, D., Hampson, S., and y Arcas, B. A · 2017
Earlier work this paper cites.
Neural episodic control
Pritzel, A., Uria, B., Srinivasan, S., Badia, A. P., Vinyals, O., Hassabis, D., Wierstra, D., and Blundell, C · 2017
Earlier work this paper cites.
Continual learning with deep generative replay
Shin, H., Lee, J. K., Kim, J., and Kim, J · 2017
Earlier work this paper cites.
Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
Tarvainen, A. and Valpola, H · 2017
Earlier work this paper cites.
Impala: Scalable distributed deep-rl with importance weighted actor-learner architectures
Espeholt, L., Soyer, H., Munos, R., Simonyan, K., Mnih, V., Ward, T., Doron, Y., Firoiu, V., Harley, T., Dunning, I., et al · 2018
Earlier work this paper cites.
Rainbow: Combining improvements in deep reinforcement learning
Hessel, M., Modayil, J., Van Hasselt, H., Schaul, T., Ostrovski, G., Dabney, W., Horgan, D., Piot, B., Azar, M., and Silver, D · 2018
Earlier work this paper cites.
Reinforcement learning: An introduction
Sutton, R. S. and Barto, A. G · 2018
Earlier work this paper cites.
The forgotten part of memory
Gravitz, L · 2019
Earlier work this paper cites.
Model-based reinforcement learning for atari
Kaiser, L., Babaeizadeh, M., Milos, P., Osinski, B., Campbell, R. H., Czechowski, K., Erhan, D., Finn, C., Kozakowski, P., Levine, S., et al · 2019
Earlier work this paper cites.
Data augmentation using random image cropping and patching for deep cnns
Takahashi, R., Matsubara, T., and Uehara, K · 2019
Cited alongside, same era.
Large batch optimization for deep learning: Training bert in 76 minutes
You, Y., Li, J., Reddi, S., Hseu, J., Kumar, S., Bhojanapalli, S., Song, X., Demmel, J., Keutzer, K., and Hsieh, C.-J · 2019
Cited alongside, same era.
Towards understanding ensemble, knowledge distillation and self-distillation in deep learning
Allen-Zhu, Z. and Li, Y · 2020
Cited alongside, same era.
On warm-starting neural network training
Ash, J. and Adams, R. P · 2020
Cited alongside, same era.
An image is worth 16x16 words: Transformers for image recognition at scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al · 2020
Forgetting as a form of adaptive engram cell plasticity
Ryan, T. J. and Frankland, P. W · 2022
Later among the works it cites.
Fortuitous forgetting in connectionist networks
Zhou, H., Vani, A., Larochelle, H., and Courville, A · 2022
Later among the works it cites.
Loss of plasticity in continual deep reinforcement learning
Abbas, Z., Zhao, R., Modayil, J., White, A., and Machado, M. C · 2023
Later among the works it cites.
Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al · 2023
Later among the works it cites.
Prediction and control in continual reinforcement learning
Anand, N. and Precup, D · 2023
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Scaling laws for neural language models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D · 2020
Cited alongside, same era.
Image augmentation is all you need: Regularizing deep reinforcement learning from pixels
Kostrikov, I., Yarats, D., and Fergus, R · 2020
Cited alongside, same era.
Implicit under-parameterization inhibits data-efficient deep reinforcement learning
Kumar, A., Agarwal, R., Ghosh, D., and Levine, S · 2020
Cited alongside, same era.
Reinforcement learning with augmented data
Laskin, M., Lee, K., Stooke, A., Pinto, L., Abbeel, P., and Srinivas, A · 2020
Cited alongside, same era.
Data-efficient reinforcement learning with self-predictive representations
Schwarzer, M., Anand, A., Goel, R., Hjelm, R. D., Courville, A., and Bachman, P · 2020
Cited alongside, same era.
Toward training recurrent neural networks for lifelong learning
Sodhani, S., Chandar, S., and Bengio, Y · 2020
Cited alongside, same era.
Disentangling trainability and generalization in deep neural networks
Xiao, L., Pennington, J., and Schoenholz, S · 2020
Cited alongside, same era.
Later among the works it cites.
Exponential moving average of weights in deep learning: Dynamics and benefits
Anonymous · 2023
Later among the works it cites.
Resetting the optimizer in deep rl: An empirical study
Asadi, K., Fakoor, R., and Sabach, S · 2023
Later among the works it cites.
Maintaining plasticity in deep continual learning
Dohare, S., Hernandez-Garcia, J. F., Rahman, P., Sutton, R. S., and Mahmood, A. R · 2023
Later among the works it cites.
Reset it and forget it: Relearning last-layer weights improves continual and transfer learning
Frati, L., Traft, N., Clune, J., and Cheney, N · 2023
Later among the works it cites.
For sale: State-action representation learning for deep reinforcement learning
Fujimoto, S., Chang, W.-D., Smith, E. J., Gu, S. S., Precup, D., and Meger, D · 2023
Later among the works it cites.
Gemini: a family of highly capable multimodal models
Google, G., Anil, R., Borgeaud, S., Wu, Y., Alayrac, J.-B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A. M., Hauth, A., et al · 2023
Later among the works it cites.
Td-mpc2: Scalable, robust world models for continuous control
Hansen, N., Su, H., and Wang, X · 2023
Later among the works it cites.
Plastic: Improving input and label plasticity for sample efficient reinforcement learning
Lee, H., Cho, H., Kim, H., Gwak, D., Kim, J., Choo, J., Yun, S.-Y., and Yun, C · 2023
Later among the works it cites.
Curvature explains loss of plasticity
Lewandowski, A., Tanaka, H., Schuurmans, D., and Machado, M. C · 2023
Later among the works it cites.
Understanding plasticity in neural networks
Lyle, C., Zheng, Z., Nikishin, E., Pires, B. A., Pascanu, R., and Dabney, W · 2023
Later among the works it cites.
Revisiting plasticity in visual reinforcement learning: Data, modules and training stages
Ma, G., Li, L., Zhang, S., Liu, Z., Wang, Z., Chen, Y., Shen, L., Wang, X., and Tao, D · 2023
Later among the works it cites.
Language model alignment with elastic reset
Noukhovitch, M., Lavoie, S., Strub, F., and Courville, A · 2023
Later among the works it cites.
Bigger, better, faster: Human-level atari with human-level efficiency
Schwarzer, M., Ceron, J. S. O., Courville, A., Bellemare, M. G., Agarwal, R., and Castro, P. S · 2023
Later among the works it cites.
A closer look at rehearsal-free continual learning
Smith, J. S., Tian, J., Halbe, S., Hsu, Y.-C., and Kira, Z · 2023
Later among the works it cites.
The dormant neuron phenomenon in deep reinforcement learning
Sokar, G., Agarwal, R., Castro, P. S., and Evci, U · 2023
Later among the works it cites.
Drm: Mastering visual reinforcement learning through dormant ratio minimization
Xu, G., Zheng, R., Liang, Y., Wang, X., Yuan, Z., Ji, T., Luo, Y., Liu, X., Yuan, J., Hua, P., et al · 2023
Later among the works it cites.
When does re-initialization work?
Zaidi, S., Berariu, T., Kim, H., Bornschein, J., Clopath, C., Teh, Y. W., and Pascanu, R · 2023
Later among the works it cites.
Addressing loss of plasticity and catastrophic forgetting in continual learning
Elsayed, M. and Mahmood, A. R · 2024
Closest in time.
Plasticity-optimized complementary networks for unsupervised continual learning
Gomez-Villa, A., Twardowski, B., Wang, K., and van de Weijer, J · 2024
Closest in time.
Disentangling the causes of plasticity loss in neural networks
Lyle, C., Zheng, Z., Khetarpal, K., van Hasselt, H., Pascanu, R., Martens, J., and Dabney, W · 2024
Closest in time.
Nauman, M., Bortkiewicz, M., Ostaszewski, M., Miłoś, P., Trzciński, T., and Cygan, M · 2024
Closest in time.
Small batch deep reinforcement learning
Obando, C., Johan, Bellemare, M., and Castro, P. S · 2024
Closest in time.