The effect of task and training on intermediate representations in convolutional neural networks revealed with modified RV similarity analysis

Understand

Centered Kernel Alignment (CKA) was recently proposed as a similarity metric for comparing activation patterns in deep networks.

• Here we experiment with the modified RV-coefficient (RV2), which has very similar properties as CKA while being less sensitive to dataset size.
• We compare the representations of networks that received varying amounts of training on different layers: a standard trained network (all parameters updated at every step), a freeze trained network (layers gradually frozen during training), random networks (only some layers trained), and a completely untrained network.
• We found that RV2 was able to recover expected similarity patterns and provide interpretable similarity matrices that suggested hypotheses about how representations are affected by different training recipes.