Non-Iterative Knowledge Fusion in Deep Convolutional Neural Networks

Understand

Incorporation of a new knowledge into neural networks with simultaneous preservation of the previous one is known to be a nontrivial problem.

• This problem becomes even more complex when new knowledge is contained not in new training examples, but inside the parameters (connection weights) of another neural network.
• Here we propose and test two methods allowing combining the knowledge contained in separate networks.
• One method is based on a simple operation of summation of weights of constituent neural networks.