Information Aggregation via Dynamic Routing for Sequence Encoding

Understand

While much progress has been made in how to encode a text sequence into a sequence of vectors, less attention has been paid to how to aggregate these preceding vectors (outputs of RNN/CNN) into fixed-size encoding vector.

• Usually, a simple max or average pooling is used, which is a bottom-up and passive way of aggregation and lack of guidance by task information.
• In this paper, we propose an aggregation mechanism to obtain a fixed-size encoding with a dynamic routing policy.
• The dynamic routing policy is dynamically deciding that what and how much information need be transferred from each word to the final encoding of the text sequence.