Softmax is not Enough (for Sharp Size Generalisation)

Understand

A key property of reasoning systems is the ability to make sharp decisions on their input data.

• For contemporary AI systems, a key carrier of sharp behaviour is the softmax function, with its capability to perform differentiable query-key lookups.
• It is a common belief that the predictive power of networks leveraging softmax arises from "circuits" which sharply perform certain kinds of computations consistently across many diverse inputs.
• However, for these circuits to be robust, they would need to generalise well to arbitrary valid inputs.