Concept-Best-Matching: Evaluating Compositionality in Emergent Communication

Understand

Artificial agents that learn to communicate in order to accomplish a given task acquire communication protocols that are typically opaque to a human.

• A large body of work has attempted to evaluate the emergent communication via various evaluation measures, with \emph{compositionality} featuring as a prominent desired trait.
• However, current evaluation procedures do not directly expose the compositionality of the emergent communication.
• We propose a procedure to assess the compositionality of emergent communication by finding the best-match between emerged words and natural language concepts.