Style-transfer and Paraphrase: Looking for a Sensible Semantic Similarity Metric

Understand

The rapid development of such natural language processing tasks as style transfer, paraphrase, and machine translation often calls for the use of semantic similarity metrics.

• In recent years a lot of methods to measure the semantic similarity of two short texts were developed.
• This paper provides a comprehensive analysis for more than a dozen of such methods.
• Using a new dataset of fourteen thousand sentence pairs human-labeled according to their semantic similarity, we demonstrate that none of the metrics widely used in the literature is close enough to human judgment in these tasks.