SQUINKY! A Corpus of Sentence-level Formality, Informativeness, and Implicature

Understand

We introduce a corpus of 7,032 sentences rated by human annotators for formality, informativeness, and implicature on a 1-7 scale.

• The corpus was annotated using Amazon Mechanical Turk.
• Reliability in the obtained judgments was examined by comparing mean ratings across two MTurk experiments, and correlation with pilot annotations (on sentence formality) conducted in a more controlled setting.
• Despite the subjectivity and inherent difficulty of the annotation task, correlations between mean ratings were quite encouraging, especially on formality and informativeness.