Crossword: A Semantic Approach to Data Compression via Masking

Understand

The traditional methods for data compression are typically based on the symbol-level statistics, with the information source modeled as a long sequence of i.i.d.

• random variables or a stochastic process, thus establishing the fundamental limit as entropy for lossless compression and as mutual information for lossy compression.
• However, the source (including text, music, and speech) in the real world is often statistically ill-defined because of its close connection to human perception, and thus the model-driven approach can be quite suboptimal.
• This study places careful emphasis on English text and exploits its semantic aspect to enhance the compression efficiency further.