Reflections on Shannon Information: In search of a natural information-entropy for images

Understand

It is not obvious how to extend Shannon's original information entropy to higher dimensions, and many different approaches have been tried.

• We replace the English text symbol sequence originally used to illustrate the theory by a discrete, bandlimited signal.
• Using Shannon's later theory of sampling we derive a new and symmetric version of the second order entropy in 1D.
• The new theory then naturally extends to 2D and higher dimensions, where by naturally we mean simple, symmetric, isotropic and parsimonious.