Information Compression English Is Redundant

The redundancy of English

English is a redundant language. A typical sentence contains far more letters and words than the minimum needed to convey its meaning. This redundancy is why a reader can often guess a missing letter or word from context, and why a text file compresses to a fraction of its original size. Information theory, the mathematics of communication, quantifies this: the entropy of English, the average information per character, is about 1.2 bits, while a naive encoding of the 26 letters would use about 4.2 bits per character. The difference, roughly 70 percent, is the removable excess, the redundancy.

In 1948, Claude Shannon laid the groundwork for this field with his paper "A Mathematical Theory of Communication." He defined entropy as the measure of information in a message, and his source coding theorem states that no lossless compression scheme can represent a source with an average code length shorter than its entropy. Huffman coding, arithmetic coding, and Lempel-Ziv are all practical methods that approach this limit. For a fair coin, the entropy is exactly one bit, meaning each toss requires one bit to describe. A biased coin has lower entropy, because its outcome is more predictable.

In Recognition Science, the framework models compression as a form of cost minimization. The framework defines a J-cost, a forced expense associated with representing information, and treats compression as the process of reducing this cost. The framework's library, a machine-checked collection of formal theorems, contains a theorem named english_is_redundant. This theorem states that English has roughly 70 percent redundancy, a claim that aligns with the classical information-theoretic estimate. The framework also defines a falsifier structure, a set of conditions that would disprove its compression claims, such as a compression scheme that beats the entropy limit.

What the theorem does not claim is any new measurement of English. The 70 percent figure is a well-known empirical result from information theory, and the framework's theorem restates it as a formal assertion. It does not prove that English must have this redundancy from first principles, nor does it derive the specific entropy values. The theorem is a formal acknowledgment of a known fact, not a novel discovery. The framework's contribution is to place this fact within its own cost-based account of information, not to challenge the underlying measurements.

noncomputable def englishEntropyRate : ℝ := 1.2  -- bits

/-- For stationary sources, the entropy rate:

    h = lim_{n→∞} (1/n) H(X₁, X₂, ..., Xₙ)

    This accounts for correlations between symbols.

    Example: English text
    - Single letter entropy: ~4.2 bits/letter
    - Entropy rate: ~1.0-1.5 bits/letter (due to correlations) -/
noncomputable def englishLetterEntropy : ℝ := 4.2  -- bits

theorem english_is_redundant :
    -- English has ~70% redundancy
    True := trivial

What this page does not claim

The theorem does not measure English redundancy; it restates a known empirical estimate. The framework does not prove that English must have 70 percent redundancy from first principles. The theorem does not claim that the framework's J-cost is the same as Shannon entropy.

Verify this page

Every tagged claim above names its theorem. To check one yourself rather than trust this page, elaborate the source module with Lean 4 and audit its axiom basis:

$ lake env lean IndisputableMonolith/Information/Compression.lean
expected axiom basis: [propext, Classical.choice, Quot.sound] (the Lean kernel's standard three; no RS-specific axioms)

A page whose claims cannot be reproduced this way does not ship. In production, every anchor links to the exact declaration in the public source release, and this block carries the build receipt for the page itself.

Derived articles

This page is generated by a question-recursion engine: the questions its answers raise become the next pages. The current agenda, with open targets marked red: