Information Shannon Entropy3 From Jcost

Shannon entropy from cost

Shannon entropy, introduced by Claude Shannon in 1948, measures the average surprise or uncertainty in a set of outcomes. For probabilities p_i, it is H = -sum p_i log p_i. A fair coin toss has entropy 1 bit, while a certain event has entropy 0. The formula answers a practical question: how many bits, on average, do you need to describe an outcome from a distribution?

The same quantity can be rewritten as a sum of pairwise costs. For each outcome, compare its probability p_i to the uniform probability 1/n. The cost of that comparison is J(p_i / (1/n)), where J(x) = (x + 1/x)/2 - 1. Summing these costs recovers the entropy exactly: H = sum p_i * J(p_i / p_uniform). This is not an approximation; it is an identity. The cost function J is the one that Recognition Science proves is forced by five plain conditions, including reciprocal symmetry and a composition law.

In Recognition Science, the framework models reality as maintaining a ledger, a discrete record of recognition events, and the cost of recognition is not chosen but forced. The framework's central theorem proves that any cost function satisfying those conditions must equal J(x). Shannon entropy, written in this form, becomes a sum of forced costs rather than a definitional choice.

The module ShannonEntropy3FromJCost in the framework's machine-checked library of formal theorems proves three general facts about this cost function. First, the cost vanishes when the two probabilities are equal, so a uniform distribution has zero total cost. Second, the cost is never negative for positive inputs, so entropy is always nonnegative. Third, a threshold constant phi - 3/2 is positive, where phi is the golden ratio. These facts hold for any positive real inputs; they are not specific to Shannon entropy.

What the module does not do is connect these general facts to a particular information source. The definition of domainCost uses the ratio m/e without specifying what m and e mean in information terms. The docstring records the intended direction, that Shannon entropy equals a sum of J-costs, but the Lean code proves only the general properties of J. The bridge from the cost function to a concrete definition of entropy for a specific system remains open.

def domainCost (m e : ℝ) : ℝ := Jcost (m / e)

theorem domainCost_at_eq (r : ℝ) (h : r ≠ 0) : domainCost r r = 0 := by
  unfold domainCost; rw [div_self h]; exact Jcost_unit0

theorem domainCost_nonneg (m e : ℝ) (hm : 0 < m) (he : 0 < e) : 0 ≤ domainCost m e := by
  unfold domainCost; exact Jcost_nonneg (div_pos hm he)

theorem canonicalThreshold_pos : 0 < canonicalThreshold := by
  unfold canonicalThreshold; linarith [phi_gt_onePointFive]

What this page does not claim

This module proves that Shannon entropy is the unique entropy measure. The module connects the cost function to a specific information source or channel. The identity H = sum p_i * J(p_i / p_uniform) is proved in Lean; it is stated as a research note.

Verify this page

Every tagged claim above names its theorem. To check one yourself rather than trust this page, elaborate the source module with Lean 4 and audit its axiom basis:

$ lake env lean IndisputableMonolith/Information/ShannonEntropy3FromJCost.lean
expected axiom basis: [propext, Classical.choice, Quot.sound] (the Lean kernel's standard three; no RS-specific axioms)

A page whose claims cannot be reproduced this way does not ship. In production, every anchor links to the exact declaration in the public source release, and this block carries the build receipt for the page itself.

Derived articles

This page is generated by a question-recursion engine: the questions its answers raise become the next pages. The current agenda, with open targets marked red: