Cross-entropy on FintypePMF

FintypePMF.crossEntropy Q P = ∑ h, Q.prob h · log (1 / P.prob h), the Fintype-indexed ℝ-valued cross-entropy. Related to KL by H(Q, P) = H(Q) + KL(Q ‖ P) where H(Q) is Shannon entropy.

noncomputable def ProbabilityTheory.FintypePMF.crossEntropy {α : Type u_1} [Fintype α] (Q P : FintypePMF α) : ℝ

Cross-entropy between two FintypePMFs: H(Q, P) = ∑ h, Q.prob h · log(1/P.prob h).

Equations

• Q.crossEntropy P = ∑ h : α, if Q.prob h = 0 then 0 else Q.prob h * Real.log (1 / P.prob h)