A.4 The fundamental theorem (5-way equivalence)
Listing A.8:
Fundamental theorem of statistical learning (Theorem/PAC.lean:293)
theorem fundamental_theorem (X : Type u) [MeasurableSpace X]
[MeasurableSingletonClass X]
(C : ConceptClass X Bool)
[MeasurableConceptClass X C] :
(PACLearnable X C (*@$\leftrightarrow$@*) VCDim X C < (*@$\top$@*)) (*@$\wedge$@*)
((VCDim X C < (*@$\top$@*)) (*@$\leftrightarrow$@*)
exists (k : N) (cs : CompressionSchemeWithInfo0 X Bool C), cs.size = k) (*@$\wedge$@*)
((VCDim X C < (*@$\top$@*)) (*@$\leftrightarrow$@*)
forall e > 0, exists m0, forall (D : Measure X),
IsProbabilityMeasure D ->
forall m (*@$\geq$@*) m0, RademacherComplexity X C D m < e) (*@$\wedge$@*)
(PACLearnable X C ->
exists (L : BatchLearner X Bool) (mf : R -> R -> N),
-- sample complexity sandwich + lower bound
...) (*@$\wedge$@*)
((VCDim X C < (*@$\top$@*)) (*@$\leftrightarrow$@*)
exists (d : N), forall (m : N), d (*@$\leq$@*) m ->
GrowthFunction X C m (*@$\leq$@*)
Finset.sum (Finset.range (d + 1)) (Nat.choose m)) :=
-- Proof: assembles from component theorems
(*@$\langle$@*)vc_characterization X C,
fundamental_vc_compression X C,
(vc_characterization X C).symm.trans (fundamental_rademacher X C),
pac_sample_complexity_sandwich X C,
(*@$\langle$@*)vcdim_finite_imp_growth_bounded X C,
growth_bounded_imp_vcdim_finite X C(*@$\rangle\rangle$@*)