EntropyLab · Independent Research Series · May 2026

Irreducible
Path Entropy
in Neural Networks

A quantitative information-theoretic framework for characterising entropy propagation and computational irreducibility across decision trajectories in artificial neural networks.

Ω(N) = 1 − H_irr^(L) / H_path^(L) ∈ [0, 1]

Read Paper Explore

01 — Abstract

The Problem of Computational Opacity

This paper introduces Irreducible Path Entropy (H_path), a quantitative metric for characterising the structural accumulation and reducibility of entropy across computational decision trajectories in artificial neural networks.

The construct is grounded exclusively in information-theoretic and systems-level analysis — without recourse to semantic, cognitive, or anthropomorphic assumptions.

irreducible path entropy information theory entropy accumulation neural networks observability inference dynamics

02 — Mathematical Formalism

Core Constructs

Def 01

Local Path Entropy

H_path(l) = −Σ p_{l,k} log p_{l,k}

Shannon entropy at layer l

Def 02

Cumulative Path Entropy

H_path^(L) = Σ H(P_l)

Total entropy across L layers

Def 03

Reducibility Condition

I(h_l; M_l(y)) ≥ H_path(l) − δ*

Layer is reducible if...

Def 04

Irreducible Path Entropy

H_irr = H_path − H_red

Entropy not recoverable

Def 05

Observability Index

Ω = 1 − H_irr / H_path

0=opaque, 1=transparent

Corollary

Entropic Leakage

Δ(L) = H_path − I(x; h_L)

Uncertainty unexplained

IrreduciblePath Entropyin Neural Networks

The Problem of Computational Opacity

Core Constructs

Samir Baladi

Irreducible
Path Entropy
in Neural Networks