Distributional Neurons: Making Uncertainty a Unit of Computation

Abstract

Neural networks usually propagate point-valued hidden activations. Uncertainty is commonly introduced through weights, outputs, ensembles, stochastic perturbations, or global latent variables. We study a finer granularity: the neuron as a distributional computational unit. We introduce EVE, a variational neuron with an input-dependent posterior, local prior, reparameterized sampling, neuron-level KL regularization, and internal diagnostics. EVE computes through sampled local latent states rather than only parameterizing uncertainty. A single-unit microscope isolates neuron-level latent computation from depth, width, and composition. Against deterministic and capacity-matched deterministic controls, EVE preserves comparable point accuracy and improves NLL, CRPS, and alignment with true heteroscedastic structure. A direct heteroscedastic output control provides a strong output-level uncertainty reference; EVE adds neuron-level posterior states and local KL diagnostics. We then test true dense 128 × 128 MLP composition on Energy Efficiency. EVE improves NLL, CRPS, and pinball loss on all five test seeds; MSE and MAE remain close but are not uniformly improved on test. Finally, a PG-19 Transformer feed-forward bridge shows improved validation CE, perplexity, accuracy, and Monte Carlo NLL over a matched deterministic Transformer, with non-degenerate Monte Carlo uncertainty signals. The results support neuron-level uncertainty as a feasible computational primitive that can be isolated at the unit level, composed in deep dense networks, and inserted into Transformer feed-forward computation.

Keywords

• distributional neurons
• variational neurons
• uncertainty quantification
• local latent states
• heteroscedastic regression
• Transformer language modeling
• EVE

Citation

Ruffenach, Yves. “Distributional Neurons: Making Uncertainty a Unit of Computation.” Research Square, 2026. Research Article. DOI: 10.21203/rs.3.rs-10401116/v1.
Related Zenodo archive: 10.5281/zenodo.20237047.

Related research

• Variational Distributional Neuron
• The Neuron Is the Distribution
• The Neuron as a Latent State: Classical Variational Readout in Distributional Neural Units
• Measuring and Controlling Internal Activity in Variational Neural Units
• Variational Neurons in Transformers for Language Modeling

Variational neuron · Variational Transformer · Internal uncertainty · Variational closure