Measuring and Controlling Internal Activity in Variational Neural Units
DOI: 10.21203/rs.3.rs-9585590/v1
Abstract
Neural networks usually compute with deterministic internal activations. Probabilistic uncertainty is often modeled at the level of weights, global latent variables, or predictive outputs. This motivates a complementary design question: can probabilistic inference be placed inside the units that carry the forward pass? We study whether a neuron can be written as a small variational inference module with measurable internal probabilistic activity. We introduce a local variational neuron. Each unit has a scalar latent variable, a standard normal prior, an amortized Gaussian posterior, a reparameterized sample, and a local Kullback–Leibler regularization term. Its internal activity can be tracked through local KL, squared posterior mean, out-of-range fraction, and optional autoregressive persistence. We evaluate the unit on long-horizon forecasting tasks. DET denotes the matched deterministic baseline and provides the point-accuracy reference. EVE denotes the local variational unit and adds internal measurements induced by the local posterior: KL activity, squared posterior mean, out-of-range fraction, and optional autoregressive persistence. These measurements are active, respond to simple controls, and relate to prediction error across runs. The contribution is a variational computational unit for measuring and controlling local probabilistic activity inside neural networks.
Keywords
- variational inference
- local variational neuron
- internal probabilistic activity
- uncertainty quantification
- forecasting
- model interpretability
- EVE
Citation
Ruffenach, Yves. “Measuring and Controlling Internal Activity in Variational Neural Units.” Research Square, May 4, 2026. Method Article. DOI: 10.21203/rs.3.rs-9585590/v1.
Related research
Variational neuron · Variational Transformer · Internal uncertainty · Variational closure