Four open preprints, each with its own permanent HTML landing page, direct PDF address, DOI, complete abstract and machine-readable bibliographic metadata.
July 29, 2026 · Preprint · Open
Variational distributional neurons make internal uncertainty directly measurable by representing each activation as a learned posterior distribution conditioned on the current input. Elemental Variational Expanse (EVE) implements this principle in a language-model unit that exposes local Kullback–Leibler (KL) activity, posterior scale, posterior mean energy, and active-unit fraction during forward computation. A controlled next-token prediction experiment compares EVE with a deterministic model and Monte Carlo (MC) Dropout across five shared random seeds, with a five-member deterministic ensemble serving as an additional predictive reference. Raw EVE achieves a mean Monte Carlo negative log-likelihood of 5.359, compared with 5.620 for the deterministic model and 5.621 for MC Dropout. Mean perplexity is 212.743, versus 275.845 and 276.161, respectively. Mean next-token accuracy is 0.166, versus 0.148 and 0.153, while mean tail-aware conditional value-at-risk negative log-likelihood (CVaR-NLL) is 12.713, versus 13.200 and 13.213. Mean expected calibration error remains comparable across the models, at 0.042 for EVE, 0.043 for the deterministic model, and 0.046 for MC Dropout. Validation-selected readouts provide explicit calibration-oriented, balanced, and tail-oriented operating regimes. As context-token perturbation increases from 0.05 to 0.30, mean internal Kullback–Leibler activity rises from 0.291 to 0.362, posterior scale increases from 0.874 to 0.899, and posterior mean energy rises from 0.052 to 0.075. These coordinated measurements demonstrate that unit-level posterior activity can be quantified alongside predictive behavior during language-model computation. They provide a concrete measurement basis for scaling variational distributional units to larger model architectures.
Publication page → DOI ↗
July 29, 2026 · Preprint · Open
Large language-model research increasingly relies on scale, which can make mechanism isolation difficult. We study the complementary small-scale regime as a controlled setting for architectural discovery. We examine EVE, a variational distributional neuron that replaces selected deterministic hidden activations with input-conditioned Gaussian latent states and exposes their local mean, variance, and Kullback–Leibler activity during computation. We evaluate EVE on a language-modeling benchmark comprising 1% of the WritingPrompts-Filtered dataset, using five matched random seeds and a reproducible experimental pipeline. Relative to the deterministic baseline, EVE reduces mean test negative log-likelihood by 4.63% (5.359 ± 0.039 versus 5.620 ± 0.022), reduces perplexity by 22.88%, increases next-token accuracy by 11.93%, and improves conditional value-at-risk negative log-likelihood by 3.69%. EVE outperforms the deterministic baseline across all five paired seeds for negative log-likelihood, accuracy, and conditional value-at-risk negative log-likelihood. The paired negative log-likelihood difference is −0.260, with an exploratory 95% interval ranging from −0.330 to −0.191, while the mean calibration error remains comparable. Under covariate non-stationarity, aggregate negative log-likelihood increases from 5.29 to 8.06 as context-token corruption rises from 0 to 0.30, while internal Kullback–Leibler activity increases from 0.274 to 0.362. Validation-selected readouts further reveal a risk-sensitive readout gap. In a representative run, a readout selected for expected calibration error preserves negative log-likelihood while reducing conditional value-at-risk negative log-likelihood from 12.84 to 12.08. A balanced readout increases next-token accuracy to 0.170 while reducing conditional value-at-risk negative log-likelihood to 11.94. These results establish EVE as a small-scale mechanism for measurable internal distributional activity and motivate further parameter-matched and larger-scale evaluations.
Publication page → DOI ↗
July 28, 2026 · Preprint · Open
Language-model evaluation commonly focuses on final predictive distributions through likelihood, calibration, accuracy, entropy, and variability across predictions. Internal probabilistic activity offers a complementary measurement level by describing how distributions are represented within the computation that produces those predictions. EVE, the Elemental Variational Expanse, is a variational language-model unit that maps a selected hidden activation to an input-conditioned Gaussian posterior. During the forward computation, the unit exposes divergence from a reference prior, posterior scale, posterior mean energy, and active-unit fraction. These quantities form an internal measurement panel evaluated alongside deterministic, dropout-based, and ensemble-based predictive references. Controlled evaluations show competitive predictive behavior together with directly observable posterior activity. Validation-selected posterior readouts provide distinct calibration and tail-risk operating points, while controlled changes in context quality and target frequency produce systematic variation in both output metrics and internal posterior statistics. The resulting framework combines predictive evaluation with explicit probabilistic measurements inside language-model computation.
Publication page → DOI ↗
July 27, 2026 · Preprint · Open
Language models commonly represent hidden computation through deterministic activations, while uncertainty is estimated primarily from their outputs. We introduce Elemental Variational Expanse (EVE), an architecture built from variational distributional neurons that represent selected hidden activations as input-conditioned posterior distributions. Each unit produces a posterior mean and scale, samples a latent activation through reparameterization, and exposes local Kullback–Leibler activity together with activation and disagreement diagnostics during the forward pass. This design makes probabilistic structure directly observable within the computation and connects neuron-level dynamics with predictive behavior, calibration, and robustness. We evaluate EVE in controlled next-token prediction against deterministic, Monte Carlo Dropout, and ensemble references, supported by ablations and non-stationarity analyses that examine stochastic sampling, posterior scale, regularization, and internal–external relationships. Across these evaluations, EVE improves predictive quality, provides informative internal diagnostics, and supports reliability-oriented monitoring and readout strategies. Variational distributional neurons therefore offer a general computational primitive for uncertainty-aware language models and, more broadly, for neural systems designed to preserve and use multiple plausible internal states.
Publication page → DOI ↗