One input. One activation.
A conventional neuron compresses its input into a single scalar activation. It is precise, stable and efficient, but it collapses local possibilities immediately into one point.
h = φ(wᵀx + b)
Variational intelligence, built from the neuron up
We’re building the next generation of models: Variational LLMs.
JWAYU 杰维宇 develops language models in which uncertainty is not added only at the output. It becomes part of the internal computation itself, through EVE — the Elemental Variational Expanse.
Local distributionsinside hidden computation
Measurable uncertaintyKL, variance, mutual information
Transformer-readyintegrated into language modeling
Our thesis
The network is not the starting point. The neuron is.
Modern language models are extraordinarily capable, yet their internal units remain largely deterministic. JWAYU begins with a different premise: when ambiguity is intrinsic to intelligence, the computational unit should be able to represent it.
One input. A structured space of possibilities.
An EVE unit constructs an input-conditioned posterior, samples a latent state through reparameterization and uses that state directly in computation.
qφ(z|x) = 𝒩(μ(x), diag(σ²(x)))
EVE — Elemental Variational Expanse
What is a variational neuron?
It is a local probabilistic computational unit with an explicit prior, an amortized posterior and unit-level variational regularization. Its activation is no longer merely a value: it is generated from a learned distribution.
Condition
The unit receives the current representation and produces posterior parameters.
Represent
A local Gaussian posterior represents a continuous set of plausible latent states.
Sample
Reparameterization produces a differentiable latent sample used in the forward pass.
Constrain
A local KL term regularizes the posterior against its prior and makes the unit’s operating regime measurable.
Point computation
Propagates one activation. Uncertainty is not represented inside the unit.
Randomized computation
Introduces randomness through masks or sampled paths, without turning each unit into an explicit input-conditioned latent posterior.
Probabilistic computation
Maintains a learned local distribution with a prior, a posterior, a latent sample and internal diagnostics.
A language model should not have to collapse uncertainty before it has used it.
A Variational language model introduces EVE units into the model’s hidden computation, including Transformer feed-forward blocks. The overall architecture remains recognizable, while its internal computational primitive changes.
Instead of treating uncertainty as a final confidence score, the model can carry local latent structure across layers and use internal signals such as posterior variance, KL divergence and mutual information for evaluation, calibration and control.
From neuron to model
A research program across four layers.
JWAYU develops the primitive, studies its operating regime, integrates it into Transformers and turns internal uncertainty into an actionable signal.
The neuron is the distribution
Define local variational computation and make its internal probabilistic state observable.
Build networks from EVE units
Study stability, capacity, latent dimensionality, temporal persistence and collapse control.
Variational Transformers
Integrate EVE into feed-forward computation while preserving a practical Transformer backbone.
Turn uncertainty into an operational signal
Use internal diagnostics for monitoring, checkpoint selection, routing and inference-time intervention.
Earlier work
Foundational arXiv publications.
These papers introduce the neuron, map its design space, integrate it into Transformers and explore uncertainty as an agentic control signal.
Variational Distributional Neuron
Introduces the proof of concept: a computational unit formulated as a local variational primitive with a prior, amortized posterior and local regularization.
Exploring the Dimensions of a Variational Neuron
Studies latent dimensionality, local capacity, temporal persistence and the measurable internal operating regime of EVE.
Variational Neurons in Transformers for Language Modeling
Places EVE units inside Transformer feed-forward computation and evaluates predictive quality, calibration and uncertainty signals.
Agentic Control in Variational Language Models
Explores internal uncertainty as an operational signal for regulation, checkpoint retention and inference-time intervention.
Research
Publications.
Research Square articles and Zenodo preprints.
Variational Neurons and Bayesian Neural Networks: Distinct Probabilistic Objects, Inference Loci, and Computational Granularity
Separates Bayesianity, variationality, stochasticity and distribution-valued computation; formalizes a unit-level variational-neuron specification criterion and tests it through controlled EVE experiments, including a fresh exact-capacity 50-seed comparison.
DOI: 10.5281/zenodo.21870089
Distributional Neurons: Making Uncertainty a Unit of Computation
EVE makes uncertainty a neuron-level computational primitive and evaluates it from a single-unit microscope to dense networks and Transformer feed-forward computation.
The Neuron Is the Distribution
Develops the architectural thesis that distributional representation can reside inside the neuron rather than only in weights, outputs or global latent variables.
The Neuron as a Latent State: Classical Variational Readout in Distributional Neural Units
Freezes the learned EVE posterior and varies only the readout rule to test whether a neuron-level latent state carries predictive information beyond its mean.
Measuring and Controlling Internal Activity in Variational Neural Units
Defines a local variational neuron and a measurement protocol for KL activity, posterior mean energy, out-of-range units and optional autoregressive persistence.
Measuring Internal Probabilistic Activity with Variational Distributional Neurons
Yves Ruffenach
Measures unit-level posterior activity alongside predictive quality, calibration and tail-aware evaluation in next-token prediction.
DOI: 10.5281/zenodo.21668942
Variational Distributional Neurons for Measurable Internal Uncertainty in Language Models
Yves Ruffenach
Evaluates EVE as a controlled small-scale mechanism for measurable internal distributional activity across five matched random seeds.
DOI: 10.5281/zenodo.21669216
Measuring Local Posterior Activity in Variational Language Model Units
Yves Ruffenach
Introduces an internal measurement panel for local posterior activity and relates it to output-level predictive behavior.
DOI: 10.5281/zenodo.21641827
Learning Distributions Inside a Language Model: Variational Neurons for Measurable Internal Uncertainty and Reliability Monitoring
Yves Ruffenach
Presents EVE as a language-model architecture for measurable internal uncertainty, reliability monitoring and uncertainty-aware computation.
DOI: 10.5281/zenodo.21632472
For investors and strategic partners
A new computational primitive for uncertainty-aware AI.
JWAYU 杰维宇 is building intellectual property, experimental evidence and implementation expertise around Variational neurons and language models.
Change the unit, open a new model space.
Distinct technical thesis
Move probabilistic structure from only global mechanisms or outputs into hidden computational units.
Measurable internal state
Expose local signals that can support calibration, robustness analysis, routing and model control.
Architecture-compatible path
Develop the primitive inside familiar neural and Transformer backbones rather than replacing the entire ecosystem.
Published research trajectory
A growing sequence of public preprints establishes the concept, design space, language-model integration and agentic use.
Research · Engineering · Collaboration
Build the next generation of language models with us.
We welcome conversations with researchers, frontier-model teams, engineers, doctoral partners and organizations interested in uncertainty-aware multimodal and language models.