Provable learning from the loop wetware wants to own

What the work claims

This is a primary theory paper with a supporting empirical section, and its central claim is precise: for a framework the authors call NeuroEvolutionary Online Learning (NEOL), in which an outer evolutionary process selects among neural network architectures while each candidate network adapts its weights online through reward-modulated local plasticity, expected regret grows sub-linearly in time.¹ Regret here is the cumulative performance gap against the best fixed choice in hindsight; sub-linear growth means the average gap vanishes asymptotically, so the learner is, in a formal sense, eventually as good as the best fixed alternative it could have committed to.

The authors claim novelty on the theory side: prior formal work on neuroevolution concerns runtime analysis of architecture search for discrete problems, and to their knowledge no regret analysis of the online, continuous variant existed.¹ Empirically, they report that a NEAT-based implementation, NEAT-NEOL, beats standard NEAT on three of four control benchmarks with statistical support from one-sided Wilcoxon rank-sum tests at p under 0.05 across thirty seeds, and is competitive with PPO and SAC, two strong reinforcement learning baselines, on the two easier tasks under a matched budget of ten million environment steps.¹²

How it works

The architecture is a loop within a loop, running at two timescales. The slow outer loop is evolutionary: a population of network topologies is maintained, and selection reweights them by fitness. For the theory, this messy population process is modeled as an exponential-weights (Hedge) update, a softmax reallocation of probability over a finite menu of architectures based on observed losses. The fast inner loop is where the biology enters: within an episode, each network's weights change by three-factor plasticity rules. In a three-factor rule, a synapse's change depends on presynaptic activity, postsynaptic activity, and a third gating signal correlated with reward. The paper analyzes three classical forms: reward-modulated Hebb (change proportional to reward times the activity product), Oja (Hebb with a decay that keeps weights bounded), and BCM (a sliding threshold that switches potentiation and depression). No gradients are backpropagated anywhere; every update is local, which is exactly what makes such rules biologically plausible.¹³

The regret bound then decomposes cleanly: the inner loop, viewed as projected online learning on a linear surrogate loss, contributes a term scaling with the square root of time multiplied by the weight-space diameter and the update-magnitude bound; the outer Hedge loop contributes a term scaling with the square root of time multiplied by the logarithm of the number of architectures. Both are sub-linear, so their sum is. The empirical section instantiates the framework on NEAT, the classic neuroevolution algorithm,⁴ and runs CartPole, LunarLander, Hopper, and BipedalWalker with population sizes from 50 to 300, 500 generations, and thirty seeds per configuration, with ablations that disable the plasticity to isolate its contribution.¹

Where a skeptic should push

The single most load-bearing assumption is A5, which the authors call surrogate domination: the decrease in true loss is assumed to be controlled by the decrease in a linear surrogate whose gradient is the plasticity update itself. Informally, it assumes the reward-modulated plasticity direction is positively aligned with descent on the real objective. But that alignment is the entire open question about Hebbian-family learning; assuming it moves the hard part of the problem into the hypotheses. The result is best read as: if plasticity locally helps, a two-timescale system built on it provably converges on average. That is genuinely useful, and honestly framed in the paper as a simplified abstraction, but it is not a proof that reward-modulated Hebbian learning solves control problems.

Second, the outer loop of the theory is a caricature of the outer loop of the practice. NEAT maintains species, crosses over genomes, and grows topologies in an unbounded space; the theorem covers softmax reweighting over a finite, fixed menu (assumptions A1 and A6). Third, regret is measured against the best fixed architecture running the same inner trajectory, not against the best achievable policy, which is a weaker comparator than it may appear. Fourth, the empirical scale is classic control: on the two harder tasks, Hopper and BipedalWalker, PPO and SAC still win outright, and the claimed advantage narrows to higher medians and tighter variance over pure NEAT. Fifth, this is a version-1 preprint; the appendix discloses LLM assistance in preparation, the code sits in a personal GitHub repository, and none of the results have completed review. Nothing here is suspicious, but nothing here is settled either.

A guarantee silicon signs and tissue cannot

The relevance to computing on living neural tissue is structural, and the paper never mentions it. The two-timescale loop NEOL formalizes is precisely the architecture of every serious wetware training pipeline: an outer optimizer in software chooses what to try (stimulation protocols, encodings, culture parameters) while an inner living substrate adapts through activity-dependent plasticity it does not expose to inspection. The published closed-loop neuron experiments, from the Pong-playing cultures onward, are instances of exactly this outer-designer, inner-plastic-learner configuration.⁵ NEOL hands that configuration its first regret-style vocabulary, and vocabulary is capability: it tells a platform builder what to log, what to bound, and how to split an interaction budget between exploring protocols and exploiting adaptation.

The second implication is a hype correction with procurement consequences. The sample-efficiency and robustness gains this paper demonstrates come from reward-modulated plasticity as an algorithm, running on ordinary hardware. Every property of biological learning that can be abstracted into a local update rule and shown to work in silico shrinks the residual claim that living neurons are uniquely necessary to obtain it. Vendors of biological compute lean heavily on learning efficiency as the differentiator; a buyer reading NEOL should ask which part of that differentiator survives once the learning rule itself is commoditized in software, where it is cheaper, faster, reproducible, and ethically free. What plausibly survives is what resists abstraction: the energetics, the rich temporal dynamics, and whatever else a real culture does that Hebb, Oja, and BCM do not capture. That residue is currently unmeasured by anyone.

The third implication runs the opposite direction, and it is the governance one. Read the paper's assumptions as a checklist of what living tissue refuses to provide: bounded activities, weights confined to a compact set by explicit projection, a chosen step size, a stationary update rule. A cultured network drifts, saturates, reorganizes, and dies on a timescale of months. The same mathematics that lets a silicon two-timescale learner carry a performance guarantee marks the biological version as structurally guarantee-resistant: wetware service agreements will be empirical claims about a particular culture on a particular week, not analytic properties of the system. Any procurement or regulatory framework that expects contract-grade predictability from tissue is asking for what the theory says is not on offer. And there is a genuine threat inside the blueprint: better formal tools for outer loops make outer loops better at shaping inner learners, and an outer optimizer relentlessly applying selection pressure to living neural tissue is precisely the configuration that gives welfare frameworks pause. The non-obvious point is that regret itself, cumulative shortfall against a benchmark, is a better-shaped quantity for governing adaptive living systems than accuracy: a welfare-aware variant would bound cumulative harm to the substrate the way NEOL bounds cumulative loss to the operator. Nobody has written that paper.

The bottom line

Established: a clean formal framework, a regret bound that is correct under its stated assumptions, and statistically supported evidence that adding reward-modulated plasticity to NEAT improves fitness and reliability on three small control tasks. Hypothesis: that the alignment assumption A5 holds in realistic settings, that the result extends beyond a finite architecture menu and classic control, and that any of this transfers to substrates whose plasticity nobody chose. Confirmation would look like peer review, harder environments, and a relaxation of A5 into something checkable; a break would be evidence that the surrogate domination condition fails exactly where the tasks get interesting. For the living-tissue computing field, the paper matters regardless: it moves the claimed magic of biological learning one step further into the set of things silicon does with proofs attached, and it quietly documents why the tissue version will never come with one.