Co-search proposes, mutates, and discards thousands of structurally distinct programs. NDVM keeps them as runtime data and makes the one differentiable evaluator fast: exact-gradient calibration becomes cheap enough to sit inside the search loop.
The cost is representation, not arithmetic
Before proposing anything, the paper locks a cost model of the system it replaces: DMCI's eager PyTorch backend, profiled on one CPU core over five programs spanning a 2,300× range of cost, from a scalar expression to an 80-step Kalman-filter rollout.
Phase-0 cost model of the eager backend.
Where an interpreter's time goes
An interpreter that keeps the program as data must represent, at runtime, every intermediate value the program produces: not just numbers but tags, symbols, pairs, addresses, and closures. In an eager tensor framework the natural encoding is a small tagged tensor per value, and that encoding, not the floating-point work, is where the time goes. A single forward pass of the 80-step Kalman likelihood issues about two million value-boxing calls against roughly three thousand arithmetic calls.
The same profile shows forward time is essentially independent of how many parameter vectors are evaluated at once: widening the payload 1,024× changes forward time by under about 4%. The marginal cost of one more batch lane is the dense arithmetic and nothing else.
The diagnosis, and the target
If 85–90% of the cost is how values and control are represented, and batching is nearly free, then the right move is not to compile each program away into its own graph. It is to make the interpreter itself fast, keep programs as data, and let one structural walk carry an entire population of parameter vectors.
Split structure from numbers
Discrete structure is never differentiated, so it becomes native scalar data. Numbers are the only differentiable quantities, so they get dense buffers that carry the batch axis. Control flow stays discrete and exact; the reverse pass records payloads only.
pid) into a dense payload table whose leading axis is the batch, with
separate primal and adjoint buffers. Non-numeric values allocate no tensor at all.Structural values, scalar
Tags, interned symbols, heap addresses, closures, and
environments are native scalar data. The heap is a write-once arena (the modeled subset excludes
mutation): cons appends a cell, car/cdr are direct loads.
None of it is differentiable; none of it allocates tensors.
Control: the exact realized trace
Branch predicates, tag tests, and dispatch are discrete routing decisions, never smoothed. Gradients are exact on the trace that actually executed; when lanes of a batch diverge at a numeric branch, per-lane masks keep every lane's result correct. A parameter that appears only in a predicate honestly receives no gradient through it.
Reverse pass: a payload-only tape
The forward pass appends a tape node only for differentiable numeric primitives (add, multiply, exp, matmul, reduce, and so on). Symbol lookups, tag tests, allocation, and branch dispatch emit nothing. Tape size tracks the numeric work actually done, not program size or interpreter recursion depth.
The defining invariant
Make the interpreter fast without compiling away the interpreted program. NDVM unrolls the evaluator's instruction stream, never the object program: the runtime memoizes decoding and variable lookups but never residualizes a program into its own graph. That is what separates it from staging and partial evaluation, and it is why any program the search proposes inherits gradients immediately.
Co-search: an LLM proposes, NDVM calibrates
In program-and-parameter co-search, an outer loop such as OpenEvolve (Sharma, 2025), an open-source implementation of DeepMind's AlphaEvolve approach (Novikov et al., 2025), proposes discrete program structures, and an inner loop calibrates each candidate's continuous constants by exact gradients. The program page introduces the concept; this section measures what NDVM changes: the inner loop stops being the bottleneck.
Discovery frontier and per-calibration throughput, measured on a single CPU node.
The Amdahl honesty
The inner fit is 96.8% of per-candidate wall-clock on the eager backend; NDVM drops that share to 0.654, so calibration stops being the bottleneck and the model-proposal step becomes the next floor. With a live LLM at 0.5 s per candidate the end-to-end gain is 405× (108× at 2.0 s): the loop becomes LLM-bound. The paper reports calibration throughput as the headline and the end-to-end number as Amdahl-bounded, not as an unbounded discovery speedup.
Scope of the measurement
Both fixed-budget searches replay offline-cached streams of LLM-proposed programs, compile-validated on both backends in advance, so the proposing model is out of the timed loop and the two backends see the identical candidate stream. Each candidate has a distinct structural skeleton, the regime where per-candidate staging would have to restage every one.
The science case studies of this research program, battery capacity-fade knee recovery, the LIM-ENSO Kalman-filter MLE, and the FluZoo negative result, are results of the DMCI paper, not of NDVM: see co-search on the DMCI page. NDVM's claim is narrower and mechanical: the same calibrations, orders of magnitude cheaper.
Measured results
All charts are drawn from the paper's committed results (a single CPU node,
float32, code at tag paper-ndvm-arxiv). Hover for exact values.
1Throughput becomes discovery
Same candidate stream, same 900-second budget, two backends. NDVM reaches its first held-out R² > 0.9 at 12.2 s on the scalar task (the eager backend needs 288.2 s) and finishes at R² 0.9962 vs 0.9691. On the recurrence-heavy task the eager backend completes only 117 calibrations and never reaches a successful fit.
2One walk carries the population; cores carry the candidates
Batch-native execution keeps the structural walk scalar and widens only the payloads, so per-lane cost falls as B grows: about 60× on the native path from B=1 to B=256, and about 21× when measured through the PyTorch autograd boundary, as plotted here. Independent candidates additionally fan out across cores near-linearly.
3The inner loop, as throughput
The robust, schedule-independent headline is how many candidate programs a fixed compute budget can calibrate. Both backends run the same 30-step Adam fit to a bit-identical NLL trajectory; the difference is purely runtime representation and native execution.
What the one big number conflates
The raw forward speedup of the native runtime over the eager backend mixes two effects. A tuned-eager baseline disentangles them: removing the boxed representation alone is worth about 4–5× in eager Python, and native execution is the residual, about 8× to 2,133× on top, growing with rollout depth. The clean claims are exact-gradient equivalence, the batch multiplier, and this decomposed speedup.
Correctness is checked continuously: the native runtime reproduces the frozen reference backend's forward outputs and per-parameter gradients across the program suite, including the 80-step Kalman rollout's noise-covariance gradients through the matrix-adjoint path.
Two independent clients, one runtime
NDVM is a runtime representation, not a faster DMCI. Two clients with different dispatch models share the same 120-line value and autodiff interface: the DMCI tree-walking evaluator, and a differentiable stack-bytecode VM that adds only 219 lines of its own.
A differentiable stack-bytecode VM
The second client is a linear instruction stream with an explicit operand stack and 15 opcodes:
PUSH LOAD ADD SUB
MUL DIV NEG EXP LOG
SIN COS DUP LOOP BRANCH
DOT
It contains no DMCI parser or evaluator code; it imports only the structural/numeric-split value box. Against a naive baseline that fuses every stack value into a single tagged tensor, the split is 1.77–3.34× faster single-lane, and because one walk over the instruction stream serves all lanes, per-lane cost falls about 240× from B=1 to B=256. Gradients are bit-identical to reverse-mode autodiff and agree with central finite differences on every leaf.
| workload | fused eager, B=1 (ms) | split, B=1 (ms) | split per-lane, B=256 (µs) | speedup |
|---|---|---|---|---|
| W1 scalar expression | 0.297 | 0.124 | 0.512 | 2.39× |
| W2 counted loop | 1.387 | 0.430 | 1.775 | 3.23× |
| W3 branch | 0.136 | 0.076 | 0.308 | 1.77× |
| W4 matrix-vector | 0.668 | 0.200 | 0.824 | 3.34× |
Randomized differential testing
A randomized differential tester compares the native runtime against the frozen reference backend over the supported language surface, gating forward outputs, gradients, and central finite differences: 600/600 checks pass at the primary seed and 1500/1500 on a second seed. The same tester documents the precise boundary of the supported subset.
When NDVM wins, and when it does not
The paper states its own crossover points. NDVM is the right tool in the co-search regime; outside it, other engines are.
NDVM wins when, together:
- Many structurally distinct candidate programs run through one fixed evaluator, so nothing can be staged and reused.
- Each candidate is fitted for only a handful of gradient steps before it is mutated or discarded.
- Representation and control dominate the numeric work, as the Phase-0 model measures (85–90% of forward time; arithmetic about 1%).
It does not win when:
- A fixed program is reused for thousands of optimizer steps: per-candidate staging amortizes and overtakes NDVM after roughly 213 to 42,000 steps per candidate (co-search does far fewer). Cached-template staging overtakes only above about 80 constant-only reuses per skeleton.
- Control diverges heavily across the population: lane masking stays correct, but each side of a divergent branch is walked under its active lanes, so the single-walk amortization erodes toward per-lane cost.
- Dense numeric kernels dominate: high-dimensional, throughput-bound work belongs in numeric libraries and accelerators, not in the VM.
- The program needs features outside the supported surface: the heap is a write-once arena, with no mutation, no exceptions, and no first-class continuations.
What is measured, and what is future work
The gap between what is measured and what remains designed is the most important thing for a reader to calibrate.
scope All interpreter results are CPU
The measured workload is allocation- and control-bound, the regime where a latency-optimized core beats a throughput-optimized accelerator. NDVM is a CPU-native runtime by design; a GPU appears only as a forward-only numeric-ceiling proof of concept in an appendix, with no interpreter dispatch, heap, tape, or gradient.
future GPU interpreter
The persistent-kernel evaluator that would run the structural walk on the device under warp-ballot lane masks is designed, not built. Its branchy structural walk is the real risk on a GPU, so the paper gates the claim behind that measurement rather than making it.
future MLIR / Enzyme lowering
An optional compiler lowering of the evaluator through MLIR or Enzyme is named as future work. The baseline semantics must keep working without it: programs remain runtime data.
future Formal gradient-correctness proof
Property 1 (trace-constant gradients) is stated as intended semantics and validated empirically against the frozen reference backend across the program suite; it is not a machine-checked theorem. A formal operational semantics and proof are planned.
Read it, run it, cite it
The code, the profiling harness, the differential tester, and the second-client
demonstrations are released; paper-ndvm/REPRODUCE.md maps every figure and table to a
script. The paper is on arXiv at
arXiv:2607.03574.
The code →
github.com/sheneman/nncompile, branch ndvm, frozen at tag
paper-ndvm-arxiv. Native C++ runtime, harnesses, tests, and the artifact guide.
Paper 2: DMCI →
Compile Once, Differentiate Everywhere. The differentiable meta-circular interpreter whose eager backend NDVM's cost model dissects. arXiv:2606.09930.
Paper 1: Neural Compiler →
The Neural Compiler. Where the program began: compiling first-order Scheme into exact differentiable modules. arXiv:2605.22498.
See the full Neural Compiler → DMCI → NDVM lineage on the program page.
Cite this paper
@article{sheneman2026ndvm,
title = {Differentiate the Evaluator, Not the Program:
An Efficient Runtime Representation for Neuro-Symbolic Learning},
author = {Sheneman, Lucas},
year = {2026},
journal = {arXiv preprint arXiv:2607.03574},
eprint = {2607.03574},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
doi = {10.48550/arXiv.2607.03574}
}