Bounded-Horizon Local Transformer Training on CPUs: Quality, Throughput, and Memory
A prospectively specified study of bounded-horizon local training on many-core CPUs
Vikram Lex
Preprint status: Posted. The journal manuscript is under review at the Journal of Artificial Intelligence Research.
Research Square · Under Review at JAIR
Last updated: July 28, 2026, 7:38 GMT+5:30
Paper Summary
Local objectives expose parallelism across network depth by removing cross-stage backward dependencies, but they alter end-to-end credit assignment and complicate ownership of a shared output readout. We evaluate the quality, throughput, and memory trade-off of bounded-horizon local training for 24-layer byte-level Transformers on a many-core CPU, and investigate how upstream local losses should participate in shared-readout updates. We introduce readout-gradient consensus (RGC), in which upstream stages differentiate local losses through same-microbatch readout snapshots and the final stage averages their contributions. Six matched methods are evaluated under a prospectively specified protocol using paired seeds, held-out testing, repeated systems measurements, two corpora, and three model widths. On a 31-core allocation, asynchronous RGC reaches 1.382 times the throughput of matched global backpropagation, with a paired 95% interval from 1.364 to 1.400, while peak process-tree proportional set size increases from 1.94 to 4.31 GiB. The comparison uses 29 stage-worker threads versus 24 intra-operation threads for backpropagation. The selected asynchronous broadcast method has a held-out mean quality gap of 0.841%, but its one-sided 95% upper bound is 2.095%; therefore, the prespecified 1% non-inferiority criterion is not met. Additional enwik8 widths satisfy their joint criteria, whereas TinyStories does not, and readout equivalence is not established. The evidence supports a bounded CPU quality–throughput–memory frontier for this implementation, not quality-preserving training or a general readout-recovery mechanism.
Bounded-horizon Training
The study separates truncated block credit, shared-readout participation, and readout snapshot lag under a declared CPU protocol.
Bounded-Horizon Surrogate
A 24-layer byte-level Transformer is partitioned into detached stages. Each block receives reverse-mode gradients from a local exit rather than the full downstream pullback, exposing concurrency while changing the objective.
Readout-Gradient Consensus
Upstream stages publish exact same-microbatch gradients for the readout snapshot they used, and the final-stage owner averages them. No activation gradient, block gradient, or optimizer state crosses a stage boundary.
Prospective Comparison Protocol
Six matched methods use paired seeds, an independent held-out decision, repeated systems measurements, two corpora, and three widths without relaxing criteria after results are observed.
Measured Outcomes
Throughput improved in the tested CPU setting, but the confirmatory quality, transfer, and readout-recovery decisions remain negative where their criteria were not met.
CPU Systems Frontier
31-core allocation with repeated paired measurements
The paired 95% interval is 1.364–1.400, satisfying the declared throughput criterion in this implementation and hardware setting.
Peak proportional set size more than doubles, so the throughput result is part of a quality–throughput–memory trade-off rather than a free speedup.
RGC uses 29 stage-worker threads while matched global backpropagation uses 24 intra-operation threads; the asymmetry is reported explicitly.
Held-out Quality Decision
Prespecified one-sided 1% non-inferiority criterion
The selected asynchronous broadcast method's mean gap is below the nominal margin, but the decision depends on the confidence bound as prespecified.
Because the upper bound exceeds 1%, held-out non-inferiority is not established even though the observed mean is below 1%.
The study does not claim quality-preserving training or practical equivalence to global backpropagation.
Transfer and Mechanism Checks
Additional widths, corpus transfer, and shared-readout diagnostics
The separately reported 1% joint criteria pass at widths 128 and 384.
The asynchronous evaluation does not satisfy the declared joint quality and speed criteria on TinyStories.
Zero-lag raw-gradient correspondence does not imply equivalence of implemented training trajectories under lag, separate optimizers, clipping, and parameter divergence.
Scope Conditions
Boundaries on interpretation
The experiments use one many-core CPU host. CPU thread scaling does not establish behavior for fused or IO-aware GPU kernels, and no GPU claim follows.
The setting is causal byte-level Transformers with vocabulary size 256 and short fixed training budgets; conclusions are bounded to this protocol.
The evidence supports a measured CPU quality–throughput–memory frontier, not a general replacement for backpropagation.
Cite This Paper
@article{lex2026boundedhorizon,
title={Bounded-Horizon Local Transformer Training on {CPUs}: Quality,
Throughput, and Memory},
author={Lex, Vikram},
year={2026},
month={jul},
journal={Research Square},
publisher={Research Square},
doi={10.21203/rs.3.rs-10490289/v1},
url={https://doi.org/10.21203/rs.3.rs-10490289/v1},
note={Preprint, version 1; manuscript under review at the Journal of
Artificial Intelligence Research}
}