Regularized Emphatic TD (RETD) proposed to address constant-stepsize instability in ETD
The arXiv preprint (arXiv:2609.19170v1) constructs an ergodic two-state counterexample showing that emphatic TD (ETD) can have contracting mean dynamics while sampled trajectories exhibit positive top Lyapunov exponents, and introduces Regularized Emphatic TD (RETD): a normalized post-shock repair that stores the emphatic signal in a leaky scalar and releases a delayed correction. The paper proves almost-sure convergence for harmonic diminishing stepsizes, gives a conditional constant-stepsize moment-contraction bound, shows RETD recovers ETD fixed points under certain readouts, and validates results with large-scale experiments including a Baird point and 10,000-run trials.