Terminal-RSI: Recursive Self-Improving Terminal Agents via Recursive Self-Rewrite and Policy-Refine

From “Harness², Terminal-RSI and Personal AI” (2026-10-28)

I located two papers and a company technical disclosure with a related code branch. I did not find an academic paper evaluating 通衍灵智's complete Personal AI improvement loop in the bounded search. The findings below separate author/company reports, inspected code and this review's interpretations. No model calls, training, downloaded code execution or benchmark reruns were performed. The source manifest and archive retain versions, hashes, reading scope and access limits.

Terminal-RSI: train from verified behavior and preserve strict success

Mechanism. This approach changes model parameters. Recursive Self-Rewrite discovers successful trajectories under alternative harnesses, converts them into runbooks, screens them, executes fresh attempts under a target harness, and trains the actor on accepted replay. Critic weights are carried through this supervised transition. Recursive Policy-Refine then calibrates inherited values and jointly trains actor and critic with PPO. The retained sequence is Base → RSR-1 → RPR-1 → RSR-2 → RPR-2. Paper §§2.1–2.4

Three technical distinctions matter:

• Credit follows decisions. Turn-Boundary Skip-Observation GAE applies decay across interaction turns, with observations conditioning values rather than adding their own recursion steps. More verbose logs should not automatically make an earlier consequential action more distant. Fixed-input chunk continuation and semantic summarization are different operations; a summary can change what the model knows.
• Learning rewards and completion are different. Family Dense Reward scales passed-check counts by task-family divisors. Success-Prioritized Reward adds strict-success credit and bounds the partial-progress contribution. Its formula puts every individual failure below every individual success, but the authors explicitly note that maximizing this reward is not equivalent to maximizing success probability.
• A critic predicts a training return. Preserving and recalibrating its weights does not give it the authority to define task satisfaction. Rewards here are calculated at rollout completion; “dense” refers to progress resolution, not necessarily feedback after every action.

Results and scope. The study uses one Qwen3.8-27B backbone and a 3,000-task training pool. Retained-checkpoint results include:

Metric	Base	Final retained checkpoint
Terminal-Bench 4.0 strict success	0.00%	19.84%
Terminal-Bench 2.1 strict success	51.70%	86.52%
LHTB mean process reward	0.21	0.37
LHTB mean strictly completed tasks	0/46	5/46
SWR100 strict success	3.00%	6.00%

The reported terminal-suite results are Mean@3, the arithmetic mean of three evaluation runs, rather than pass@3 or a union of solved tasks. LHTB completion is the strict 1.00 criterion, not a near-completion threshold. SWR100 is disjoint from the training pool. Its final three retained stages all score 6.00%, so the second cycle does not show a further gain there. Table 2, §3.3 and Appendix A

The most relevant evaluation finding. In 4,352 audited trajectories, 48 failures passed every executed scenario check but failed the terminal gate. Under the previous reward, failures received 50.09% of reward mass. Rescoring the same trajectories under the full success-prioritized reward reduced their share to 28.72%. This is a fixed-data reward-allocation result; it does not isolate an online training gain. §§2.2 and 3.4

The final gate can also omit something essential. In a Rust–C example, a final-model trial passes output tests while using C only for a linkage smoke test and delegating expression evaluation to Python. The authors say this establishes tested output equivalence, rather than a native Rust–C execution path. Thus even strict success means satisfying the enforced verifier, whose coverage must be checked against the task's actual requirements. §3.5 and limitations

Limitations and code. The authors acknowledge that actor learning rates, reward functions and initialization change together. Equal-compute alternation, isolated credit-estimator and critic controls, and independent training seeds remain necessary. Three inference evaluations do not resolve training variance. Their retained stage sequence does not validate a prospective checkpoint-selection rule; supplementary checkpoint/log provenance also has unresolved mappings. These are author-stated limits, separate from the observed suite-level gains. Limitations and Appendices D–G

The repository at 8f1e700 is publicly accessible at this review date. It includes reinforcement-learning recipes, reward functions, TB-GAE code and tests. Some older search results described unavailable code; that is not the current state I observed. The README explicitly excludes the RSR data-generation pipeline. Current source/configuration can be inspected, but does not independently establish which estimator/configuration produced a historical reported checkpoint. Release README, reward implementation, TB-GAE

Lesson for us. Preserve partial progress, whole-task verifier success and task-owner acceptance as different judgments. A changed training reward is an execution-learning intervention when the external task requirements remain fixed; raw reward values from different definitions are not a common quality scale. A claim that a particular component caused improvement requires a narrower comparison than the reported bundled training sequence. Our append-only rules still need faithful coverage; keeping a rule's text cannot guarantee that a checker enforces its meaning.