Record · analyzed
Harness²: Recursive Agent Harnessing for an Open World
Edits the agent's harness for each task, then learns from those edits to update the harness that does the editing: up to +11.5 points on WorkBuddy-Bench.
1
Listed
Added to the list. How entries were checked when they were listed is in the journal.
2
Collected
| File | Source | SHA-256 |
|---|---|---|
| paper PDF, 2610.14224v1harness2-v1.pdf · 8.1 MB | arxiv.org/pdf/2610.14224v1 | 440be1f694f3b0fdfb8e612e9b2288ff0d8ab38641cfd1064fd77052ba360282 |
| code at commit 1cfe254e8d25harness2-repository.tar.gz · 7.1 MB | github.com/google-research/harness2/tree/1cfe254e8d25b15f37048c3730134686abe60335 | 2337f91b71a130e055d8dfb99ead2667a46087c29df939d4d4f0a3c357b5748f |
To check a file, download it from its source and compare its SHA-256 with the one above. Collected copies are kept in the editor's RSI system; manifest: design-life/literature/rsi-design/three-rsi-approaches-2026-10-28/sources.json.
3
Analyzed
Quoted verbatim from design-life/research/rsi-design/three-approaches-review-2026-10-28.md (2026-10-28): Harness²: learn how to improve the next harness. Parts about applying the paper to the editor's private work are left out.
From “Harness², Terminal-RSI and Personal AI” (2026-10-28)
I located two papers and a company technical disclosure with a related code branch. I did not find an academic paper evaluating 通衍灵智's complete Personal AI improvement loop in the bounded search. The findings below separate author/company reports, inspected code and this review's interpretations. No model calls, training, downloaded code execution or benchmark reruns were performed. The source manifest and archive retain versions, hashes, reading scope and access limits.
Harness²: learn how to improve the next harness
Mechanism. A frozen model executes each new task from a common base harness. An editor proposes two kinds of revisions: general execution improvements and domain-specific improvements. Each candidate is run on the same task under matched model/environment conditions. A composer inspects reference outputs, candidate outputs, trajectories and code differences, then combines, revises or rejects edits based on observed behavioral contrasts. For larger budgets, an agent selects an existing anonymized final deliverable using the task and sources. After the task, reflection can retain the adopted executable harness and an experience record describing applicability, lesson and evidence. The next task starts again from the base harness and can consult this library. Paper §§2.1–2.3
This adds a second improvement target to ordinary task repair: reusable guidance about improving execution. The paper's improvement harness is not the same object as the representation definition in our study. Its model weights, stage instructions and decision rule remain fixed; the library evolves. The paper does not demonstrate unconstrained revision of its entire improvement algorithm.
The editable harness has eight components: system prompt, guardrails, memory, tool guidance, subagents, skills, scripts and plugins. Parallel recursion explores more candidates; sequential recursion chains refinements. Their form and budget are specified in advance. This is a useful implementation example, not evidence that those roles or stages should become mandatory in our dynamically chosen execution.
Results and scope. WorkBuddy uses 40 Code, 35 Web and 25 Office test tasks. The average is a macro-average of domain scores, themselves based on fractions of checks passed. With GLM-5 under OpenCode, the reported average rises from 64.2 ± 1.4 to 75.7 ± 2.2 with one improvement step. With Gemini 3.5 Flash under OpenCode, it rises from 72.4 ± 0.9 to 77.5 ± 2.5. These are score points, not percentages of wholly completed tasks. Main results report three streams with shuffled task order. Table 1 and Appendix B
JobBench has 34 held-out tasks scored against weighted rubrics; LAB uses 100 held-out legal tasks and reports per-criterion pass rate. In the Gemini/OpenCode ablation, removing the improvement library lowers WorkBuddy from 77.5 to 74.4, and JobBench from 36.0 to 34.2. The paper also reports JobBench-library transfer gains of 0.6, 0.5 and 2.9 points on WorkBuddy Code, Web and Office. These support the usefulness of reusable improvement evidence in those settings; they do not establish general transfer or human acceptance. Table 3, Figure 7, Appendix B
A useful negative result: direct editing without reference executions, candidate probes or a library reduces Code 73.4 → 63.2 and Web 69.9 → 65.0. This challenges an assumption that a plausible revision is sufficient evidence of improvement. Actual resulting behavior matters. The default editor/selector is Claude Opus 4.8, while executors are Gemini or GLM; a separate same-model experiment is also supplied. Main gains should therefore be described using their actual model roles. Figures 7–8
Costs and reproducibility. Figure 6 counts execution and improvement tokens before submitting the deliverable. This is more complete than reading only the selected execution's steps. Token counts across models still do not establish monetary cost, human effort or a universal optimal budget. Authors explicitly note added compute and the fallibility of execution contrasts/model judgments. Figure 6 and limitations
The inspected release at 1cfe254 contains recursion, composition, selection, reflection and library code, rather than only selected final harnesses. The library stores complete adopted component trees with EXPERIENCE.md; an entry requires cue, lesson and evidence fields. This structural check does not independently validate the lesson. Library code also supports replacing/removing entries for resume handling; it is not an implementation of our append-only-active-rules requirement. Library, recursion, selection/reflection
One material caveat is unusually explicit in the release: its experiment guide reports a validation rollout downloading a public JobBench rubric. The release leaves network access unrestricted and instructs users to configure restrictions. This is evidence of a release-level evaluation-contamination risk, not proof that the paper's reported runs leaked. The guide also distinguishes the default full LAB stream from the paper's 100-task test set, and records downloaded JobBench revisions rather than fixing one at release. Reproduction needs the actual task partition, dataset revision, model roles and access conditions. Experiment guide
Lesson for us. Retain the execution change together with the conditions and evidence explaining where it helped, failed or remains uncertain. A selected output alone is weak evidence for a reusable improvement lesson. This supports design 1.2's traceability and comparison requirements and suggests a future explicit representation of lesson applicability. It does not establish that our data-dependency scheduler or missing-first policy is optimal.
4
Replicated
Not done yet. The review states: "No model calls, training, downloaded code execution or benchmark reruns were performed." It notes that reproduction "needs the actual task partition, dataset revision, model roles and access conditions."
5
Audited
Not done yet. Not attempted yet.
The editor's view
None recorded yet.