HarnessAdapt-RL:共训智能体策略与工具脚手架,多语 SWE-bench 提升 12.6 个百分点
论文 HarnessAdapt-RL(arXiv:2610.28069)提出在同一次训练中共训策略与脚手架:在 9B 规模相对 GRPO 基线,SWE-bench Multilingual pass@1 提升 12.6 个百分点;部署时若去掉共训工具与钩子,多语成绩最多回落 7.7 个百分点。
推文介绍预印本 HarnessAdapt-RL(arXiv:2610.28069):常规智能体强化学习常**冻结脚手架(harness)**只训策略,该工作则在同一次训练里共训两者。一个冻结教师为策略的缺陷行为写「机制级技能」(解释何时、为何采取某类修复,而不是直接给答案);另一个教师演化工具与钩子,仅当通过率或可训练率上升且另一方不掉时才保留改动。
作者给出的 9B、pass@1、相对 GRPO 基线数字为:Verified +9.8 个百分点,Multilingual +12.6 个百分点,Pro +6.4 个百分点;并称即使用 OpenHands 手工设计脚手架,共训结果仍更好。部署侧结论是:上线的是模型–脚手架成对产物——技能可以卸,工具/钩子不能随便拆,否则 Multilingual 最多掉 7.7 个百分点。推文也标明限制:单篇未审稿、自报结果、9B 规模,且与某一策略共训的脚手架未必迁移到下一策略。
对编码智能体研发,这把「环境/脚手架」从部署细节抬成训练决策。引用时应保留作者自报与未审稿限定,不宜写成已广泛验证的生产结论。
flowchart LR
A[策略 RL] --> C[共训运行]
B[工具/钩子演化] --> C
C --> D[模型-脚手架成对部署]
D --> E[去掉脚手架则成绩回落]
若脚手架本身可学习,线上服务就不能再假设「换模型、留原工具链」无成本。工程上更稳妥的是把模型版本、工具清单与钩子配置钉在同一发布单元,并在回归集上同时测卸工具与换模型两种破坏。论文仍是预印本自报结果,引入前应用自有仓库复现关键数字,再决定是否把共训脚手架纳入发布产物。
HarnessAdapt-RL co-trains policy and harness
+12.6 percentage points on SWE-bench Multilingual — from training the harness, not the model.
The standard agentic-RL recipe freezes the harness and RLs only the policy. HarnessAdapt-RL (arXiv:2610.28069) co-trains both in a single run: one frozen teacher writes mechanism-level skills — the "when and why" behind repair strategies, not the answers — for the policy's flawed behaviors; a second teacher evolves the tools and hooks themselves, keeping each change only if pass rate or trainable rate rises without the other falling.
9B model, pass@1, all over a GRPO baseline: +9.8pp Verified, +12.6pp Multilingual, +6.4pp Pro. It also beats the same GRPO run inside OpenHands' hand-designed harness — the evolved tools are doing real work, not decoration.
The deployment finding is the one that changes how you ship: you deploy a model–harness pair. Strip the evolved tools and hooks at serving time and Multilingual drops up to 7.7pp. The skills can come off; the harness can't. Your scaffold has become load-bearing.
Asterisk: one paper, unreviewed, self-reported, 9B scale — and the lock-in cuts both ways. A harness co-trained with one policy may not transfer to the next, which is the generalization question this whole line still owes an answer to. But the direction is unmistakable: "agent" vs "environment" was always a bookkeeping convenience. Now it's a training decision.