OAT:只用成功轨迹做智能体失败归因,称比提示词裁判快数百到数千倍
NeurIPS 2026 论文介绍 OAT:将失败归因视为单类学习,仅用成功轨迹训练,推理时对失败轨迹逐步打漂移分;称约 100 条成功样本即可,速度较提示词基线快 200–5000 倍,F1 约高 20%。
长任务智能体出问题时,难点往往不是「失败了」,而是「哪一步开始偏了」。威斯康星大学等研究者在 NeurIPS 2026 论文《Tracing Agentic Failure from the Flow of Success》中,把失败归因(failure attribution)当成可规模化的诊断问题。
作者指出两条主流路径的成本:提示词管道让大模型通读整条轨迹逐步评判,又慢又贵;事后训练的归因模型又依赖逐步标注的失败轨迹,采集困难。他们提出的 OAT 把问题做成单类学习:只在成功轨迹上训练,用神经受控微分方程刻画「健康轨迹」在潜空间中的流动;推理时对失败轨迹逐步打「偏离成功动力学」的分数,偏离最大的步骤被标为可疑错误点,且不需要逐步失败标注。
帖内给出的结果是:仅用约 100 条成功轨迹训练时,OAT 比含 GPT-5 judge 在内的提示词基线快约 200–5000 倍,F1 约高 20%。合作方包括微软研究院。对平台与评测团队,这意味着可以用大量廉价成功日志做诊断,而不是先堆标注失败集。论文数字来自作者自报实验设置,迁移到你们的工具链与任务分布前仍需复现。
flowchart TD
S[成功轨迹] --> T[学习成功动力学 OAT]
F[失败轨迹] --> I[逐步漂移打分]
T --> I
I --> E[高漂移步骤 = 可疑失败点]
适用读者:做智能体评测、轨迹调试与可观测性的研究与工程团队。
Tracing Agentic Failure from the Flow of Success
When an LLM agent executes a long task, the hardest question is usually not whether it failed but where. Which step actually broke the run? 🧵
That’s the failure attribution problem, and it’s becoming central to debugging agentic systems. But current approaches have a real cost problem:
- Prompting-based pipelines, where an LLM reads the whole trajectory and judges each step, are slow and expensive to run.
- Post-trained attribution models need failure trajectories labeled at the step level, which are painful to collect and hard to scale.
We think a practical failure attribution model should be lightweight and shouldn’t need step-level labels on failures at all.
So in our #neurips2026 paper, “Tracing Agentic Failure from the Flow of Success,” we ask: can you find where an agent went wrong by learning only what going right looks like?
Meet OAT 🌾
The idea: treat failure attribution as one-class learning. We train only on successful trajectories, using neural controlled differential equations to model how successful runs evolve in latent space. Think of it as learning the “flow” of a healthy agent trajectory.
At inference, we take a failed trajectory and score every step by how far it drifts from those learned success dynamics. Steps that deviate most get flagged as the likely error steps. The algorithm doesn't rely on any step-level annotation.
What we found: ⚡ With just 100 successful trajectories for training, OAT runs 200-5000× faster than prompting-based baselines (including GPT-5 judge). 📈 It also outperforms them on accuracy: about +20% F1 score.
So you don’t have to trade cost for quality here. You can get both.
Why I’m excited about this: successful runs are cheap and plentiful, while carefully annotated failures are not. Building diagnostics on top of the data we already have feels like the right direction as agent systems grow longer and more complex.
Huge thanks to my amazing student @Samuel861025 for leading the effort, and collaborators from Microsoft Research @ShaleenDeep and Yiwen Zhu.
📄 Paper: https://t.co/tVwkEhyISH