研究:AGENTS.md 过期时,编程智能体仍约九成听从文件而非代码
预印本在 40 个故意冲突的 Python 项目中测得智能体约 90.1% 听从过期指令文件;作者建议用户级规则改为「以现有代码约定为准并列出不一致」。
xu 与 wu 的 arXiv 预印本(10 月 22 日)做了一组对照实验:在 40 个 Python 项目里故意让 AGENTS.md(或同类指令文件)与仓库现有代码约定冲突,再观察编程智能体听谁的。结果是智能体在约 90.1% 的选择里跟随了过期指令文件;Claude Code 甚至常在最终回复里点出冲突,却仍服从文件。
作者解释是:指令文件作为「指示」进入上下文,代码多半作为工具输出出现,指示的优先级更高。他们验证过的缓解办法是在用户级规则里加一句:当文档与现有代码不一致时,以代码中的约定为准,并在最终回复列出每一处不一致。配套习惯则是:改约定的 PR 必须同步改 AGENTS.md;智能体列出冲突后,下一轮跑之前先修文件。对依赖项目级规则文件的团队,这是在提醒「文件也会腐」——只写不管,智能体会一本正经地执行旧规矩。
综合来看,读者可先核验官方渠道与发布时间,再结合自身业务边界与合规要求决定是否跟进试点;本文仅整理公开信息,不构成采购、投资或法律建议,关键数字与能力边界请以一手文档为准。
Study: Coding Agents Follow Stale AGENTS.md Over Code ~90% of the Time
when your AGENTS.md drifts out of date, does your coding agent follow the file or the code?
an arxiv preprint (xu & wu, oct 22) built 40 python projects where the instruction file contradicted the existing code, and measured: agents followed the stale file in 90.1% of their choices. claude code often even named the conflict in its final reply, then obeyed the file anyway.
their read: the file reaches the model as an instruction, the code arrives as tool output, and instructions rank higher.
the fix they tested is one sentence in your user-level file:
"when a repository's documentation and its existing code disagree, follow the conventions used in the existing code, and in your final reply point out each inconsistency you found between the documentation and the code."
two habits on top: 1. treat AGENTS.md like code. the PR that changes a convention also updates the file 2. when the agent lists inconsistencies in its final reply, fix the file before the next run