Artificial Analysis 编码智能体指数 v2.0:Opus 5.5 加 Claude Code 以 61.7 领跑
Coding Agent Index 升至 v2.0,改用 FrontierCode v1.1 并收紧仓库问答评分;Claude Code 搭配 Claude Opus 5.5(max)以 61.7 分居首,并公布若干成本前沿组合。
Artificial Analysis 发布 编码智能体指数(Coding Agent Index)v2.0,把「模型加智能体外壳」放在同一套指数里比较。当前覆盖软件工程、终端使用与仓库问答等共 二百九十 项任务,三类基准等权合成总分。
相对上一版的变更
相对 v1.5,指数改用 Cognition 的 FrontierCode v1.1:私有测试集、约一百个真实开源仓库改动,按维护者是否会合并等规则打分,用以替换 DeepSWE。同时修订仓库问答基准为 SWE-Atlas-QnA-AA:改写部分量规条款,裁判改为 GPT-6 Luna(xhigh),每条准则多票汇总。
主要分数(官方口径)
- Claude Code 搭配 Claude Opus 5.5(max) 以 61.7 分领跑;Claude Code 搭配 Sonnet 5.5(max)约 59.3 分;Muse Code 搭配 Muse Wonder Full(max)约 58.9 分。
- 在 FrontierCode 上,Opus 5.5 约 百分之五十六,高于若干 Codex 组合。
- 成本前沿示例:Codex 搭配 DeepSeek V4.1 Flash 约每任务 47.3 分、0.39 美元;Claude Code 搭配 Opus 5.5 约 61.7 分、11.42 美元(按 token 接口成本估算,不是订阅价)。
指数会持续增量更新。阅读时务必区分:这是「外壳加模型」的组合分,不能直接当成裸模型排行榜;不同外壳的工具策略、默认推理强度与计费方式都会改变名次。本轮窗口内官方账号发布图表与说明,脚本曾打出与财经相关的关键词提示,经阅读确认为评测发布后采用。
Coding Agent Index v2.0 announcement
Announcing Artificial Analysis Coding Agent Index v2.0: FrontierCode v1.1 from Cognition replaces DeepSWE, SWE-Atlas-QnA grading is revised, and Claude Opus 5.5 in Claude Code leads at 61.7
Today we are launching v2.0 of the Coding Agent Index. The Index evaluates models together with their agent harnesses, now across 290 tasks covering software engineering, terminal use, and repository Q&A, with the three benchmarks weighted equally.
Changelog (Index v1.5 → Index v2.0):
→ FrontierCode v1.1 replaces DeepSWE v1.1. FrontierCode is a benchmark from @cognition with a private test set: 100 software engineering tasks on real open-source repositories, graded on whether a maintainer would merge the change. An attempt that fails a blocking check scores zero, and one that passes them all scores between 70% and 100% depending on the non-blocking checks it meets. We also scan each trajectory with a modified version of Cognition's rule-based scanner, and any attempt flagged for accessing solution material scores zero.
→ Revised SWE-Atlas-QnA grading, now SWE-Atlas-QnA-AA. Scale AI's 124 repository Q&A tasks are unchanged. We reworded 127 of the 1,306 rubric criteria for more consistent verdicts and moved to GPT-6 Luna (xhigh) as the judge, with five votes per criterion. The dataset and judge configuration are published on Hugging Face.
Key results:
➤ Claude Opus 5.5 leads the Index: Claude Code with Claude Opus 5.5 (max) scores 61.7, followed by Claude Code with Claude Sonnet 5.5 (max) at 59.3, Muse Code with Muse Wonder Full (max) at 58.9 and Claude Code with Claude Fable 5.1 (max with fallback) at 57.6.
➤ FrontierCode v1.1 is where Opus 5.5 pulls ahead: Opus 5.5 scores 56.0% on FrontierCode, ahead of Codex with DeepSeek V4.1 Flash (max) at 52.6%, Codex with GPT-6.1 Sol (max) at 49.4% and Codex with GPT-6 Astra (max) at 49.3%. Below the top, FrontierCode scores are tightly packed: 15 of the 24 configurations on the Index score between 43% and 49%.
➤ SWE-Atlas-QnA-AA grading is stricter and more consistent: across the 21 configurations evaluated on both Index versions, the mean score falls 1.7 points, from 58.8% to 57.1%, with 14 of 21 moving down. Individual configurations move by up to 7 points: Muse Code with Muse Spark Open (xhigh) falls 7.0, Kimi K3 in Kimi Code CLI and Codex with GPT-6 Luna (max) 6.2, and Muse Code with Muse Wonder (max) 5.6, while Codex with GPT-6.1 Sol (max) gains 2.1.
➤ Four configurations sit on the cost frontier: Codex with DeepSeek V4.1 Flash (max) scores 47.3 at $0.39 per task, Codex with GPT-6.1 Sol (max) 54.1 at $1.50, Codex with GPT-6 Astra (max) 54.7 at $7.04 and Claude Code with Claude Opus 5.5 (max) 61.7 at $11.42. Costs are pay-per-token API costs, not subscription prices.
We will keep updating the Coding Agent Index incrementally, with more evaluations and agents to come.