Amazon 开源 ATTUNE:用编码智能体在 GPU 小时预算内自动调 LLM 训练吞吐
amazon-science/attune 介绍 ATTUNE:在给定 GPU 小时预算内,由编码智能体按技能包诊断并一次改一个旋钮,无人值守调高 LLM 训练配方吞吐;作者称在自有后训练配方上平均约 1.7×,并在多项公开基准上超过既有自动调参。
Amazon Science 开源 ATTUNE(GitHub:amazon-science/attune,论文 arXiv:2610.17528)。问题很工程化:LLM 训练吞吐取决于并行切分、显存与算力折中、内核与通信重叠等「配方」;配方随模型、加速器与框架版本而变。传统自动调参往往只在固定旋钮空间里搜索,且只检查「能跑」,可能漏掉空间外的杠杆,甚至在静默改变训练语义的配置上交差。
ATTUNE 把调参做成编码智能体:在 GPU 小时预算内无人值守地改动任一可运行配方。智能体带一套搜索规则、脚本与历史战役经验的 skill,每次诊断后只动一个杠杆。作者报告:在亚马逊生产后训练配方、H100/H200/B200 集群上平均吞吐提升约 1.7×;在 NVIDIA AutoModel 框架上,相对四份已发布的 NVIDIA 基准配方提升约 6%–17%;在开发期未见过的模型上,对「为别的硬件写的配方」提升可观(文中举例 MiniMax-M2 最高约 2.6×),对「厂商已按同硬件调过」的配方提升较小。同预算下,作者称其优于贝叶斯优化、随机搜索与近期 LLM-agent 调参器 OptiCo;去掉 ATTUNE skill 的同一编码智能体收益更低且约一半启动失败。
这些数字均来自论文/项目方自述,复现依赖具体集群与配方。对训练平台与 ML Infra 团队,意义在于把「会改分布式训练脚本的编码智能体」做成有预算约束的调参工人,而不是一次性聊天建议。仓库:https://github.com/amazon-science/attune ;论文:https://arxiv.org/abs/2610.17528 ;来源帖:https://x.com/kalyan_kpl/status/2114214882178703795 。
ATTUNE: Agentic Throughput Tuning for LLM Training
ATTUNE: Agentic Throughput Tuning for LLM Training
The throughput of an LLM training job depends on its recipe: how the model is split across GPUs, how memory is traded for compute, and which kernels and communication overlaps it uses.
The best recipe changes with every model, accelerator and framework version. Automated tuners search a fixed space of knobs and check only that a job runs, so they miss levers outside that space and can ship a configuration that silently changes training.
ATTUNE is a coding agent that tunes any working recipe unattended within a GPU-hour budget.
Guided by a skill of search rules, scripts and lessons from past tuning campaigns, it diagnoses each run and changes one lever at a time.
ATTUNE tunes Amazon's production post-training recipes on H100, H200 and B200 clusters, raising throughput by 1.7x on average.
It beats the published throughput of four of NVIDIA's own benchmark recipes on NVIDIA's AutoModel framework by 6-17%.
On models held out from its development, it improves the strongest public recipes most where they were written for another setting, by up to 2.6x on MiniMax-M2, and least where the vendor tuned them for the same hardware.
Given the same GPU-hour budget, it outperforms existing automated tuners, including Bayesian optimization, random search and OptiCo, a recent LLM-agent tuner.
Its gain on GLM-4.5 is more than four times the best alternative's. The same coding agent without ATTUNE's skill gains less and fails half its launches.
Attune Repo - https://t.co/JAsCaRh3Mp
Attune Paper - https://t.co/YImMCn8ZMe