GLM 5 turbo
A lower-table result with a few useful bright spots: 44/151 tasks solved at least once, 19/151 solved in all three attempts, with the clearest wins around large Python/Django application repairs plus localized Go security-scanner changes. 这是一个排名靠后但仍有局部亮点的结果:151 题中至少一次解出 44 题,三次都解出 19 题;强项主要落在大型 Python/Django 应用修复以及边界相对清楚的 Go 漏洞扫描器改动。
How to read this result可以这样读
- GLM 5 turbo ranks #16 with a 28.95 Final Score. The headline is 44 reached tasks, but the stability number is 19 pass-in-all-three tasks.GLM 5 turbo 排名 #16,Final Score 为 28.95。表面信号是 44 道题至少成功一次,稳定性信号是 19 道题三次都成功。
- The strongest evidence clusters around large Python/Django application repairs plus localized Go security-scanner changes.最强证据集中在大型 Python/Django 应用修复以及边界相对清楚的 Go 漏洞扫描器改动。
- The failure shape is mostly Go product plumbing across configuration, storage, and service APIs plus automation and configuration-management work.失败形态主要是横跨配置、存储和服务 API 的 Go 产品工程以及自动化和配置管理类改动。
- The OpenCode run is more sensitive to the underlying model family: the same harness can look sharp or brittle depending on where the model puts its search budget.OpenCode 这组更能体现底层模型家族差异:同一套 harness 下,模型如何分配搜索预算会直接决定它显得锋利还是脆弱。
GLM 5 turbo is best read through the gap between reach and repeatability. It reaches 44/151 tasks at least once, but 19/151 tasks survive all three attempts. That gap is the personality of the row: the model can find solutions across a fairly wide surface, but the dependable core is narrower than the headline Pass@3 number.
In leaderboard terms, rank #16 and a 28.95 Final Score put it in direct comparison with nearby models, but the more useful question is where the wins come from. In this run the strongest signal is large Python/Django application repairs plus localized Go security-scanner changes; the weak side is Go product plumbing across configuration, storage, and service APIs plus automation and configuration-management work. The OpenCode run is more sensitive to the underlying model family: the same harness can look sharp or brittle depending on where the model puts its search budget.
The suite chart is the fastest way to read the model. High bars mean the agent repeatedly found the right subsystem and produced patches the verifier accepted at least once. Low bars are not just misses; they are hints about the task shape that made the model overfit a local edit, stop before the second-order consumer, or fail to keep a multi-package change coherent.
The case notes above keep the article grounded in individual SWE-Bench-Pro instances. A stable 3/3 solve means the task is inside the model’s dependable operating region. A 1/3 solve means it can reach the idea, but the path is retry-sensitive. A 0/3 miss is more diagnostic: it marks a task shape where this model-agent pairing did not find a verifier-backed patch in three independent attempts.
The verifier audit block below is included because this row has re-verification data.
The available audit keeps 99 of 99 initial solved attempts. Read this as a robustness check, especially when the audit sample is smaller than 453 attempts.当前可用复核保留了 99 次初始成功中的 99 次。这更适合作为稳健性检查,特别是在复核样本小于 453 次尝试时。
For practical use, I would treat GLM 5 turbo as strongest when the task resembles the high-performing suites and weaker when it resembles the low-performing suites. The raw attempt score is 99/453; that is enough signal to compare it with neighboring rows, but not enough to assume the same behavior on every repository family.
Supporting suite table
| Suite | Repo | Solved | Pass^3 | Rate |
|---|---|---|---|---|
release-zh-012-future-architect-vuls |
future-architect/vuls | 3/4 | 2 | 75.0% |
release-zh-013-internetarchive-openlibrary |
internetarchive/openlibrary | 6/10 | 4 | 60.0% |
release-zh-014-internetarchive-openlibrary |
internetarchive/openlibrary | 6/10 | 3 | 60.0% |
release-zh-015-internetarchive-openlibrary |
internetarchive/openlibrary | 5/10 | 1 | 50.0% |
release-zh-007-flipt-io-flipt |
flipt-io/flipt | 4/10 | 2 | 40.0% |
release-zh-010-future-architect-vuls |
future-architect/vuls | 4/10 | 0 | 40.0% |
release-zh-018-qutebrowser-qutebrowser |
qutebrowser/qutebrowser | 0/9 | 0 | 0.0% |
release-zh-017-navidrome-navidrome |
navidrome/navidrome | 0/5 | 0 | 0.0% |
release-zh-005-flipt-io-flipt |
flipt-io/flipt | 1/10 | 0 | 10.0% |
release-zh-006-flipt-io-flipt |
flipt-io/flipt | 1/10 | 0 | 10.0% |
读 GLM 5 turbo,最有用的是看“覆盖能力”和“重复稳定性”的差距。它在 151 题中至少一次解出 44 题,但三次尝试都解出的只有 19 题。这个差距就是这一行的性格:模型能在相当宽的任务面上摸到解法,但真正可靠的核心比 Pass@3 的表面数字更窄。
从排行榜数字看,排名 #16、Final Score 28.95 让它可以和附近模型直接比较;但更重要的问题是胜利来自哪里。这次运行最强的信号在大型 Python/Django 应用修复以及边界相对清楚的 Go 漏洞扫描器改动,弱侧则主要是横跨配置、存储和服务 API 的 Go 产品工程以及自动化和配置管理类改动。OpenCode 这组更能体现底层模型家族差异:同一套 harness 下,模型如何分配搜索预算会直接决定它显得锋利还是脆弱。
suite 图是最快的读法。高柱子说明 agent 能反复找到正确子系统,并至少一次产出 verifier 接受的补丁。低柱子不只是失败列表,它们提示了让模型过拟合局部编辑、漏掉第二层消费者,或无法维持跨包改动一致性的任务形状。
上面的案例把文章拉回到具体 SWE-Bench-Pro instance。3/3 稳定通过说明任务落在模型可靠区;1/3 说明它能摸到思路,但路径依赖重试;0/3 则更有诊断价值,表示这个模型-agent 组合三次独立尝试都没有找到 verifier-backed patch。
下面保留 verifier audit 模块,因为这一行有复核数据。
The available audit keeps 99 of 99 initial solved attempts. Read this as a robustness check, especially when the audit sample is smaller than 453 attempts.当前可用复核保留了 99 次初始成功中的 99 次。这更适合作为稳健性检查,特别是在复核样本小于 453 次尝试时。
实际使用时,我会把 GLM 5 turbo 用在更接近高分 suite 的任务上;如果任务形态接近低分 suite,就要更谨慎。它的单次尝试成功数是 99/453,足够用来和邻近模型比较,但不足以推断它在所有 repository family 上都会保持同样表现。
支撑这个判断的 suite 表
| Suite | Repo | 解出 | Pass^3 | 通过率 |
|---|---|---|---|---|
release-zh-012-future-architect-vuls |
future-architect/vuls | 3/4 | 2 | 75.0% |
release-zh-013-internetarchive-openlibrary |
internetarchive/openlibrary | 6/10 | 4 | 60.0% |
release-zh-014-internetarchive-openlibrary |
internetarchive/openlibrary | 6/10 | 3 | 60.0% |
release-zh-015-internetarchive-openlibrary |
internetarchive/openlibrary | 5/10 | 1 | 50.0% |
release-zh-007-flipt-io-flipt |
flipt-io/flipt | 4/10 | 2 | 40.0% |
release-zh-010-future-architect-vuls |
future-architect/vuls | 4/10 | 0 | 40.0% |
release-zh-018-qutebrowser-qutebrowser |
qutebrowser/qutebrowser | 0/9 | 0 | 0.0% |
release-zh-017-navidrome-navidrome |
navidrome/navidrome | 0/5 | 0 | 0.0% |
release-zh-005-flipt-io-flipt |
flipt-io/flipt | 1/10 | 0 | 10.0% |
release-zh-006-flipt-io-flipt |
flipt-io/flipt | 1/10 | 0 | 10.0% |