GLM 4.7
A lower-table result with a few useful bright spots: 36/151 tasks solved at least once, 13/151 solved in all three attempts, with the clearest wins around large Python/Django application repairs plus automation and configuration-management work. 这是一个排名靠后但仍有局部亮点的结果:151 题中至少一次解出 36 题,三次都解出 13 题;强项主要落在大型 Python/Django 应用修复以及自动化和配置管理类改动。
How to read this result可以这样读
- GLM 4.7 is best read as wide but retry-sensitive: rank #27, 36 reached tasks, 13 stable solves.GLM 4.7 更适合读成覆盖不窄但依赖重试:排名 #27,触达 36 题,稳定解出 13 题。
- Best suite signal: Open Library · release 013 at 6/10 (60.0%).最强 suite 信号:Open Library · release 013,6/10(60.0%)。
- Weakest visible area: Flipt · release 005 at 0/10 (0.0%).最弱可见区域:Flipt feature flag 服务 · release 005,0/10(0.0%)。
- Because the agent shell is OpenCode, the result mostly exposes the underlying model's planning habits rather than a heavily opinionated workflow.因为 agent shell 是 OpenCode,这个结果更直接暴露底层模型的规划习惯,而不是强工作流包装后的表现。
GLM 4.7 is broad but volatile. It can touch 36/151 tasks, yet only 13 become 3/3 solves, so much of its value comes from retrying the same benchmark surface.
The closest family reference is GLM 5.2 at rank #1. Compared with that row, this one is 13.56 points behind, with 21 fewer reached tasks and 23 fewer stable solves.
The volume win is Open Library · release 013 at 6/10 (60.0%), while the cleanest pass-rate spike is vuls · release 012 at 3/4 (75.0%). The warning label is Flipt · release 005 at 0/10 (0.0%), so the contrast is not generic strength versus weakness; it is large Python/Django application repairs holding together better than Go product plumbing across configuration, storage, and service APIs on this run. Because the agent shell is OpenCode, the result mostly exposes the underlying model’s planning habits rather than a heavily opinionated workflow.
The important visual cue is the gap between high-reach suites and low Pass^3 counts. This model can often locate the neighborhood of the fix, but many patches do not survive three independent runs.
The useful contrast is between Scan results miss Package URL (PURL) information in library output (future-architect/vuls · solved 3/3) and Deprecation of UnsafeProxy causes inconsistency in variable wrapping (ansible/ansible · solved 2/3). The model reaches both kinds of problems, but only one becomes dependable.
The audit trims 11 solved attempts from GLM 4.7 but still keeps 85% of the solved set, so the suite shape remains useful even where individual wins are debatable.
The available audit keeps 62 of 73 initial solved attempts. Read this as a robustness check, especially when the audit sample is smaller than 453 attempts.当前可用复核保留了 73 次初始成功中的 62 次。这更适合作为稳健性检查,特别是在复核样本小于 453 次尝试时。
Use it when breadth matters more than deterministic replay. It can find openings around Open Library · release 013 at 6/10 (60.0%), but the 23-task reach gap says a second or third run may tell a different story. The 73/453 attempt score is best read as exploration bandwidth: 36 tasks are reachable, but many need retry luck.
Supporting suite table
| Suite | Repo | Solved | Pass^3 | Rate |
|---|---|---|---|---|
release-zh-012-future-architect-vuls |
future-architect/vuls | 3/4 | 2 | 75.0% |
release-zh-013-internetarchive-openlibrary |
internetarchive/openlibrary | 6/10 | 1 | 60.0% |
release-zh-002-ansible-ansible |
ansible/ansible | 4/10 | 1 | 40.0% |
release-zh-014-internetarchive-openlibrary |
internetarchive/openlibrary | 4/10 | 1 | 40.0% |
release-zh-004-ansible-ansible |
ansible/ansible | 1/3 | 1 | 33.3% |
release-zh-003-ansible-ansible |
ansible/ansible | 3/10 | 1 | 30.0% |
release-zh-005-flipt-io-flipt |
flipt-io/flipt | 0/10 | 0 | 0.0% |
release-zh-018-qutebrowser-qutebrowser |
qutebrowser/qutebrowser | 0/9 | 0 | 0.0% |
release-zh-016-internetarchive-openlibrary |
internetarchive/openlibrary | 0/5 | 0 | 0.0% |
release-zh-017-navidrome-navidrome |
navidrome/navidrome | 0/5 | 0 | 0.0% |
GLM 4.7 的特点是覆盖不窄但波动较大。它能至少一次摸到 36/151 题,但只有 13 题能做到 3/3,因此很大一部分价值来自重试。
最接近的同系参照是排名 #1 的 GLM 5.2。和它相比,这一行最终分低 13.56 分,触达题少 21 个,稳定题少 23 个。
从数量看,主要胜利来自Open Library · release 013,6/10(60.0%);从通过率看,最干净的高点是vuls 漏洞扫描器 · release 012,3/4(75.0%)。需要警惕的是Flipt feature flag 服务 · release 005,0/10(0.0%),所以这里不是泛泛地说强弱项,而是大型 Python/Django 应用修复在这次运行中比横跨配置、存储和服务 API 的 Go 产品工程更能闭环。因为 agent shell 是 OpenCode,这个结果更直接暴露底层模型的规划习惯,而不是强工作流包装后的表现。
这张图最重要的信号,是高触达 suite 和较低 Pass^3 之间的落差。模型经常能找到修复附近的位置,但很多补丁不能在三次独立运行中稳定复现。
最有用的对比是 Library 输出中的扫描结果缺少 Package URL (PURL) 信息(future-architect/vuls · 3 次中成功 3 次)和 UnsafeProxy 的弃用导致变量 unsafe 包装不一致(ansible/ansible · 3 次中成功 2 次):模型都能触达,但只有前者变成可靠结果。
复核从 GLM 4.7 中剔除了 11 次成功,但仍保留 85% 的成功集合,因此即便个别胜利有争议,suite 形状仍然有参考价值。
The available audit keeps 62 of 73 initial solved attempts. Read this as a robustness check, especially when the audit sample is smaller than 453 attempts.当前可用复核保留了 73 次初始成功中的 62 次。这更适合作为稳健性检查,特别是在复核样本小于 453 次尝试时。
当你更看重覆盖面而不是确定复现时,它更合适。它能在Open Library · release 013,6/10(60.0%)附近找到入口,但 23 题的覆盖-稳定差说明第二、第三次运行可能给出不同结果。73/453 的单次尝试成功数更像探索带宽:36 道题能触达,但很多仍需要重试运气。
支撑这个判断的 suite 表
| Suite | Repo | 解出 | Pass^3 | 通过率 |
|---|---|---|---|---|
release-zh-012-future-architect-vuls |
future-architect/vuls | 3/4 | 2 | 75.0% |
release-zh-013-internetarchive-openlibrary |
internetarchive/openlibrary | 6/10 | 1 | 60.0% |
release-zh-002-ansible-ansible |
ansible/ansible | 4/10 | 1 | 40.0% |
release-zh-014-internetarchive-openlibrary |
internetarchive/openlibrary | 4/10 | 1 | 40.0% |
release-zh-004-ansible-ansible |
ansible/ansible | 1/3 | 1 | 33.3% |
release-zh-003-ansible-ansible |
ansible/ansible | 3/10 | 1 | 30.0% |
release-zh-005-flipt-io-flipt |
flipt-io/flipt | 0/10 | 0 | 0.0% |
release-zh-018-qutebrowser-qutebrowser |
qutebrowser/qutebrowser | 0/9 | 0 | 0.0% |
release-zh-016-internetarchive-openlibrary |
internetarchive/openlibrary | 0/5 | 0 | 0.0% |
release-zh-017-navidrome-navidrome |
navidrome/navidrome | 0/5 | 0 | 0.0% |