VQV Signal
According to a post on X by @xiangkaizeng, a new benchmark called Multi-step SWE includes 69 software tasks, each with 5 to 15 dependent multi-hour steps on one codebase. Early design mistakes can carry forward through later steps. Across 9 frontier models, the best solve rate was 18.5%. The post says Klavis AI promoted the benchmark as part of its work on RL environments for agents.
According to a post on X by @xiangkaizeng, a new benchmark called Multi-step SWE includes 69 software tasks, each with 5 to 15 dependent multi-hour steps on one codebase. Early design mistakes can carry forward through later steps. Across 9 frontier models, the best solve rate was 18.5%. The post says Klavis AI promoted the benchmark as part of its work on RL environments for agents.
AI-assisted summary based on the listed source.
Multi-step SWE has 69 software tasks, each with 5 to 15 dependent multi-hour steps on one codebase, so early design mistakes carry forward. Across 9 frontier models, the best solve rate was 18.5%. Klavis AI promoted it as part of its work on RL environments for agents. Why it matters: It shows today's coding...
Builders and teams using coding agents for long projects should note that the benchmark suggests these tools still struggle to maintain their own code over extended work. Anyone evaluating agents for multi-hour, interdependent software tasks may want to look for results from independent testing before relying on them. The source is a social post, so the details should be checked against the benchmark itself.
VQV organizes public signals from inspectable sources. It does not independently verify the underlying report.
Signal Strength reflects source quality, relevance, freshness and evidence. Public Interest helps organize discovery; it is not proof of truth.
VQV surfaced this signal because it is recent, relevant to AI Coding Tools, connected to Scobleizer News Pulse (X).
No login, cookies, social SDKs, or automatic posting.
