SWE-bench - Mitzael Serna AI Thoughts

Two old leather-and-wood horse collars hanging from wires under timber roof beams above a brick floor

Swapping a coding agent’s harness mostly changes the bill, not the score

adminOctober 25, 20260

From Mexico: a new study ran the same models through Claude Code, OpenCode and mini-SWE-agent. Pass rates barely moved, but cost per task changed by up to 3x. What the harness actually buys, and what I’d measure first.

► Necessary Cookies Always Active

Necessary cookies enable essential site features like secure log-ins and consent preference adjustments. They do not store personal data.

► Functional Cookies Remark

Functional cookies support features like content sharing on social media, collecting feedback, and enabling third-party tools.

► Analytical Cookies Remark

Analytical cookies track visitor interactions, providing insights on metrics like visitor count, bounce rate, and traffic sources.

► Advertisement Cookies Remark

Advertisement cookies deliver personalized ads based on your previous visits and analyze the effectiveness of ad campaigns.