AI
Better Models: Worse Tools
Developer Armin Ronacher reports that newer Anthropic Claude models (Opus 4.8 and Sonnet 5) sometimes call Pi’s edit tool with extra, invented fields
Key takeaways
- Newer Claude models (Opus 4.8 and Sonnet 5) emit malformed tool calls with invented fields in the edits[] array.
- Older Claude models do not exhibit this schema-compliance regression.
- The regression may stem from Reinforcement Learning training to improve use of Claude Code’s built-in edit tools.
- Third-party coding harnesses like Pi may face degraded compatibility due to this training.
Developer Armin Ronacher reports that newer Anthropic Claude models (Opus 4.8 and Sonnet 5) sometimes call Pi’s edit tool with extra, invented fields in the nested edits[] array, causing schema mismatches and tool-call rejections. The issue is absent in older Claude models. Armin theorizes this regression stems from Reinforcement Learning training to improve use of Claude Code’s built-in edit tools, which may degrade compatibility with third-party coding harnesses like Pi.
In their words
“The short version is that newer Claude models sometimes call Pi’s edit tool with extra, invented fields in the nested edits[] array.”
“This has the unfortunate effect that other coding harnesses, such as Pi, may find that their own custom edit tools are more likely to be used incorrectly.”
By the numbers
- 4.8
- Claude Opus version exhibiting regression
- 5
- Claude Sonnet version exhibiting regression
Turn stories like this into views
Ravenclip finds the AI news, makes the video, and posts it before attention moves on.
Common questions
- What happened with Claude Opus 4.8?
- Newer Claude models (Opus 4.8 and Sonnet 5) emit malformed tool calls with invented fields in the edits[] array.
- Where can I read the original report?
- Read the full report at simon_willison.
