Programmatic Tool Calling: The Architecture Replacing JSON Function Calls

The presentation analyzes a paradigm shift in AI agent architecture: moving away from structured JSON output for tool calling and toward letting models write and execute short, typed Python scripts. Using data from the Berkeley Function Calling Leaderboard (BFCL v4) and internal benchmarks, the talk demonstrates that JSON-based tool calling suffers from 'context rot' when faced with numerous tool definitions. In contrast, 'programmatic tool calling' keeps the model's performance stable even with large tool sets and complex call chaining. The speaker highlights key advantages, including significantly lower token usage, superior performance in parallel function execution, and better reliability in complex, multi-step tasks. While acknowledging that JSON was a necessary starting point when models lacked code-writing reliability, the talk asserts that programmatic execution is now the superior, industry-standard approach.