The interesting part here is that the API design is doing more than simplifying tool calling it is reducing the amount of state the model has to reconstruct on every turn. A {slot, value} interface gives the model a small grammar while the workflow engine owns progression, validation, and side effects. That separation makes reliability much easier to reason about.
The 43/44 result is especially useful because the benchmark measures what actually reaches search_availability and book_court, rather than judging whether the conversation merely sounds correct. That is the kind of boundary I’d want in production agent evaluation too: measure the deterministic outcome, not just the quality of the model’s prose.
This maps closely to what we see in production AI work at IT Path Solutions: keep the model flexible at the language boundary, but make the business-critical state transitions explicit and testable. The next interesting step is probably graceful handling of exceptions letting the deterministic workflow remain the default while selectively handing genuinely ambiguous states back to the model.
The interesting part here is that the API design is doing more than simplifying tool calling it is reducing the amount of state the model has to reconstruct on every turn. A
{slot, value}interface gives the model a small grammar while the workflow engine owns progression, validation, and side effects. That separation makes reliability much easier to reason about.The 43/44 result is especially useful because the benchmark measures what actually reaches
search_availabilityandbook_court, rather than judging whether the conversation merely sounds correct. That is the kind of boundary I’d want in production agent evaluation too: measure the deterministic outcome, not just the quality of the model’s prose.This maps closely to what we see in production AI work at IT Path Solutions: keep the model flexible at the language boundary, but make the business-critical state transitions explicit and testable. The next interesting step is probably graceful handling of exceptions letting the deterministic workflow remain the default while selectively handing genuinely ambiguous states back to the model.