A few weeks ago, we wrote about giving our AI tools instead of dumping data into its context window. Here’s how that actually works.
The Short Version
Instead of throwing 400,000 cells of information at the model and hoping for the best, we let the AI ask for what it needs.
But there’s more to it than that. The interesting part is what happens when the AI makes a request.
The Tool Call Architecture
When the AI responds, it can do so in a special format. That format tells our server: “I need to make a tool call.” The server then goes and does the processing, gathers all the relevant data for the solve, and returns it. The AI doesn’t need to hold everything in memory. It just needs to know what to ask for.
This does two things.
First, it offloads some of the processing the AI would otherwise have to do. We’re doing that work in standard computational ways. Deterministic. Predictable. We know the data is 100% correct.
Second, it plays to the AI’s strengths. AI can be a bit stochastic. A bit random-ish. That’s fine for some things, but not for data integrity. So we let the AI orchestrate, and we let the server do the precise work.
The Intermediary
If you’ve used something like LangChain, this pattern might sound familiar. You talk to the AI. The AI talks back. But in most setups, there’s no intermediary. It’s just you and the model.
We’ve added that intermediary. It’s our server. And it now does a lot more than pass messages back and forth. It does complicated processing to figure out the solution, then hands the result back to the AI.
The result is a system where the AI can orchestrate complex workflows by calling tools, and our server does the heavy lifting behind the scenes. More capability. Less context bloat. Better outcomes.
What This Unlocks
Long term, we’re giving the AI tools to interact directly with the front-end web application. That enables stock optimisation, dynamic allocation from inventory to open orders, and more.
We’re still working towards launch, but it’s coming together nicely.
