We migrated DTO to Gemini-based AI. Then we hit a wall. A big one. The context window.
The Challenge: Why Context Limits Matter
From day one, we knew the move to generative AI would come with a challenge. How do you feed a massive optimisation dataset into an LLM without overwhelming its context limits? You can’t just dump everything in and hope for the best.
We see this clearly with clients like Kubota Australia. A typical optimisation run involves 40,000 rows of data across 10 to 20 columns. That’s well over 400,000 individual cells of information. The model caps out long before it can process all of that.
This isn’t a niche problem. Any organisation working with large datasets faces similar constraints when integrating with generative AI. The context window of even the most advanced models is limited, and trying to force large datasets through it leads to truncation, loss of context, and poor results.
The Initial Workaround: A Temporary Fix
Our initial workaround was crude but necessary. Strip out context to fit the limit. It worked for some scenarios, but not all. We knew we needed a better solution.
The problem with stripping context is that it’s a blunt instrument. You lose information that might be relevant to the optimisation problem. For simple scenarios, it might be acceptable. But for complex, real-world problems, it’s not sustainable.
The Real Solution: A Toolkit Approach
So we stepped back and rethought the approach. Instead of throwing the entire dataset at the model, we’re now giving the AI a set of tools. A summary of available data, with the ability to retrieve specific information by sales order, by item, or by any other relevant identifier.
The model queries only what it needs, exactly when it needs it. That gives us fine-grained control over how data is served without the context bloat. The AI doesn’t need to hold the entire dataset in memory. It just needs to know what’s available and how to access it.
What This Unlocks
The approach solves the immediate problem and opens up a more agentic way of working with DTO. Long term, we’re giving the AI tools to interact directly with the front-end web application. That enables stock optimisation, dynamic allocation from inventory to open orders, and more.
For our clients, this means more accurate optimisation results, faster processing times, and the ability to handle larger and more complex datasets. For us, it’s a step towards a more flexible and powerful DTO platform.
We’re currently working towards launching this. It’s going to fix the core limitation and position DTO for a much more flexible future.
