Beyond the Hype: Building Production-Ready AI Systems

I recently watched a talk by the creator of Retrieval Augmented Generation (RAG), Dan Kila, who developed the technology as part of the research team at Meta. RAG was introduced to give context to large language models (LLMs). These models are known as general-purpose, meaning they’re intended to work with basically anything you give them. By using their pretraining dataset (which, for the most part, is the public internet), they’re able to infer what you want and give a relatively accurate response.

Introducing RAG constrains this general-purpose capability by providing data (or “context”) for the model to work with. For example, if you are asking about the law in the Maldives, you’re less likely to want a model to infer what you’re asking for—you’re going to need the model to nail it, or its usefulness trends down to zero.

To achieve this, RAG functions as a knowledge base using either vector or graph databases to store similar semantic vectors or capture entity relationships to provide the LLM with those necessary information constraints. This is important because it provides the required environment for LLMs to operate meaningfully on an enterprise-compliant scale. But we can go beyond that.

The big takeaway for me from this talk is the shift from models to systems. With only 20% of AI implementations succeeding globally, we need to think of success as a series of precise decisions built into an AI system. This means emphasising retrieval, processing, and presentation rather than the power of a specific model to achieve a specific niche goal.

General-purpose models are not supposed to be perfect—but many enterprise systems are. And by “perfect,” I mean 100% accurate, 100% of the time. While this isn’t inevitable (let’s face it, no systems are perfect), we can fine-tune the systems we build to achieve outcomes within acceptable tolerances.

Which brings me to the next big piece: we build AI systems not as ends in themselves but to leverage domain expertise. When we deploy AI systems into enterprises, we are leveraging institutional knowledge—that is the true currency of business. We do that by using RAG systems and agents to track what to know and where to go according to intricate operational standards, with a production-ready mindset.

We’ve seen AI deployments sadly fail when the approach is to build for testing as proof, rather than to think on an operational scale. These systems are at their best when we provide them with the right context and deploy them within those contexts to thrive. These are business problems, not technology problems, after all.

At Retro Rabbit/Smartek21, we’ve seen great success in applying this thinking in our Smartboxx solution. As a turnkey AI solution for secure, reliable, ready-to-go use cases, we are proud of our ability to work with business leaders and teams to quickly automate, enhance, and augment existing processes—for a 20x reduction in cost and 200x return on investment.

We’d love to demo our Smartboxx solution to you. Get in touch to stay ahead of the innovation curve with us today.