Enterprise RAG: moving from prototype to production

A RAG (Retrieval-Augmented Generation) prototype can be built in an afternoon: a language model, a handful of documents, a vector index, and the demo impresses everyone. The distance between that prototype and a production-ready RAG system, on the other hand, is measured in months — and most projects underestimate that distance because the initial demo hides exactly the problems that make industrialization difficult.

What the prototype doesn't show

A prototype runs on a clean, hand-picked corpus, without differentiated access rights, without traceability requirements, and without concurrent users. Production imposes all four at once. The real corpus contains contradictory, outdated or poorly classified documents. Each user must see only what they're entitled to see — a salesperson shouldn't be able to retrieve, through the RAG, HR information they'd never have had access to otherwise. And every generated answer must be traceable back to its sources, for reasons of trust as much as compliance.

Four workstreams for industrialization

Data quality and structure. A RAG is never better than the corpus it queries. Before indexing, you need to decide what enters the knowledge base, who owns it, and how often it's updated — an outdated document confidently cited by a language model is more dangerous than no answer at all.

Security and access control. Vector indexing must respect the same permissions as the source systems. This requires an architecture that propagates access rights all the way to the retrieval layer, not just to the interface.

Monitoring. A production RAG system needs its own indicators: relevance of retrieved documents, rate of correctly sourced answers, cost per query, model drift over time. Without these measures, a quality degradation goes unnoticed until a user reports it — too late.

GDPR and EU AI Act compliance. GDPR applies as soon as personal data passes through the system, including in indexed documents. The EU AI Act classifies certain uses by risk level and imposes proportionate transparency and traceability obligations. These requirements are handled upstream, in the architecture, not downstream as a patch.

The role of governance

Industrializing a RAG isn't only a technical undertaking. It's a governance question: who decides what enters the corpus, who is accountable for answer quality, what compute budget is allocated and how it's arbitrated. Without this governance, every team that adopts RAG does so its own way, with its own implicit rules — recreating exactly the fragmentation enterprise architecture is meant to prevent.

• A RAG prototype demo hides the problems that make industrialization hard.
• Four workstreams structure the move to production: data, security, monitoring, compliance.
• GDPR and the EU AI Act are handled in the architecture, not as an afterthought.
• Without explicit governance, every team adopts RAG its own way — and fragmentation returns.