Retrieval-augmented generation: architecture that holds up in production
A working retrieval pipeline is straightforward. Making it reliable enough to trust with business questions is where the engineering is.
ZooBieDoo Infotech Engineering TeamRead
Key takeaways
- Start with an inventory — shadow deployments are the usual gap.
- Document purpose, data, evaluation and oversight while building.
- Human review must be genuine, not a rubber stamp.
- Monitor drift and override rates after deployment.
Know what you are running
The first governance failure is not knowing which AI systems exist. Shadow deployments — a team wiring a model into a process without review — are common and are what regulators and auditors find.
Maintain an inventory: purpose, data used, model, who owns it, what decisions it influences and what oversight exists.
Document each system
- Intended use and, explicitly, out-of-scope uses.
- Training or retrieval data provenance and any known limitations.
- Evaluation results including where it performs poorly.
- Human oversight: who reviews what, and when.
- Monitoring in place and the escalation path.
Written at the time, this is a modest document. Reconstructed a year later under audit pressure, it is a project.
Design the human review step
Human oversight is often described as a safeguard and implemented as a rubber stamp. If a reviewer sees hundreds of items and approves nearly all, the oversight is not real.
Design it so the human sees the evidence, has time to assess, and can disagree without friction. Route only genuinely uncertain cases to review so attention is not diluted.
Monitor after deployment
Data drifts, usage changes, and behaviour that was acceptable at launch may not remain so. Sample outputs for quality on an ongoing basis and track the rate of human overrides — a rising override rate is an early warning.
Set a review cadence per system proportional to its risk, and record that the review happened.
Being able to explain a decision
Where an AI system influences decisions affecting people, being able to explain a specific outcome matters both to regulators and to the person affected.
Retain the inputs, the model version, the retrieved context and the output for each consequential decision, for a period proportionate to the risk. Reconstructing why a system produced a particular answer months later is otherwise impossible.
This is a design decision with storage and privacy implications, which is why it belongs in the initial architecture rather than being added when the first challenge arrives.
Frequently asked questions
Yes. Governance concerns how you use the system and what decisions it influences, regardless of who trained the model.
Use enterprise endpoints with training opt-out and retention terms, or self-host open-weight models where policy requires it. Record which applies to each system.
It works best shared: a business owner accountable for outcomes and a technical owner accountable for behaviour. Placing it solely with either produces gaps.
