The simplest RAG that works, and what a constrained environment teaches you
Retrieval augmented generation is four steps, and most explanations make it sound like fourteen. Here is the shape of it, and the design lessons that only appear once the model you are working with has limited permissions and a small context window.
Written generically. This is the pattern and the lessons, not the internals of any employer's systems.
The problem RAG actually solves
A language model knows what it was trained on. It does not know what your technical specification said after somebody edited it this morning, and it will answer questions about that document with total confidence anyway.
You can solve that by retraining the model on your documents, which is expensive, slow, and stale the moment the next edit lands. Or you can stop trying to put the knowledge inside the model and instead hand it the relevant passage at the moment of the question. That second approach is RAG, and its entire appeal is that updating knowledge means updating a document.
Four steps, that is the whole pattern
One, the spec changes. Somebody updates the source of truth. That edit is the only thing anyone has to do for the system to become current.
Two, it gets chunked and indexed. The document is split into passages and written to a retrieval store, each chunk searchable by meaning rather than by keyword.
Three, a question retrieves the relevant chunks. A user asks something in plain language, the store returns the handful of passages most likely to contain the answer.
Four, the model answers using those passages. The retrieved text goes into the prompt alongside the question, and the model writes an answer grounded in the passage rather than in its training data.
That is it. Everything else in a RAG system is an optimization of one of those four steps.
What the constraints teach you
In an enterprise setting the model does not get to see everything, and the context window is smaller than you would like. Both constraints are usually treated as obstacles. They are better treated as design pressure, because they force decisions that improve the system anyway.
Limited permissions mean retrieval is a filter, not just a search. When a model can only be shown approved material, the retrieval layer becomes the boundary that decides what is eligible to be seen at all. That is a feature. A system that structurally cannot surface something it should not is far easier to trust than one that relies on the model behaving.
A small context window makes chunking the whole game. If you can only pass a few passages, those passages have to be the right ones and they have to be self-contained. Chunks that split a definition across a boundary produce confidently wrong answers. Chunk optimization sounds like a tuning detail and is actually the difference between a system people trust and one they quietly stop using.
Prompt structure has to carry the discipline. With little room to spare, the instructions have to state plainly that the answer comes from the supplied passages, and that not knowing is an acceptable output. A model that invents a plausible answer when retrieval fails is worse than no system, because the failure is invisible.
Where it pays off
The benefits are unglamorous and large. Knowledge stays current because the document is the interface. Answers can be traced to a source passage, which matters enormously in any environment where being right is not enough and you also have to show why. Nobody has to learn a query language. And the operating cost of keeping it accurate is one person editing one document.
At JPMorganChase, work on internal AI and automation of this kind contributed to a 62% improvement in operational speed. I will leave the specifics there, for reasons any engineer who has worked in a bank will recognise.
The honest limitation
RAG does not make a model correct. It makes it grounded, which is a different and lesser claim. If the retrieved passage is wrong, outdated, or subtly the wrong section, the answer will be wrong with a citation attached, and a citation makes people trust it more, not less.
That is why the boring parts matter most: keeping the source documents accurate, chunking so passages stand alone, and instructing the model that "I do not have that" is a valid answer. Get those right and the four-step pattern above is genuinely all you need.
Thinking about a retrieval layer over your own documents? Get in touch.