Chris M.Reviewed
How a System Finds the Right Record and Cites It
This is part four of the AI and Software Architecture Translation Guide, the six part series on how modern AI systems are actually assembled, following on from part three on waiting and where data lives. This part covers the machinery behind every credible claim that a system can answer questions about your own project information, rather than about the internet in general.
Five terms that build on each other
| Term | What it actually is |
|---|---|
| JSON | A common machine readable format for passing structured information between applications: key and value pairs for project, package, date, status, description |
| Metadata | Information about information. A site note tagged with project, package, location, event type, author and date so it can be filtered, permissioned and governed |
| Embeddings | Numerical representations of meaning, which let a system find conceptually similar information even where the wording is completely different |
| Vector search | Search based on semantic similarity rather than exact words. A search for water ingress may find a record describing dampness around a window sill, even though neither phrase appears in the other |
| RAG | Retrieval augmented generation: retrieval finds the relevant records first, then supplies them to the model as evidence, so it answers from your project information rather than from general training |
Metadata, in a construction context, is usually more valuable than the AI layer sitting above it. It is the filter that decides what a system is even allowed to look at, before meaning ever enters the picture. The retrieval step is what makes citation possible in the first place.
Read the diagram left to right and note where the permissions sit. They are applied at the metadata filter, before the model ever sees a record. Retrieval is the control point, not the model itself. If the wrong records are retrieved, or a person's permissions are not applied at the filter stage, a perfectly good model will still produce a confident and unusable answer.
Why semantic search finds things keyword search misses, and what it still gets wrong
A query for "water ingress" will not literally appear in most site records that describe the same problem. Semantic search finds "damp patch under the cill on C4" in a site diary with no keyword match at all, "staining to plasterboard, roof leak?" in an inspection note, and "penetration through the vapour barrier" in an RFI, because it is matching meaning rather than exact wording. It can also produce a false positive, matching "water bowser delivery to compound" on shared vocabulary rather than shared meaning, and it can miss evidence entirely if the record that matters is a note that just says "see attached, as discussed" with the substance sitting in an attachment the system never indexed.
Picture a director asking what recurring envelope delays a portfolio has had. The system filters by project and package metadata, retrieves the relevant site diaries, minutes and correspondence by meaning rather than by keyword, then supplies that selected evidence to the model. The answer comes back as a structured summary where every point links to its underlying record. The value here is not that the model is clever. It is that a question which would previously have taken a graduate two weeks of folder trawling now takes minutes, and the output is traceable rather than anecdotal.
The limitation has to be stated every time. Vector similarity is not proof. Retrieval can miss relevant records and can return irrelevant ones. It will miss evidence living in an unindexed attachment, a photograph with no caption, a scanned document that was never processed, or a conversation that only ever happened verbally. High consequence outputs must preserve their source references, surface their own uncertainty, and go to human review rather than being taken as settled.
A retrieval system produces records that may support a position. It does not produce a delay analysis on its own, and it should never be described as doing so. The distinction matters the moment the output is ever tested.
Three questions worth asking any supplier claiming search over your data:
- What is indexed, and what is deliberately or accidentally left out?
- Are permissions applied before retrieval, or only after the model has generated an answer?
- Can I open the actual source record from the answer in one click?
The academic paper that first named this pattern, retrieve then generate, is still worth reading for the underlying idea even though real implementations have moved on substantially since it was published; see Lewis et al.'s original RAG paper for that grounding, and Anthropic's writeup on contextual retrieval for how the retrieval step itself has improved. What retrieval hands off to, once a task genuinely needs more than lookup, is covered next in agents, tools and orchestration.
Sources
- 1.AI Metric Ltd, AI and Software Architecture Translation Guide, edition v3.1PrimaryAccessed
- 2.Lewis et al., Meta AI Research, Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksPrimaryAccessed The paper that named the pattern in 2020. Implementations have moved on substantially since; cited for the underlying idea, retrieve then generate, not as a description of any current product.
- 3.Anthropic, Introducing contextual retrievalAuthorityAccessed
Published , last reviewed . This guide explains general principles and is not legal, contractual or safety advice. The position on any project depends on the contract signed and the facts of that project.