Three things that get confused with each other
Data governance, retrieval architecture and model safety solve different problems and fail independently. Governance decides which records may be used and by whom. Retrieval architecture decides how candidates are found and filtered. Model safety constrains what the model does with text it has already been given. A system can be strong in two of the three and still disclose something it should not.
The distinction matters because teams routinely substitute one for another. A guardrail that blocks the model from repeating a salary figure is model safety; it does nothing about the fact that the salary document was retrievable in the first place. A permission filter on the index is retrieval architecture; it does nothing about whether that collection should have been indexed at all. Only the governance decision answers that.
A useful test when someone proposes a control: ask which of the three layers it sits in, and what happens if the other two fail. Controls that only work when the model cooperates are the weakest kind, and they are the easiest to demonstrate in a meeting.
Approve sources; do not inherit a file share
Every collection in the corpus should be there because somebody approved it, with a named owner and a stated purpose. The alternative — pointing ingestion at a shared drive and indexing what it finds — produces a corpus nobody can describe, containing drafts, superseded policies, personal notes and material that was only ever protected by being hard to find.
Approval is cheap at the start and nearly impossible to retrofit. Once forty collections are indexed and the assistant is in use, removing one becomes a negotiation about who loses a feature, instead of a decision about whether the content belonged there. Doing it the other way round means the first question about a source is whether it earns its place.
Two records make the difference between a governed corpus and a large one. A source register: what is indexed, who owns it, what it is for, when it was approved, and what classification it carries. And a canonical designation where collections overlap — because when three folders hold versions of the same policy, the retrieval system will surface whichever ranks highest, and ranking is not an authority judgement.
- A named owner per collection who can confirm whether an answer is correct
- A stated purpose, so a later request to add content can be assessed against something
- A canonical designation where sources overlap, instead of letting relevance decide authority
- An approval date, so the register shows its own staleness
Entitlement has to be evaluated at query time
Permissions change constantly — people move teams, contractors leave, projects close — and an index captures them as they were at ingestion. Unless entitlement is evaluated against current identity when the query runs, the system serves an access model that is quietly out of date, and the gap grows every day without producing a single error.
Platform support for this is now concrete rather than theoretical. Azure AI Search documents four approaches to document-level access control: security filters that compare a caller-supplied identity string against a field on the document; POSIX-like ACL and RBAC scope preservation for storage-backed content; Microsoft Purview sensitivity labels evaluated at query time; and SharePoint ACL ingestion. In the token-based patterns the caller's directory token is attached to the request, and results are trimmed to documents whose permission metadata grants that caller access.
The caveat in that documentation is the one to plan around: query-time enforcement compares the caller's claims against permission metadata already stored in the index, so a permission change at source only takes effect once that metadata has been synchronised. Enforcement is current; the metadata it enforces against is as fresh as the last sync. That interval is a governance decision with a number attached, and somebody should have chosen the number.
Where content is chunked, the permission metadata has to travel to the chunk, not stay on the parent document. This is the failure that gets found late: the document-level filter looks correct, the chunks are what actually get retrieved, and nobody checked that the projection carried the ACL fields down.
Classification decides what may be indexed at all
Some material should be excluded from the corpus regardless of who could theoretically see it. Credentials pasted into documents, personal data the task does not need, records under legal hold, and anything whose retention obligations would be complicated by copying it are all cheaper to keep out than to govern inside. Exclusion is the only control that cannot fail at query time.
The operating rule is to index on need rather than on availability. "We have access to it" is not a reason to include a collection; "a question in scope requires it" is. Corpora assembled on availability grow until nobody can state what is in them, and every subsequent question about retention, deletion or disclosure becomes an investigation.
Where classification already exists in the estate, reuse it instead of inventing a second scheme for the AI system. A parallel classification is a second thing to keep current, and the two will disagree within a year. This is the same argument for one classification scheme owned centrally that applies to the rest of the estate — the retrieval corpus is not a special case, it is another consumer.
What deletion means once embeddings exist
Deleting a record at source does not delete it from the system. It remains in the index, in any derived chunks, in the vector store, in caches, and potentially in logs that captured retrieved text. Each of those is a separate deletion path, and a request to remove a document is only honoured when all of them have been walked.
The embedding is the part most often overlooked, because it does not look like a copy. It is a derived representation of the text, it sits in a store the deletion process may not know about, and it will keep being returned as a nearest neighbour until something removes it. Treating the vector store as in scope for retention and deletion — rather than as an index that can be rebuilt whenever convenient — is the difference between a deletion policy and a deletion intention.
Logs deserve their own decision. A system that records full prompts and retrieved passages for debugging has created a second uncontrolled copy of the corpus, with weaker access controls and a different retention schedule. Decide before launch what evidence is genuinely needed to investigate a fault, and capture that rather than everything.
The practical consequence is that deletion should be designed as a propagation path, with the same seriousness as ingestion, and tested. A deletion test belongs in the acceptance gate alongside a permission-change test, because both fail silently and neither is visible in ordinary use.
- Enumerate every derived store: index, chunks, vectors, caches, logs, evaluation sets
- Treat embeddings as in-scope copies, not as a rebuildable convenience
- Set a deletion latency target and measure it, the way freshness is measured
- Test deletion before launch; it is not observable afterwards without testing
Freshness, versions and the answer that is quietly out of date
A stale retrieval index does not look stale. It returns a confident answer with a citation to a document that was superseded last month, and nothing in the response signals the age of the evidence. That makes index lag a governance concern instead of a performance one: the risk is not slowness, it is unmarked obsolescence.
Two controls do most of the work. Carry a version or revision marker from the source through parsing and chunking, so a citation can resolve to a specific version instead of a filename. And surface the source's own last-modified date in the answer, which hands the reader back the judgement they would have exercised if they had opened the document themselves.
Lag targets belong per source rather than per system, for the same reason freshness is stated per data product elsewhere. A pricing sheet revised weekly and a regulation revised every few years do not need the same refresh cadence, and a single platform-wide schedule will be wasteful for one and dangerous for the other.
Conflicting sources need an authority rule, not better ranking
When two approved documents disagree, relevance ranking will pick one and the answer will be stated with full confidence. Retrieval has no concept of which source is authoritative — only of which text best matches the query. Resolving that requires a rule about authority that somebody wrote down, applied as a retrieval constraint or a re-ranking signal.
The rules that work are unglamorous: the current version outranks superseded ones; the policy library outranks a summary in a deck; a document from the owning domain outranks a copy held elsewhere. What they have in common is that they are properties of the source register rather than properties of the text, which is exactly why ranking cannot infer them.
The alternative failure is worth naming, because it feels like caution: indexing everything and hoping the model reconciles contradictions. It will not. It will produce a fluent synthesis of two incompatible positions, which is harder to detect than either position stated plainly.
Controls, the failure each prevents, and who verifies it
Governance for retrieval is a small number of controls, each attached to a specific failure and a named owner. Listing them this way is more useful than a policy document, because it makes the gap visible: a control with no verification method is a stated intention, and a control with no owner is nobody's job.
Read the last column as the real test. Most of these can be verified in an afternoon on a single collection, and most estates have never run any of them.
| Dimension | Failure it prevents | Owner | How to verify |
|---|---|---|---|
| Source approval register | Unknown content answering questions nobody scoped. | Domain data owner. | Reconcile the index contents against the register; anything unlisted is a finding. |
| Canonical source designation | Two versions of a policy both retrievable, resolved by ranking. | Domain data owner. | Query a known-duplicated topic and check which source is cited. |
| Query-time entitlement | A reader receiving material their current role does not permit. | Access governance. | Run the same question as users in different roles and compare answers and citations. |
| Chunk-level permission projection | Document ACLs that never reached the chunks actually retrieved. | Ingestion pipeline owner. | Inspect a chunk record directly and confirm the permission fields are populated. |
| Permission sync cadence | Yesterday's access model enforced against today's identity. | Access governance. | Change a group membership, then measure how long until retrieval reflects it. |
| Classification-based exclusion | Sensitive material indexed because it was available. | Central data office. | Sample the corpus for classified content that should have been excluded. |
| Deletion propagation | Withdrawn records answerable indefinitely. | Access governance. | Delete a test document at source and query for it until it disappears; record the interval. |
| Version carried to the index | A superseded document quoted as current. | Ingestion pipeline owner. | Check that a citation resolves to a specific revision, not just a filename. |
| Source authority rule | A confident synthesis of two contradictory sources. | Domain data owner. | Ask a question with a known conflict and inspect which source wins. |
| Retrieval and prompt log policy | A second uncontrolled copy of the corpus in logging. | Platform owner. | Read what the logs actually contain, rather than what the design says. |
What belongs before ingestion and what has to run at query time
Approval, classification and exclusion are pre-ingestion decisions: they determine what enters the corpus, and once something is in, removing it is a project. Entitlement, authority resolution and freshness signalling are runtime controls, because the facts they depend on — who is asking, what their role is now, how old the evidence is — are not knowable at ingestion.
Getting this split wrong in either direction is expensive. Pushing approval decisions to runtime means indexing everything and filtering later, which leaves sensitive material one filter bug away from disclosure. Pushing entitlement to ingestion means baking a permission snapshot into the index and watching it drift.
The OWASP Top 10 for LLM Applications frames several of these as first-class risks rather than hygiene: sensitive information disclosure, data and model poisoning, vector and embedding weaknesses, and excessive agency all appear in the 2025 list. It is a useful cross-check on a control set, because it is organised around how systems are actually attacked rather than around how they are meant to be built.
Where this stops and something else has to take over
Governance decides what a system is allowed to read. It does not decide whether the system reads it well. A perfectly governed corpus can still be chunked badly, retrieved poorly and summarised into an answer that misrepresents the document it cites — and none of the controls above will notice, because every one of them passes.
That gap is measurement, and it needs its own treatment: what to evaluate at each stage of a retrieval system, and which metrics mislead. Governance and evaluation are complementary in the same way stated quality rules and behavioural monitoring are — one decides what is allowed, the other finds out what actually happened.
Two further limits worth stating. This covers retrieval over documents the organisation controls; a system retrieving from the public web or a third-party API has a different problem, because the source cannot be governed at all and the honest posture is to treat it as untrusted input. And nothing here addresses agents that take actions rather than answer questions — permission to read and permission to act are separate grants, and conflating them is how autonomy becomes a problem.
Questions and trade-offs
Question
Can we start with everything indexed and add governance later?
It is the most common sequence and the most expensive one. Once the corpus is in use, removing a collection becomes an argument about losing a capability instead of a decision about whether the content belonged there, and the deletion, permission and retention questions all have to be answered retrospectively across everything at once. Starting with one approved collection and proving the controls on it is slower for a fortnight and far faster after that.
Question
Is a permission filter enough on its own?
It is necessary and it is not sufficient. A filter enforces who may see what, against permission metadata that is only as current as the last synchronisation, and it says nothing about whether the collection should have been indexed, whether the version is current, or whether deletion at source has propagated. Those are separate controls with separate owners, and a filter passing is not evidence that they exist.
