What "AI-ready" actually has to mean
Readiness is not a property a dataset acquires by being loaded somewhere. It is a set of questions the estate can answer about a record: who owns it, whether it is current, who is entitled to see it, which version an answer used, and what happens to it when the source deletes it. A corpus that cannot answer those is not unready because of its format. It is unready because nobody decided.
This is why readiness work so often stalls. Teams treat it as an ingestion problem — parse the documents, chunk them, embed them — and the hard part turns out to be organisational: there is no owner for the policy library, nobody knows which of four overlapping spreadsheets is authoritative, and the access rules exist only as folder permissions that were last reviewed when the folder was created.
The useful reframe is that a retrieval system does not lower the bar on data quality. It raises it, because it removes the reader who used to catch the problem. An analyst looking at a dashboard notices when a number is implausible. Someone reading a fluent paragraph with a citation has no equivalent instinct, and the citation makes it look more trustworthy rather than less.
1. Define the task and its failure cost
Write a task contract before choosing a model: who asks, which knowledge is in scope, what an acceptable answer contains and which actions remain with a person. A policy-search assistant and an operational decision agent have different risk and validation requirements. Begin with a read-only task when it can establish usefulness without adding write access.
The failure cost is the part that decides everything downstream. A system that suggests a relevant internal document and is occasionally unhelpful costs someone a minute. A system that states a policy position that turns out to be from a superseded version costs a customer commitment made on wrong information. Those two need different evidence standards, different refusal behaviour and different review, and they are frequently built as though they were the same product.
Scope is the other half. "All company knowledge" is not a scope; it is an absence of one, and it guarantees that the corpus contains drafts, personal notes, superseded policies and material somebody is not entitled to read. Naming the collections in scope, and the owner of each, is what converts readiness from an aspiration into a finite piece of work.
- Build an evaluation set from actual questions, including unsupported questions
- Identify the owner who can judge whether an answer is correct
- Define a fallback path and an acceptable latency/cost envelope
2. Preserve the source contract
For each source, record owner, stable ID, version, update cadence, permitted users and deletion behavior. Carry these attributes through parsing and indexing. If the original source removes a document or changes access, the derived index must follow. Otherwise the system can answer from information its user should no longer receive.
The attribute most often lost in transit is version. A document is parsed, split and embedded, and what survives into the index is text plus a filename — at which point the system can retrieve a passage and cite it without being able to say whether it came from the current policy or the one it replaced. Carrying a stable identifier and a version through chunking is unglamorous plumbing and it is what makes a citation mean something.
The same reasoning that produces a data contract between two engineering teams applies here, with the retrieval index as the consumer: the producer declares what a record means, how often it changes and what happens when it is withdrawn, and the consumer builds on that promise rather than on an assumption.
3. Keep authorization outside the prompt
The retrieval path should constrain results to the requesting user's permitted scope before any text reaches the model. Asking a model not to reveal restricted material is not an access-control design. Test users with different permissions against the same questions and verify both direct answers and citations. Include role changes and revoked access in the test cases.
The reason this has to be structural is that the prompt is the wrong place to enforce a rule that must hold under adversarial phrasing. An instruction not to disclose something is a preference expressed in the same channel as the attack. A filter applied to the candidate set before retrieval returns is a constraint the model cannot talk its way past, because the restricted text was never in the context.
Permissions also change, and the index does not notice. Someone moves team, a contractor leaves, a project closes. Unless entitlement is evaluated at query time against current group membership — rather than baked in at ingestion — the system keeps serving yesterday's access model. How that gets governed, and who owns it, is a separate decision from how it gets implemented.
4. Structure, parsing and what survives the split
Most enterprise knowledge is not prose. It is tables in PDFs, slides with the argument in the speaker notes, spreadsheets where the meaning is in the column header, and documents whose structure carries as much information as the sentences. Parsing decides how much of that survives, and a bad split is not recoverable later by a better model.
Three failures recur. A table is flattened into a row of numbers with no headers, so a retrieved passage reports figures whose meaning is gone. A section is split mid-argument, so the qualifying sentence — the one beginning "this does not apply where" — lands in a different chunk from the rule it qualifies. And a document's heading hierarchy is discarded, so a passage from an appendix looks identical in authority to one from the operative policy.
The mitigations are mostly structural rather than clever. Keep headings with the text beneath them. Keep a table with its caption and column headers, or extract it as structured data and retrieve it that way. Split on document structure rather than on a fixed character count where the format allows it. And where a source is genuinely tabular, consider whether it belongs in a retrieval index at all — a query against the warehouse may be the correct answer, and the warehouse model already holds the grain and the definitions.
- Preserve heading hierarchy so passage authority is legible
- Keep tables with their headers, or lift them out as structured records
- Split on structure where the format allows; fixed-size chunking is a fallback, not a design
5. Freshness is per-source, not per-system
Index lag is the gap between a source changing and the retrieval system reflecting it, and it has to be stated per source rather than as one number for the platform. A product catalogue that changes hourly and a regulatory policy revised twice a year have entirely different tolerances, and a single refresh cadence will be either wasteful for one or dangerous for the other.
What makes this harder than ordinary pipeline freshness is that staleness in a retrieval system is silent. A stale dashboard shows an old timestamp. A stale index returns a confident answer citing a document that was withdrawn last week, and nothing in the response indicates the age of the evidence. Surfacing the source's own last-modified date in the answer is a small change that gives the reader back the signal.
Deletion is the sharp edge of the same problem, and it deserves planning before launch rather than after. A record removed at source stays answerable until something removes it from the index too, and the interval between those two events is a window in which the system can disclose material that no longer exists. Designed in, it is a straightforward propagation job; retrofitted, it usually means a full rebuild.
6. Semantic consistency is what stops two correct answers
If "active customer" means one thing in the CRM, another in the billing export and a third in the analytics model, a retrieval system will find all three and answer confidently with whichever it ranked highest. The model has no way to know they disagree. Semantic consistency is therefore a prerequisite for AI readiness in the same way it is for reporting, and for the same reason.
This is the point where the analytics estate and the AI estate turn out to be the same problem. The work that produces governed semantic models and agreed metric definitions is precisely the work that lets a retrieval system answer a numeric question without inventing a definition. An organisation that has not done it does not get to skip it by adding a model on top; it gets the same disagreement, now in prose.
The practical implication is about routing rather than about embeddings. A question about a defined business measure should be answered from the governed model, not from whichever document happens to contain the number. Deciding which questions go to retrieval and which go to the semantic layer is an architecture decision, and making it explicitly is what stops the system from confidently citing a slide.
7. Sensitive data and what should never be indexed
Some material should not enter a retrieval corpus at all, regardless of who could see it. Personal data that the task does not need, credentials and keys pasted into documents, material under legal hold, and records whose retention clock would be reset by copying them are all cheaper to exclude than to govern. Exclusion is a control, and it is the only one that cannot fail at query time.
The rule that makes this tractable is to index on need rather than on availability. "We could include it" is not a reason; "a question in scope requires it" is. A corpus assembled on availability grows until nobody can say what is in it, which makes every subsequent access, retention and deletion question harder than it needed to be.
Where personal data genuinely is required, minimisation still applies: redact at parse time what the task does not need, keep the identifier rather than the full record where an identifier suffices, and record the basis for holding it. These are ordinary data protection obligations, and a retrieval index is not exempt from them because it stores derived vectors rather than the original document.
- Exclude by default; include because a scoped question requires it
- Redact at parse time rather than filtering at answer time
- Treat derived indexes and embeddings as in scope for retention and deletion
8. Readiness by source, and what each gap costs
Readiness is assessed per source, not per platform, because a corpus is only as answerable as its weakest collection. The dimensions below are the ones that change system behaviour: everything else is preparation. A source failing any of them can still be indexed — but the failure should be a recorded decision with a known consequence, not a discovery made after launch.
Used as a gate, this table has a useful property: it produces a small number of concrete owners rather than a general instruction to improve data quality. Each row has someone who can fix it, and the fix is usually specific — name an owner, carry a version through parsing, evaluate entitlement at query time.
| Dimension | What it requires | Behaviour when it is missing | Owner |
|---|---|---|---|
| Ownership | A named person accountable for the collection's content. | Nobody can confirm an answer is correct, so errors are never closed out. | Domain data owner. |
| Canonical source | One collection designated authoritative where several overlap. | The system cites whichever duplicate ranked highest, inconsistently. | Domain data owner. |
| Stable identity | A durable ID carried from source through parsing into the index. | Citations cannot be resolved back to a specific document. | Ingestion pipeline owner. |
| Version | A version or revision marker that survives chunking. | A superseded policy is quoted as current, with no visible signal. | Ingestion pipeline owner. |
| Entitlement | Access evaluated at query time against current group membership. | Departed staff and moved colleagues keep yesterday's visibility. | Access governance. |
| Freshness | A per-source lag target and a visible last-modified date. | Stale evidence is served with full confidence and no timestamp. | Ingestion pipeline owner. |
| Deletion | Removal at source propagating into index and embeddings. | Withdrawn records stay answerable for an unbounded period. | Access governance. |
| Structure | Headings, tables and hierarchy preserved through parsing. | Passages lose the qualifier, the header or the authority that framed them. | Ingestion pipeline owner. |
| Semantics | Agreed definitions for measures the system may be asked about. | Numeric questions get a confident answer from an undefined source. | Metric owner. |
| Evaluation set | Real questions with known answers, including unanswerable ones. | Nobody can tell whether a change improved or degraded the system. | Product owner. |
9. Evaluate the retrieval and answer stages
A correct document can be retrieved and still be summarized incorrectly; a fluent answer can conceal a retrieval failure. Evaluate the stages separately. Record whether the expected source was available, whether it was retrieved, whether the answer is supported and whether the citation resolves to the right version. Inspect performance for ambiguous queries and stale documents, not only well-formed demonstration questions.
Readiness and evaluation are connected more tightly than they look. Most of the dimensions above are only observable through evaluation — you discover that versions were not carried through parsing when a test question cites a superseded document, and you discover the entitlement gap when a test user sees something they should not. An estate without an evaluation set cannot assess its own readiness; it can only assert it.
What to measure at each stage, and which metrics mislead, is a large enough subject to have its own treatment. The readiness point is narrower: the evaluation set is itself a readiness artefact, and building it is part of preparing the data rather than a later step.
10. Operate updates as releases
Document ingestion, chunking, retrieval configuration, prompts and models can all change behavior. Version the components and re-run an agreed evaluation suite before promotion. Use a controlled release and rollback path. Do not log full confidential prompts or retrieved documents by default; establish what evidence is needed to investigate a fault without creating another uncontrolled data store.
The component most likely to change without anyone treating it as a release is the chunking configuration, because it feels like a parameter rather than a deployment. Changing it alters every retrieval result in the system. Treat it the way a pipeline change is treated: versioned, evaluated against a suite before promotion, and reversible.
A practical acceptance gate
Before extending the collection, demonstrate a successful source update, a deletion, a permission change, a refused unsupported question and recovery from an ingestion failure. Measure task quality, source correctness, latency and cost against the agreed baseline. These are recommended tests for a bounded reference design, not evidence of a Quorlytic client deployment or a universal certification.
The order matters. Each of those five is cheap to demonstrate on a small corpus and expensive to retrofit across a large one, and the failure they catch is the same failure at any scale. A system that can show all five on one collection has proved its architecture; a system that has indexed forty collections without showing any of them has proved only that ingestion works.
What this framework does not cover
This is about the data an AI system reads, not about the system itself. Model selection, prompt design, agent architecture, guardrail implementation and inference cost are all out of scope, and a perfectly ready corpus can still be fronted by a badly designed application. Readiness removes one category of failure; it does not remove the others.
It also assumes retrieval is the right pattern. Some questions are better served by a query against a governed model, some by a deterministic lookup, and some by a form. Indexing more content is not inherently better, and a readiness programme that never concludes "this collection should not be indexed" has not been applied honestly.
Make the applicable source identifiable before retrieval.
The newest file is not necessarily the rule that applies. In this hypothetical support collection, two return policies are both legitimate: one is current for Region A, while another describes Region B and a future effective period. A reader may be allowed to see both. Readiness means carrying enough source context to resolve applicability, not asking similarity ranking to choose the governing policy.
Scope and assumptions: The collection owner defines authority, audience and effective periods; this worksheet does not infer legal precedence or approve a policy. A source can be accessible yet irrelevant to the question. Where the necessary context is unknown, preserve the uncertainty rather than assigning a global winner.
| Dimension | Source property | Decision and acceptance fixture |
|---|---|---|
| Stable source and version | Carry a document identity, revision and canonical location into every derived passage. Keep the parent relationship when one source produces many chunks. | Retrieve a fixture from each revision and trace it back to the exact source. A title match alone must not collapse two revisions into one citation or leave an orphaned old passage. |
| Effective period | Distinguish when a file was edited from when its rule applies. Record any known start, end and superseding relationship using the owner's conventions. | Ask equivalent questions before and after a synthetic policy transition. A recently edited future policy must not displace the presently applicable rule merely because its timestamp is later. |
| Audience and scope | Retain the region, product, customer class or other condition that changes the answer. Applicability is separate from whether the reader has permission to access the file. | Use two permitted sources with different regional scope. If the question omits the region, require the missing context or explain the limitation instead of merging the rules into a universal answer. |
| Conflict and authority | Name who can resolve sources that overlap without an explicit precedence rule. A higher retrieval score is evidence of a match, not of authority. | Include a contradictory pair with the same apparent scope. Preserve the conflict and route it to the collection owner; do not invent a precedence rule to make the fixture produce a fluent answer. |
| Replacement and removal | Define what happens to derived passages, links and version references when a source is replaced or withdrawn. Keep audit retention separate from eligibility for current answers. | Update and withdraw synthetic sources, then verify the intended version is eligible and retired passages cannot support a new current answer. Retain only the audit history allowed by the agreed policy. |
At narrow widths, scroll the worksheet to read every column. With the table focused, use the arrow keys to scroll.
Worked decision: Region B's future policy ranks first because its wording closely matches the question, while the request concerns a Region A order today. The source is permitted and the search result is relevant in wording, but it is not the applicable evidence. Keep the region and effective-period fixture with the source contract; improving the embedding alone does not resolve that missing decision.
- Check that parsing preserves the qualifiers needed to interpret a rule. If a heading or table note carries the region or effective period, a detached sentence is not an adequate retrieval record.
- OpenAI's retrieval example separates the searched evidence from the answer stage. The applicability fields and fixtures here are an illustrative corpus design, not metadata that a model or vendor service supplies automatically.
After establishing which source applies, exercise the enterprise search access-change checks to verify the current reader can use it through the whole application.
Official references for this example:
Questions and trade-offs
Question
Should every document be put in a vector database?
No. Choose retrieval methods from the task, document structure and access requirements. Structured queries, keyword search or a hybrid approach may be necessary. Indexing more content is not inherently better.
Question
How ready does the data need to be before we start?
Ready enough for one bounded task, not ready across the estate. Pick a single collection with a named owner, current content and workable permissions, and prove the five acceptance tests on it. Waiting for estate-wide readiness means the programme never starts; skipping readiness entirely means the first serious error is discovered by a customer rather than by a test.
