Claude can explain supplied material, but an application needs a retrieval pipeline to choose relevant private or current documents. Retrieval quality determines answer quality: poor source selection, stale content, and missing metadata cannot be fixed by more persuasive prompting.
Grounded workflows should return source identifiers alongside claims. Use citations as a user experience and audit feature, then verify that cited evidence actually supports the answer through automated tests and human review.
Retrieval is a search and ranking problem before it becomes a language-model problem. Chunk documents with enough surrounding context, retain title and location, and evaluate whether the retrieved evidence contains the answer. A model cannot cite a policy excerpt that was never retrieved.
Use deterministic filters for tenant, document type, effective date, and permission before semantic ranking.
Every source needs ownership and access metadata. A document indexed for one customer must not become retrievable for another. Keep deletion and retention events connected to the index so removed material stops appearing in answers.
Record the source version at answer time. This supports audit, cache invalidation, and user-visible “updated on” information.
Ask the model to cite supplied source IDs for each material claim, but do not trust citation presence alone. Check that IDs exist, that sources were available to the requester, and that the cited excerpt is relevant to the claim.
A citation that points to a broadly related document is not enough. Build evaluation cases where distractor sources are present.
Abstention is correct behavior when retrieval finds no sufficiently relevant approved evidence. Provide a safe next step such as opening a support case, asking a clarifying question, or pointing the user to the source library.
Measure retrieval recall, grounded-answer accuracy, citation precision, abstention quality, latency, and cost independently.
The user should be able to see which policy paragraph supports an important claim. Keep the source ID and short excerpt with the retrieval result so the answer renderer has something concrete to show.
If the allowed source set does not contain an answer, return that fact. A helpful refusal is better than an invented policy explanation.
Access control is applied before a semantic retriever or model sees the candidate documents.
documents = [
{"id": "p-1", "tenant": "acme", "text": "Refund policy"},
{"id": "p-2", "tenant": "other", "text": "Private policy"},
]
def allowed_docs(tenant: str) -> list[dict]:
return [doc for doc in documents if doc["tenant"] == tenant]
print([doc["id"] for doc in allowed_docs("acme")])['p-1']Validate that every cited source belongs to the retrieval set before displaying an answer.
retrieved_ids = {"p-17", "p-22"}
def citations_are_allowed(citations: list[str]) -> bool:
return set(citations).issubset(retrieved_ids)
print(citations_are_allowed(["p-17"]))
print(citations_are_allowed(["secret"]))True
FalseThe record contains enough information for a response UI to link a claim back to its source.
source = {
"id": "refund-policy-14",
"title": "Refund policy",
"excerpt": "Returns are accepted within 30 days.",
}
print(source["id"])refund-policy-14A missing source is a workflow state, not an error to hide with a confident answer.
def answer_state(source_ids: list[str]) -> str:
return "answer_with_citations" if source_ids else "needs_human_help"
print(answer_state([]))needs_human_helpA retrieval-design lab
0 of 2 completed
No. They create evidence traceability, but retrieval quality and citation-support evaluation remain necessary.
A clear statement that approved evidence was insufficient plus a safe next step.
Explore 500+ free tutorials across 20+ languages and frameworks.