How do you check an AI summary for missing information?

Check an AI summary in two directions: compare each important statement with the source, then read the source for important information the summary has left out. A summary can contain only true statements and still be unsuitable for a decision if it drops an exception, condition or unresolved question.

This guide is for UK office teams turning an approved report or briefing into a short internal update. The method is general editorial guidance, not a legal compliance test or a benchmark of any particular AI product. It includes a fictional briefing, a deliberately flawed summary and a corrected version you can check yourself.

• Verify what the summary says and what the reader needs to know but cannot see.
• Preserve conditions, exclusions, quantities and uncertainty when they affect the intended action.
• Record the source version and the correction. A second fluent answer is not evidence that the first one was right.

Start with a concrete purpose: “Tell staff who may join the pilot, when it could begin and what remains unapproved.” That gives you a basis for deciding what must survive compression. A general request to make a document shorter leaves that decision implicit.

Use an authorised, complete source. Check that pages, tables, footnotes and appendices relevant to the task are present and readable. A missing page or failed text extraction cannot be repaired by asking the model to sound more certain. Confirm that the document is the applicable version and that you may put its contents into the chosen service.

UCL's library guidance warns that AI summaries can miss key points and give greater attention to material occupying more space rather than material that matters most. Its advice is written for academic research; the office review method below is our own practical adaptation. UCL guidance on AI-generated summaries

Summary to source: break important sentences into separate claims. Find the supporting passage for each. Check the actor, action, number, date and strength of the statement. “May start” and “will start” are different claims even when the date is identical.

Source to summary: read the source against the intended task. Look for restrictions, dependencies, exceptions and unknowns that would change what the recipient does. Mark each as retained, deliberately excluded with a reason, or missing and needing correction. Not every detail belongs in every summary; the reason for leaving it out must fit the purpose.

Microsoft's evaluation guidance makes a related distinction: groundedness concerns consistency with supplied context, while completeness concerns coverage of expected information. Those are evaluation concepts, not a guarantee that an automated judge will catch every defect. The following manual record does not require Microsoft's service. Microsoft RAG evaluation concepts

Source OFFICE-019-v1 — invented for this exercise. No client document, real pilot or actual model test is represented.

A — Scope and start. A desk-booking pilot is proposed for volunteer employees at the Bristol office only. Its planned start is 2 November 2026, conditional on written Facilities approval by 30 October. If approval is not received by then, the start date must be decided again.

B — Capacity and duration. The pilot would cover 20 of the office's 80 desks for four weeks. Fixed accessible seating is excluded from the pilot and remains available outside it.

C — Cost. Costs and savings have not yet been estimated. No savings figure is approved for the staff update.

D — Responsibility. The Facilities manager owns the proposal and will review progress after two weeks if the pilot starts.

Deliberately flawed sample summary: “Bristol will launch a four-week desk-booking pilot on 2 November for all employees, covering 80 desks and saving £4,000.” This sentence was constructed to demonstrate errors; it is not attributed to an AI vendor or a measured test run.

The proposed coverage is 20 ÷ 80 × 100 = 25% of desks. That is a capacity proportion, not a utilisation rate, participation rate or saving. The calculation can help expose the mistaken 80-desk claim; it adds no evidence about whether the proposal should be approved.

Corrected fictional summary: “The Bristol office proposes a four-week desk-booking pilot for volunteer employees, using 20 of its 80 desks. The planned 2 November 2026 start depends on written Facilities approval by 30 October; without that approval, a new start date must be decided. Fixed accessible seating remains available outside the pilot. Costs and savings are not yet estimated. The Facilities manager owns the proposal and will review progress after two weeks if it starts.”

Completed exercise record SUM-019-v1: Purpose: staff participation and launch-status update. Source: OFFICE-019-v1, sections A–D. Review result: the flawed sample must be replaced because it changes participation, authority and capacity, invents a saving and omits an exclusion and follow-up. Corrected output: the paragraph above. Check performed: all eight table rows compared with the displayed source; the capacity proportion recomputed. Remaining boundary: this checks fidelity to the fictional briefing, not the truth or approval of a real workplace proposal. Actual release authority: none; exercise only.

For real work, add the source location, version, reviewer name, date, corrected output version and actual circulation decision. Keep the source accessible to authorised readers. If a source contradicts itself or the decision owner disputes it, record the issue and obtain clarification instead of selecting whichever passage makes the summary easiest to write.

You can give an approved AI tool this instruction: “Summarise the supplied document for [reader and task]. Preserve conditions, exceptions, quantities, dates and uncertainty that affect the task. Identify the supporting section for each important claim. Separately list relevant information omitted and why. If the source does not establish something, say so.”

This is a prompt to test, not a proven safeguard. Anthropic recommends allowing uncertainty, grounding claims in source passages and checking citations. The person reviewing the result still needs to open the passages: a convincing reference may not support the attached claim, and an omission may have no citation to inspect. Anthropic guidance on reducing hallucinations

If you operate a shared knowledge assistant, use our claim-level citation testing guide for the wider release evaluation. If the source is a meeting, verify decisions before circulating AI meeting notes addresses participant disagreement and the chair's role.

For consequential legal, financial, health or safety decisions, a faithful summary is only one input. The responsible specialist must check the underlying evidence and applicability. Do not use this short exercise as a substitute for that judgement, or treat it as proof that a model is reliable across other documents.

Primary guidance checked 19 October 2026: UCL Library, AI-generated summaries of information, page displaying an update of 15 October 2026; Microsoft Learn, RAG evaluators, groundedness and completeness definitions; Anthropic, Reduce hallucinations, source grounding and uncertainty guidance. The Microsoft and Anthropic pages are living documentation; no separate effective date is asserted.

Prepared by QSE Advisory with AI-assisted research and drafting. The fictional briefing, sample error, comparison method and corrected output are original editorial examples. Reviewer and Assessor: Laraib Naveed Ahmad. QSE Advisory is part of QSE Advisory.

Spotted an issue? Contact our editorial desk.