A Ten-Step Routine for Checking AI Drafts Before Members See Them

By the end of this tutorial you will own a repeatable routine for checking AI-assisted drafts headed to members: mark each claim, trace it to the source, open every citation, restore dropped hedges, then sign the final version and log what changed in one line. Bring the draft, its source, and a colleague for specialist lines.

Readers distrust machine-made copy: in the Reuters Institute’s 2025 report, only 12% of about 12,000 respondents across six countries were comfortable with news made entirely by AI, against 62% for news made entirely by a human journalist. FActScore researchers who in 2023 scored long-form AI biographies fact by fact found only 58% of the facts held up, and this routine catches the rest.

Those studies tested 2023-era models; today’s are far more capable, and an AI tool can now run much of this routine: extracting claims, opening citations, comparing draft against source. Use it for the mechanical parts. It cannot sign. The signature is accountability, and that belongs to a person, a principle the EU AI Act writes into law. This routine checks the draft, not the model, which is why it outlives every generation. This site is written with AI assistance, and every post here goes through its own version of this routine before it publishes.

Stage 1: set up the comparison

1. Open the source next to the draft. A 2023 ACM Computing Surveys review found language models regularly produce text “nonsensical, or unfaithful to the provided source input.” That document is your ground truth; without one, every claim stays unconfirmed.
2. Name your expert before reading. Hallucinations concentrate in low-frequency, long-tailed knowledge (ACM Computing Surveys review): the legal, technical, and regulatory lines. The EU AI Act demands consequential machine-assisted findings be “separately verified and confirmed” by people “with the necessary competence, training and authority.” The law does not govern your newsletter, but the principle travels.

Stage 2: mark claims, then trace them

3. Highlight everything a reader could verify. Numbers, dates, fees, names, titles, quotations, links, positions taken. An AI tool can extract this list for you; you still read it against the draft. A claim you skip is a claim you never checked.
4. Trace each highlight to the source, never to the chatbot’s memory. The ACM Computing Surveys review calls output that can “neither be supported nor contradicted by the source” an extrinsic hallucination. If you use AI to help check, give it the source and ask it to compare. The tool that wrote the draft cannot judge its own draft.
5. Open every link and citation. In 2023, asked for papers on two rare diseases, one AI tool returned “a thorough paper with several citations with PubMed IDs,” yet “the provided paper titles were fabricated and the PubMed IDs were associated with other papers” (Cureus). Their advice: “AI-generated responses should be verified with reliable” sources. Cut whatever you cannot open and confirm.

Stage 3: catch what the source never said

6. Flag every detail the draft added. Long AI outputs “often contain a mixture of supported and unsupported pieces of information” (FActScore), which defeats simple good-or-bad judgments. Anything the source never gave must be confirmed in a second source you opened, or cut.
7. Put the hedges back. Hallucinated text “gives the impression of being fluent and natural despite being unfaithful,” making it “hard to capture at first glance” (ACM Computing Surveys review). If the source says proposes, the draft says proposes. Certainty the source never expressed is fabrication in good grammar.
8. Ask what is missing. Read as a member would: the comment deadline, the effective date, the next step. A draft can pass every check above and still fail members by omitting the one line they needed.

Stage 4: settle every flag, then sign

9. Close every flag, then sign. Fix, cut, or confirm each one; have your expert read the specialist lines; put your name on the final version. The EU AI Act gives human overseers the power to “disregard, override or reverse the output” of an AI system. That is the job here.
10. Write one log line. Date, item, what changed, who checked. The EU AI Act requires logs to record “the identification of the natural persons involved in the verification of the results.” Your spreadsheet proves the check happened.

The routine, worked through once

This is a teaching example, not a case study. The Riverbend Society of Water Engineers asked an AI tool to turn a state licensing board notice into a newsletter item. The prompt was careful, which is exactly the point:

Turn this licensing board notice into a 60-word newsletter item for members. Keep every date and number exactly as written. Do not add anything the notice does not say.

Notice:
[paste the notice here]

The source, a licensing board notice:

The Board proposes to raise the continuing education requirement from 24 to 30 hours per two-year renewal cycle. Written comments close November 14, 2026. The Board plans to vote in early 2027.

The draft came back like this:

Big news: the licensing board has raised continuing education to 30 hours, effective January 1, 2027. A 2025 workforce study found most engineers already exceed that figure, so start planning your courses now.

The filled-in worksheet. Copy this template for each draft you check:

FACT-CHECK WORKSHEET
Item: Newsletter item, CE hours notice (Riverbend Society)
Source: State licensing board notice
Checker: comms manager | Expert: licensure chair (specialist lines)

[1] Source open next to draft? YES: board notice
[2] Expert named before reading? YES: licensure chair
[3] Claims marked: 30 hours | "has raised" | Jan 1, 2027 effective | 2025 workforce study | (no citation given)
[4] Traced to source, not chatbot? YES
[5] Every citation opened? The workforce study has no citation to open: CUT
[6] Draft added anything? Jan 1, 2027 date: not in source: CUT
[7] Hedges intact? "has raised": source says "proposes": FIXED
[8] Anything missing for members? Comment deadline Nov 14: ADDED
[9] Flags closed, signed? YES: comms manager
[10] Log line written? YES (below)

The signed version:

The state licensing board is proposing to raise the continuing education requirement from 24 to 30 hours per two-year renewal cycle. Nothing changes yet. The board plans to vote in early 2027. Written comments close November 14, 2026.

The log entry:

2026-10-02 | Newsletter item, CE hours notice | "has raised" to "is proposing"; invented effective date cut; uncited study cut; comment deadline added | Comms manager; hours confirmed by licensure chair

Check your result with a last read-through

Before it goes out, confirm every number, date, name, and quotation appears in the source or a second source you opened; every link opens to a page that says what the text claims; the hedges survived; the deadlines members need are present; a subject expert read the specialist lines; your name is on the final version; the log holds one line. Anything missing sends you back. For email drafts, our member email tutorial adds voice and sending checks.

Mistakes the polish hides

Asking the tool that wrote the draft to vouch for it fails: the system that produced an error can explain it away. That is different from using AI to check. Give a checking tool the source document and ask it to compare, and it handles steps 3 through 6; ask it whether the draft is accurate from its own knowledge, and you have commissioned a second draft, not a check. Trusting polish fails next. As early as 2019, researchers found people rate machine-written disinformation as more trustworthy than the human-written kind: in the Defending Against Neural Fake News paper, propaganda’s trustworthiness score rose from 2.19 to 2.42 out of 3 after a machine rewrote it. Newer models write more fluently: more reason to check, not less. Cureus calls these “errors with confidence.” Clicking a link without reading the page misses pages that say something else. Checking numbers while skipping gaps sends out an accurate draft missing the deadline. Skipping the expert lets a wrong citation survive. Skipping the log means nobody spots repeat errors.