Article
LLM Outputs Wobble. Run It Thrice.
Two September 2026 papers show that even greedy decoding shifts outputs across GPUs and number precisions. A one-table run-it-thrice log measures whether your association's AI drafts hold still before members see them.
AI Operations Quality Control Llm Reliability Association staff
The comforting assumption about AI tools is that turning the randomness down makes them repeatable. Two research papers posted this September show that the wobble sits deeper than the settings. Cooper and colleagues, in a paper submitted on September 22, 2026, put it in one sentence: “Large language model (LLM) outputs are expected to be reproducible under greedy decoding, yet in practice the same model, prompt, and software stack produce different outputs on different GPUs.”
Greedy decoding is the temperature-zero mode your team may treat as the safe, repeatable setting. The cause sits below that setting: floating-point non-associativity combined with hardware-dependent kernel selection, with the resulting rounding differences able to flip output tokens (Cooper et al.).
The mechanism, in plain words: answering a prompt means adding together enormous numbers of small values, and the order of the additions changes the rounding in the final digits. Different GPUs add in different orders, so the same prompt can land on a slightly different total, sometimes enough to flip the next word.
A second paper, also submitted on September 22, 2026, closed the same-machine version of the question: Du and colleagues show that greedy decoding “is commonly treated as deterministic” but “is not precision-invariant,” with the same model, prompt, and decoding algorithm producing different outputs in BF16 versus FP16 on identical hardware. Across six models (1.1B to 7B parameters, four families, divergence also characterized at 12B) and three benchmarks, 49-100% of prompts diverged, and a single token flip often cascades into trajectory-level divergence (Du et al.).
We read both September papers and took away one habit: stop treating a single generation as the answer and start measuring whether the answer holds still. The run-it-thrice log does that work, and it fits in one table.
The run-it-thrice log
Run the exact prompt three times and record each output. The row below is invented to show a filled log.
| Prompt version | Date | Run 1 output | Run 2 output | Run 3 output | Differences spotted | Verdict |
|---|---|---|---|---|---|---|
| v3, wording saved | 2026-10-07 | “Registration closes June 1.” | “Registration closes June 1.” | “Registration closes July 1.” | Run 3 moved the date | rewrite |
How to fill it in
Freeze the prompt first, because a log of a moving target tells you nothing. Save the exact wording, the date, and the tool and settings you used, then run the same prompt three times and copy each output word for word. Write down every difference between the runs, even a small one, because a single flipped word can reroute the whole rest of an answer (Du et al.). Then mark the verdict. Stable means all three runs agreed, so keep the prompt and keep checking it as you would anything member-facing. Rewrite means the runs disagreed in ways that change the meaning, so tighten the prompt or pin down the format and log again. Human-only means a person writes it instead.
What goes wrong
The most common failure is treating one good run as proof the prompt is reliable. The second is testing only the happy-path prompt, never the long, odd version your members will actually paste in. The third is turning the temperature up for a livelier draft and acting surprised when three runs disagree: that setting adds randomness by design, and the September papers show even the zero setting wobbles.
If the question is whether the draft’s claims are true rather than whether the output holds still across runs, our fact-check routine gives you the claim-by-claim procedure for that job.
What this means for you
Never present a single model run to members as the answer. Treat the first generation as untested: promising, but not yet cleared. Build your member-facing AI so the wobble gets caught before a member sees it, with a run-it-thrice check on every prompt that matters and a person holding the sign-off on everything else.
Sources
- Cooper et al., arXiv 2609.25624 (submitted September 22, 2026). https://arxiv.org/abs/2609.25624
- Du et al., arXiv 2609.26621 (submitted September 2026). https://arxiv.org/abs/2609.26621
Sources
- Cooper et al., 'Accelerating the Mitigation of LLM Inference Nondeterminism Across GPU Architectures' (arXiv 2609.25624, submitted September 22, 2026)
- Du et al., 'Greedy Decoding Is Not Precision-Invariant: Cross-Precision Output Divergence in LLM Inference' (arXiv 2609.26621, submitted September 22, 2026)