The two-run test: why determinism decides everything
Run any AI visibility tool twice on the same business an hour apart, changing nothing. If the number moves, it was never measuring the business.
The test
Evaluate a business. Wait an hour. Evaluate it again with identical inputs. Compare.
If the result moved, the tool is reporting variance in its own process and calling it a finding about your business. That is not a minor flaw. It makes every subsequent comparison meaningless, because you can no longer tell improvement from noise.
Why so many tools fail it
The quickest way to build an AI visibility checker is to ask a language model what it thinks of a business and report the reply. It produces plausible output immediately and requires almost no infrastructure.
Language models are sampled rather than queried. The same prompt can return materially different answers, all reasonable. A single reply is one draw from a distribution, so a score built on one draw moves on its own.
The commercial consequence: a business that acts on a two-point drop caused by sampling spends a quarter fixing something that never happened, and a business whose real decline is hidden inside the same noise does nothing.
How a deterministic evaluation is built
Most of it comes from signals that are stable by nature: what is present in the markup, what is reachable, what resolves, what a third party has published, whether descriptions agree across sources. Given the same inputs these produce the same output every time, because they are observations rather than opinions.
Where a sampled signal genuinely is the thing being measured, such as whether an engine names a business, it is sampled many times and reported as a rate with its sample size attached. That is honest use of a variable signal. Reporting one draw as a fact is not.
Determinism is not the same as being right
A deterministic evaluation can be consistently wrong: the wrong signals, weighted badly, would produce a stable and useless number. Determinism is a precondition, not a proof of validity. What it buys is the ability to tell whether anything changed, which is the minimum requirement for measuring improvement at all.
Apply it to us
This is not a claim to take on trust. Run the two-run test on this evaluation and on anything else you are considering. A tool that cannot survive it should not be used to make decisions, and that includes ours.