Cloudflare reported 10,247 visitors; our own cookieless analytics counted 282 sessions

# The Number That Was Lying to Us

Roadmap decisions on this estate were being steered by a number that measured the wrong thing. It was a good-looking number, too, which is the dangerous kind.

Symptom

The symptom was a gap. On 2026-07-23, first-party cookieless site_events showed paws-of-london at 282 sessions over 30 days. Cloudflare zone-analytics showed paws_cf_uniques_30d at 10247 unique visitors for the same window. That's a ratio of about 36x.

Then I looked at hilyt. First-party: 46 sessions over 30 days. Cloudflare: 3968 uniques. About 86x.

Nobody had a bug report, you know, nothing was broken in the usual sense. No error, no alert, no failing test. The dashboards were all green and the sites were all up. The only thing wrong was that two sources of truth disagreed by more than an order of magnitude, and one of them had been quietly driving decisions for a while.

What we thought it was

The first diagnosis was the boring one: the first-party tracking must be undercounting. I mean, that's the believable story, right? Cookieless analytics is a deliberate choice, and everyone knows that dropping cookies costs you some accuracy. Ad blockers eat scripts. Consent-free measurement misses people. A tracker you wrote yourself is always the suspect, because you know exactly how much you trust it.

So the working theory was that real people were visiting and our own instrumentation was failing to see them. That theory had two things going for it. It matched a general belief that first-party numbers run low, and it let the bigger figure stay comfortable, because bigger traffic is the answer everyone wants.

But a shortfall from blockers and cookieless measurement is a matter of percentages. It might explain a gap of a bit, or even a good bit. It doesn't explain 36x, and it absolutely doesn't explain 86x. When the discrepancy is that large, the problem isn't a small leak in the small number. It's that the two numbers are counting different things.

Root cause

The actual reason is just that Cloudflare counts crawler IPs as visitors. A "unique visitor" in zone-analytics is, in effect, a distinct address that made requests. It doesn't know or care whether a person was behind it. A crawler hitting the zone from a pool of addresses registers as a crowd.

And this estate invites crawlers on purpose. Every AI crawler is welcome. We prerender thousands of URLs specifically so those crawlers get fast, complete pages. That's the AI-visibility work, and it's deliberate.

Here's the part that stings a bit. The gap widens as that work succeeds. More crawlers fetching more prerendered pages means more distinct addresses, which means a bigger Cloudflare "uniques" figure, while the count of actual humans doesn't move at all. So the number we were steering by was rising for exactly the reason we wanted it to rise, and telling us nothing about audience. A success metric for the crawling strategy was being read as a success metric for the product.

The first-party figures, 282 and 46, come from site_events, which records sessions from real browsers executing our own script. Crawlers mostly don't run it. That's the whole reason it's the better audience measure. It wasn't undercounting. It was counting people, and there were fewer of them than the other number implied.

The fix

The fix was a rule, not a patch, and it was fixed the same day it was detected, 2026-07-23.

Quote first-party numbers only. When anyone asks how many people visit a site, the answer comes from site_events and from nowhere else. The Cloudflare traffic table stays in use, but it gets relabelled in our heads and in our docs: it's bot-inclusive request volume. It tells you how much load the edge is handling and how interesting the estate is to machines. It never gets called audience, visitors or reach.

That relabelling matters more than it sounds. The data didn't change; the name on it did. A figure called "unique visitors" invites a certain reading. A figure called "request volume, bots included" doesn't.

Why it happened

The correction itself was caught by a portfolio audit, and the audit had been briefed with the wrong figures. Read that again. The review that was supposed to be the safety net started from the contaminated number, so it inherited the error rather than testing it. It was only when the first-party data was set next to the Cloudflare data that the disagreement showed up.

So why did it survive? Because the number had a respectable label, and it came from a respectable tool. Nobody questions "unique visitors" on a vendor dashboard. It looked like measurement, so it was treated as measurement. And because the figure was large and growing, there was no pressure to doubt it; a number that flatters you rarely gets audited.

There was also a quieter structural problem. The thing we were doing, welcoming crawlers, and the thing we were measuring, audience, were tangled together in one metric. Success at one inflated the reading of the other. Nobody had separated them, so each win in AI visibility looked like a win in readership.

How to avoid it

If you run any site that invites automated traffic, and these days most do, a few habits transfer.

First, find out what your analytics tool counts as a "visitor" before you let it steer anything. Read the definition. If it's address-based or request-based, it includes machines, full stop.

Second, keep a second measurement that can't be inflated by crawlers, ideally first-party and executed in a real browser, and compare the two on a schedule. You're not looking for agreement. You're looking for the size of the gap, because a gap that grows is information about your bot traffic.

Third, brief audits with the raw source, not a summary someone already built. Our audit failed because it was handed the wrong figures; if it had pulled its own, it would have found this unprompted.

And finally, name things for what they are. Bot-inclusive request volume is a perfectly useful number. It just isn't people.

Anyway, 282 and 46. Those are the people.