There is a widely repeated claim that businesses are losing AI search visibility because their sites block AI crawlers. We decided to test it.
In August 2026 we ran an automated audit against **188 Dutch professional-services websites** — estate agents, law firms, accountancies, insurance brokers, architects, IT consultancies and recruiters, sampled from public OpenStreetMap business records across the Netherlands. Every site was fetched live. We checked each one for the signals that determine whether an AI answer engine can find, understand and cite a business.
The blocking claim turned out to be almost entirely false. The real problem is worse, because it is invisible.
The headline finding
**187 of 188 sites (99.5%) allow AI crawlers in without restriction.** Exactly one site disallowed GPTBot, ClaudeBot and Google-Extended.
So access is not the issue. Dutch businesses are not locking AI out.
**But 173 of 188 (92%) have neither a /llms.txt file nor FAQPage structured data.** They let the crawler in, and then hand it a page with no machine-readable statement of what the business does, what it charges, where it operates, or what questions it answers.
That is the actual gap. Not a locked door. An empty room.
The full results
| Signal | Sites passing | % |
|---|---|---|
| Title tag | 188 | 100% |
| HTTPS | 184 | 97.9% |
| Mobile viewport | 184 | 97.9% |
| robots.txt | 165 | 87.8% |
| Canonical tag | 152 | 80.9% |
| Meta description | 148 | 78.7% |
| H1 heading | 144 | 76.6% |
| sitemap.xml | 129 | 68.6% |
| JSON-LD structured data | 115 | 61.2% |
| og:image | 105 | 55.9% |
| Organization / LocalBusiness schema | 63 | 33.5% |
| /llms.txt | 14 | 7.4% |
| FAQPage schema | 1 | 0.5% |
Read the top of that table and Dutch web development looks healthy. HTTPS is universal. Mobile layouts are universal. Titles are universal.
Read the bottom and something else appears: the further down you go, the more the checks stop being about *humans reading a page* and start being about *machines extracting facts*. That is precisely where the numbers collapse.
Why the bottom four rows are the ones that matter now
A traditional search engine ranks pages and shows you links. You click, you read, you decide. The page only had to be readable.
An answer engine does something different. It reads a set of sources, extracts claims, and composes a single answer that names two or three businesses. It is not ranking pages. It is choosing who to quote.
To be quoted, a machine has to be able to state a fact about you with confidence. "Xebia is an IT consultancy in Hilversum that works on cloud and data engineering" is a citable claim. "This company has a nice homepage" is not.
The four signals at the bottom of the table are the ones that produce citable claims:
**Organization / LocalBusiness schema (33.5%)** tells the engine what kind of entity you are, where you operate, and how to reach you, in a format that requires no interpretation. Two thirds of the sample leaves the engine to guess from prose.
**FAQPage schema (0.5%)** maps a question directly to your answer. Answer engines are, structurally, question-answering machines. One site out of 188 has done this.
**/llms.txt (7.4%)** is a plain-text summary at the root of your domain, written for language models rather than people: what you do, your services, your pricing posture, your key pages. It is the single cheapest thing on this list to add and 92.6% of the sample has not.
**og:image (55.9%)** governs whether you appear as a rich card rather than a bare link when your page is surfaced or shared.
The 52 sites that made the most instructive mistake
Here is the detail we found most telling.
**115 sites have JSON-LD structured data. Only 63 of those describe the organisation itself.** The other 52 have structured data that describes their breadcrumbs, their images, their search box — everything except the business.
The most common schema types across the whole sample were `ImageObject` (62 sites), `BreadcrumbList` (61) and `ListItem` (60). `Organization` appeared on 52.
That is what happens when structured data arrives as a side effect of a CMS plugin rather than a decision. The plugin marks up navigation furniture because navigation furniture is easy to detect automatically. Describing the business requires someone to actually write down what the business is.
Those 52 sites are doing more work than the ones with no structured data at all, and getting almost none of the benefit.
What this costs, concretely
When someone asks Claude or ChatGPT "which accountancy firms in Utrecht handle international VAT", the engine assembles an answer from sources it can parse confidently. A firm with Organization schema, a FAQPage entry about international VAT, and an llms.txt naming Utrecht as a service area is straightforwardly citable.
A firm with the same expertise, expressed only in a paragraph of Dutch marketing prose inside a `<div>`, is not unavailable to the engine. It is simply harder to be certain about, and the engine has 188 other options.
You do not get a notification when this happens. There is no impression in Search Console for an answer you were not included in. That is why 92% is possible in the first place: nothing in any analytics tool tells a business owner this is happening.
What to fix, in order
**1. Add /llms.txt.** A plain text file at `yourdomain.com/llms.txt`. What you do, who you serve, your main services, your key URLs, how to contact you. An afternoon of work. 92.6% of your competitors have not done it.
**2. Add Organization or LocalBusiness schema.** Name, address, service area, contact, services offered. If you already have JSON-LD from a plugin, check what it actually describes before assuming you are covered — there is a two-in-five chance it describes your breadcrumbs.
**3. Add FAQPage schema to the questions you already answer.** Most business sites already have an FAQ section written in HTML. Wrapping it in FAQPage markup is a formatting change, not a content project. It is the single most underused signal in the sample.
**4. Then fix the ordinary things.** 31.4% have no sitemap, 21.3% no meta description, 23.4% no H1. These still matter for conventional search.
How we measured this
Every site was fetched live in August 2026. We requested the homepage, `/robots.txt`, `/llms.txt` and `/sitemap.xml` for each domain, parsed the returned HTML for structured data and meta tags, and recorded the result. Sites that did not respond were excluded rather than counted as failures, which is why the sample is 188 rather than the 220 we started with.
Crawler blocking was determined by parsing `robots.txt` for `Disallow: /` directives scoped to named AI user agents (GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot, Google-Extended, anthropic-ai, ChatGPT-User).
These are the same checks that run in our free site audit tool, which scores any domain on 13 signals across AI readiness, technical SEO, schema and content. It takes about ten seconds and needs no sign-up. We ran it on our own site before publishing this: 97 out of 100.
If you want to know where your site sits against these numbers, run it.