machinewitness

Monthly figures · September 2026

AI reservations in the EU, September 2026

Between 15 September 2026 and 22 September 2026 this archive read the robots.txt of 128,316 domains with an EU ending, the ones that appear among the million most-visited domains worldwide. 92,616 of them served a readable file. 10.6 % of those reserved against at least one AI crawler: they disallowed it completely, or declared ai-train=no.

These figures count. They do not rate anyone, and they say nothing about whether any crawler follows what a file says.

10.6 %reserve against at least one AI crawler
26.3 %name an AI crawler in their robots.txt
12.4 %name one in a file of their own

By ending

Share of domains with a readable robots.txt that reserve against at least one AI crawler. Endings with at least 1,000 readable files; all endings are in the table further down. Hover a bar for the figures.

in the domain's own filethrough a template shared by 50 or more domains
0%5%10%15%20%.cz: 17.0 % reserve (12.4 % in their own file, 4.7 % through a shared template), 3,678 readable robots.txt.cz17.0%.sk: 16.8 % reserve (16.5 % in their own file, 0.4 % through a shared template), 1,342 readable robots.txt.sk16.8%.pl: 14.5 % reserve (12.8 % in their own file, 1.7 % through a shared template), 6,536 readable robots.txt.pl14.5%.fi: 13.8 % reserve (13.3 % in their own file, 0.5 % through a shared template), 1,642 readable robots.txt.fi13.8%.dk: 13.1 % reserve (12.4 % in their own file, 0.7 % through a shared template), 1,774 readable robots.txt.dk13.1%.ro: 13.0 % reserve (10.1 % in their own file, 2.9 % through a shared template), 2,792 readable robots.txt.ro13.0%.pt: 12.6 % reserve (11.4 % in their own file, 1.2 % through a shared template), 1,358 readable robots.txt.pt12.6%.es: 12.5 % reserve (11.9 % in their own file, 0.7 % through a shared template), 3,594 readable robots.txt.es12.5%.be: 12.3 % reserve (11.2 % in their own file, 1.1 % through a shared template), 2,157 readable robots.txt.be12.3%.at: 11.8 % reserve (10.6 % in their own file, 1.2 % through a shared template), 2,164 readable robots.txt.at11.8%.se: 11.8 % reserve (8.9 % in their own file, 2.8 % through a shared template), 2,785 readable robots.txt.se11.8%.fr: 11.6 % reserve (10.7 % in their own file, 0.9 % through a shared template), 7,175 readable robots.txt.fr11.6%.it: 11.3 % reserve (9.3 % in their own file, 2.0 % through a shared template), 6,130 readable robots.txt.it11.3%.gr: 10.7 % reserve (8.5 % in their own file, 2.2 % through a shared template), 2,410 readable robots.txt.gr10.7%.hu: 10.6 % reserve (9.8 % in their own file, 0.7 % through a shared template), 2,001 readable robots.txt.hu10.6%.de: 10.2 % reserve (9.9 % in their own file, 0.3 % through a shared template), 19,861 readable robots.txt.de10.2%.eu: 9.7 % reserve (7.9 % in their own file, 1.8 % through a shared template), 3,293 readable robots.txt.eu9.7%.nl: 4.8 % reserve (4.5 % in their own file, 0.3 % through a shared template), 16,498 readable robots.txt.nl4.8%

Named is not the same as decided

26.3 % of the files name at least one AI crawler. Counted only in files that are not shared with 50 or more other domains, it is 12.4 %. The difference comes from templates: a hosting provider or a shop system writes the same robots.txt for many sites, and some of those templates list AI crawlers, sometimes only to allow them. For reservations the gap is smaller: 9.5 % of all readable files reserve in a file of their own, 1.1 points come through a template.

This month 62 rule sets were shared by 50 or more domains. 39.1 % of all readable files belong to one of them.

How these figures are made

  1. Which domains. The Tranco list of the million most-visited domains worldwide, filtered to the endings of the 27 EU member states and .eu, plus the domains of the core ring. An ending is not the country of the operator.
  2. What is read. The robots.txt of each domain, fetched daily by two independent witnesses and sealed every night (how it works).
  3. Which file counts. The latest robots.txt with status 200 in a seven-day window, www and the bare domain taken together.
  4. What is classified. Whether the file names one of 26 AI crawlers, whether one of them is disallowed completely (Disallow: / in its group), and whether a Content-Signal line declares ai-train=no. A block of all crawlers is not counted, because it is not about AI.
  5. Templates. The file without sitemap, host and comment lines is its rule set. A rule set shared by 50 or more domains counts as a template.
  6. The figure. Shares are of domains with a readable robots.txt. Method version 1; a change of method is announced and never mixed into a series.

Not counted in this version: ai.txt, TDM reservations in headers or in tdmrep.json, and terms of use. What each day holds is described on what we store.

All endings

EndingDomainsReadable robots.txtReserveof which own fileName an AI crawlerai-train=no
.de27,92819,86110.2 %9.9 %21.9 %0.5 %
.nl18,85516,4984.8 %4.5 %64.1 %0.3 %
.fr11,0207,17511.6 %10.7 %17.0 %0.7 %
.it10,1186,13011.3 %9.3 %16.7 %0.8 %
.pl8,9496,53614.5 %12.8 %18.0 %1.5 %
.eu5,6163,2939.7 %7.9 %12.3 %1.2 %
.es5,4033,59412.5 %11.9 %17.4 %1.1 %
.cz4,8913,67817.0 %12.4 %17.3 %6.5 %
.ro3,8352,79213.0 %10.1 %20.7 %0.9 %
.se3,5292,78511.8 %8.9 %26.4 %0.6 %
.gr3,3382,41010.7 %8.5 %14.3 %0.9 %
.be3,1872,15712.3 %11.2 %15.3 %0.7 %
.at2,9572,16411.8 %10.6 %15.8 %0.7 %
.hu2,8922,00110.6 %9.8 %13.3 %1.2 %
.dk2,3271,77413.1 %12.4 %16.7 %1.5 %
.fi2,2211,64213.8 %13.3 %15.8 %0.7 %
.pt1,9851,35812.6 %11.4 %18.3 %1.7 %
.sk1,7981,34216.8 %16.5 %16.5 %5.0 %
.ie1,39899513.2 %13.2 %17.6 %0.4 %
.bg1,1608899.3 %8.5 %12.6 %1.3 %
.hr1,07580510.7 %10.3 %13.9 %1.2 %
.lt1,0067139.5 %9.5 %12.9 %0.8 %
.lv8485998.2 %8.2 %10.5 %0.7 %
.si7385379.1 %9.1 %13.0 %1.1 %
.ee7225439.6 %9.4 %12.7 %1.3 %
.lu25517613.6 %13.6 %16.5 %1.7 %
.cy17410111.9 %11.9 %20.8 %1.0 %
.mt91688.8 %8.8 %10.3 %1.5 %

The figures of this issue as data: 2026-09.json.

Using these figures

MachineWitness, “AI reservations in the EU, September 2026”, 22 September 2026, https://machinewitness.eu/blog/ai-reservations-2026-09

Free to quote with that source. For a figure that is not on this page, for one ending, one month or one question of your own, write to contact@machinewitness.eu. We answer in German, English or Spanish.

We do not advise. What a reservation means in law, and what anyone should put in their own files, is a question for a lawyer.

All monthly figures