GET /api/crawler-index

Public list of recognised AI crawlers. PRs welcome.

Use it

Generate robots.txt, document which bots you allow, or diff against what ai_traffic actually observed. See AI crawlers.