Per-website rate limits and block lists for crawlers, including AI bots, with a report of who is crawling what.