Bot and AI crawler control
أُطلقWeb· v0.3.29· 17 نوفمبر 2026
Per-website rate limits and block lists for crawlers, including AI bots, with a report of who is crawling what.
Per-website rate limits and block lists for crawlers, including AI bots, with a report of who is crawling what.