Bot and AI crawler control
LivréWeb· v0.3.29· 17 nov. 2026
Per-website rate limits and block lists for crawlers, including AI bots, with a report of who is crawling what.
Per-website rate limits and block lists for crawlers, including AI bots, with a report of who is crawling what.