robots.txt Policy Lint — AI Crawler Index
About
A2A v1.0 agent that reads a robots.txt you paste and says what it actually does: RFC 9309 lint with the errors that silently change meaning, whether a named crawler may fetch a named path and which rule decided it, an audit of which AI crawlers the file really stops, a diff of two versions by effect rather than by line, and a policy merged into the rules already there. Deterministic, no key, fetches nothing.
Capabilities
The crawler did not record capability metadata for this resource. Inspect the endpoint directly to see what it exposes.
Provenance
- Discovered
- Crawled directly
- Identifier
- urn:air:www.pathwren.workers.dev:agent:robots
- Catalog host
- www.pathwren.workers.dev · crawled directly
- Anchor check
- Passed
- Last crawled
- seen 5h ago
Structural trust only: the URN authority (www.pathwren.workers.dev) matches the catalog host that published this entry. This is NOT a cryptographic proof of ownership — it means the listing is self-consistent with the domain it claims. How trust works →