Scanning authorization

Last updated November 1, 2026

This page explains how the Cleared for Agents scanner works and what you are agreeing to when you submit a domain. It exists so both sides know the rules before any request is made.

What the scanner does

• Fetches publicly reachable pages and files over HTTPS.
• Sends a documented user agent: ClearedForAgentsBot/1.0 (+https://clearedforagents.com).
• Reads HTML, robots.txt, sitemaps, markdown files, OpenAPI documents, and well-known JSON files.
• Reads agent protocol surfaces when your site publishes them: A2A agent cards, x402 payment discovery files, and MCP server cards. These checks activate only when the file exists, so a site that serves none of them is not marked down for it.
• Makes a small number of requests per scan, comparable to a page view.

What the scanner never does

• Does not authenticate, log in, or use credentials.
• Does not submit forms or trigger actions.
• Does not probe administrative or private endpoints.
• Does not run load tests or flood requests.
• Does not execute payments or call metered API routes.
• Does not call MCP tools and does not send task requests to agents.
• Does not store page content beyond what a finding requires.

What a paid audit may add, with authorization

After you confirm authorization at intake, the paid audit may send two additional probes to surfaces your site already declares. First, one unpaid request to an x402 endpoint you list, to confirm it answers with a valid payment challenge; the request is never paid. Second, one MCP initialize and tools/list handshake to your MCP endpoint, which reads the list of tool definitions and calls no tool. Both probes target your own declared surfaces only, and both appear in the report evidence with their observed responses.

When you submit a domain

Submitting a URL to the free scanner or paying for an audit confirms that you own the domain or are authorized by its owner to commission scanning. That authorization covers the public-surface checks described on this page.

If you are an agency or consultant, get the client's OK before submitting their domain. The authorization statement at checkout is a real representation, not a formality.

If you are a site owner who does not want scanning

Block the crawler in robots.txt:

User-agent: ClearedForAgentsBot
Disallow: /

Or email hello@clearedforagents.com with the domain and we will exclude it from all scanning, free and paid. We honor both. If a paid audit was already bought for a blocked domain, it is refunded in full.

Respect for existing rules

The paid audit checks whether a site publishes AI bot rules, but the scanner itself follows the authorization described here regardless of what a site's robots.txt says about it. Explicit permission from the domain owner, given at checkout, is the operative authorization for a paid audit. If robots.txt disallows our crawler, mention it before checkout so scope can be agreed in writing.

Report abuse

If you believe the scanner touched your site outside these rules, email hello@clearedforagents.com with the timestamp and your domain. It gets investigated, not argued with.