aeo-crawlers
x402Grade BLast checked 4 hours ago
GET https://payai.agentstools.dev/aeo/crawlers
robots.txt AI-crawler access matrix: for the 2026 cohort of AI and answer-engine crawlers (GPTBot, ChatGPT-User, OAI-SearchBot, ClaudeBot, Claude-User, PerplexityBot, Google-Extended and more) reports allowed, blocked or unlisted per bot, whether the default rule blocks everything, declared sitemaps and a score. One network operation.
Price$0.005per page fetched · paid in USD on Solana
Uptime100%
3/3 probes · 514 ms
Score70Grade B
Paid usage, 30 days0payers · 0 calls
How to buy
This endpoint takes pay-per-call payments over x402. Call it once without paying to see its live price; any x402 client then pays and retries automatically.
curl -i -X GET "https://payai.agentstools.dev/aeo/crawlers"
# HTTP/1.1 402 Payment Required: price, asset and payTo are in the responseRecent checks
| Checked (UTC) | Result | Status | Time | Live price |
|---|---|---|---|---|
| 2026-10-11 07:30 | Valid 402 | 402 | 3466 ms | $0.005 |
| 2026-10-11 03:00 | Valid 402 | 402 | 514 ms | $0.005 |
| 2026-10-10 22:31 | Valid 402 | 402 | 504 ms | $0.005 |
Receipts
No test purchases yet. Graded test purchases are next on our list; until then every number here comes from unpaid checks.
What a check proves