robots-allowed-for-bot
x402Grade BLast checked 4 hours ago
POST https://intel.rallylive.ca/dev/robots-for-bot
Is a specific bot allowed to fetch a URL? Fetches the site's robots.txt and evaluates the path for the named user-agent (GPTBot, ClaudeBot, Googlebot, Bingbot, CCBot, PerplexityBot or any token), returning allowed/disallowed, the matching rule and group, crawl-delay, and whether the bot has a dedicated group or falls back to *. Pre-flight for compliant scraping agents. $0.01 per check.
Price$0.01per page fetched · paid in USD on Base
Uptime100%
4/4 probes · 415 ms
Score70Grade B
Paid usage, 30 days0payers · 0 calls
How to buy
This endpoint takes pay-per-call payments over x402. Call it once without paying to see its live price; any x402 client then pays and retries automatically.
curl -i -X POST "https://intel.rallylive.ca/dev/robots-for-bot"
# HTTP/1.1 402 Payment Required: price, asset and payTo are in the responseRecent checks
| Checked (UTC) | Result | Status | Time | Live price |
|---|---|---|---|---|
| 2026-10-11 10:46 | Valid 402 | 402 | 298 ms | $0.01 |
| 2026-10-11 06:15 | Valid 402 | 402 | 483 ms | $0.01 |
| 2026-10-11 01:45 | Valid 402 | 402 | 347 ms | $0.01 |
| 2026-10-10 21:15 | Valid 402 | 402 | 580 ms | $0.01 |
Receipts
No test purchases yet. Graded test purchases are next on our list; until then every number here comes from unpaid checks.
What a check proves