robots-txt-parser
x402Grade CLast checked 4 hours ago
POST https://intel.rallylive.ca/site/robots
robots.txt fetched and parsed: every user-agent group with its allow/disallow rules and crawl-delay, declared sitemaps, and whether the file exists at all. Understand a site's crawl policy before fetching it. $0.01 per site.
Price$0.01per page fetched · paid in USD on Base
Uptime100%
4/4 probes · 830 ms
Score59Grade C
Paid usage, 30 days2payers · 2 calls
Warnings
Wallet mismatch The payout wallet in its live payment answer differs from its catalogue listing on the same network.
How to buy
This endpoint takes pay-per-call payments over x402. Call it once without paying to see its live price; any x402 client then pays and retries automatically.
curl -i -X POST "https://intel.rallylive.ca/site/robots"
# HTTP/1.1 402 Payment Required: price, asset and payTo are in the responseRecent checks
| Checked (UTC) | Result | Status | Time | Live price |
|---|---|---|---|---|
| 2026-10-11 10:16 | Valid 402 | 402 | 628 ms | $0.01 |
| 2026-10-11 06:00 | Valid 402 | 402 | 935 ms | $0.01 |
| 2026-10-11 01:30 | Valid 402 | 402 | 725 ms | $0.01 |
| 2026-10-10 21:00 | Valid 402 | 402 | 954 ms | $0.01 |
Receipts
No test purchases yet. Graded test purchases are next on our list; until then every number here comes from unpaid checks.
What a check proves