Vextorium
x402Grade BLast checked 2 hours ago
POST https://api.vextorium.com/v1/robots-txt
Parse a site's robots.txt: whether a user-agent may crawl a path, which AI crawlers (GPTBot, ClaudeBot, Google-Extended, PerplexityBot, CCBot...) are blocked, and declared sitemaps. Check before your agent scrapes.
Price$0.003per page fetched · paid in USD on Base
Uptime100%
4/4 probes · 917.5 ms
Score73Grade B
Paid usage, 30 days2payers · 2 calls
How to buy
This endpoint takes pay-per-call payments over x402. Call it once without paying to see its live price; any x402 client then pays and retries automatically.
curl -i -X POST "https://api.vextorium.com/v1/robots-txt"
# HTTP/1.1 402 Payment Required: price, asset and payTo are in the responseRecent checks
| Checked (UTC) | Result | Status | Time | Live price |
|---|---|---|---|---|
| 2026-10-11 08:46 | Valid 402 | 402 | 953 ms | $0.003 |
| 2026-10-11 04:30 | Valid 402 | 402 | 882 ms | $0.003 |
| 2026-10-11 00:00 | Valid 402 | 402 | 749 ms | $0.003 |
| 2026-10-10 19:15 | Valid 402 | 402 | 1057 ms | $0.003 |
Receipts
No test purchases yet. Graded test purchases are next on our list; until then every number here comes from unpaid checks.
What a check proves