Site crawl (pages to markdown)
x402Grade CLast checked 53 min ago
POST https://agent402.tools/api/site-crawl
Crawl a website from a starting URL and return each page as clean markdown: breadth-first over internal links, bounded by page count and depth, honouring robots.txt, with per-page title, status, depth and outbound links. Use it when an agent needs a whole section of a site rather than one known page.
Price$0.02per page fetched · paid in USD on Solana
Uptime100%
4/4 probes · 130.5 ms
Score63Grade C
Paid usage, 30 days0payers · 0 calls
Warnings
Wallet mismatch The payout wallet in its live payment answer differs from its catalogue listing on the same network.
How to buy
This endpoint takes pay-per-call payments over x402. Call it once without paying to see its live price; any x402 client then pays and retries automatically.
curl -i -X POST "https://agent402.tools/api/site-crawl"
# HTTP/1.1 402 Payment Required: price, asset and payTo are in the responseRecent checks
| Checked (UTC) | Result | Status | Time | Live price |
|---|---|---|---|---|
| 2026-10-11 12:46 | Valid 402 | 402 | 66 ms | $0.02 |
| 2026-10-11 08:30 | Valid 402 | 402 | 367 ms | $0.02 |
| 2026-10-11 04:00 | Valid 402 | 402 | 83 ms | $0.02 |
| 2026-10-10 23:30 | Valid 402 | 402 | 152 ms | $0.02 |
| 2026-10-10 18:15 | Valid 402 | 402 | 109 ms | $0.02 |
Receipts
No test purchases yet. Graded test purchases are next on our list; until then every number here comes from unpaid checks.
What a check proves