Robots.txt check
GET https://agent-data-tools.revmesh2074.workers.dev/robots
Listed in Coinbase x402 Bazaar
Robots.txt check. Read a site's robots.txt and say whether a crawler may fetch a URL, by RFC 9309 rules as Google applies them (most specific user-agent group, longest matching Allow or Disallow, Allow on ties, * and $ wildcards). Returns the deciding rule, the group used, the rule count and the sitemaps listed. A missing robots.txt (4xx) allows everything. Use before scraping or crawling a site politely; for the sitemap URLs themselves use sitemap.
Other endpoints from agent-data-tools.revmesh2074.workers.dev
The same service's other paid functions that Hagglyai tracks, each with its own price. Functions outside Hagglyai's categories are not listed. Showing 50 of 87.
| Endpoint | Category | Price per call |
|---|---|---|
| GET /ai/chat | LLM inference | $0.002 |
| GET /ai/chat-large | LLM inference | $0.01 |
| GET /ai/embed | LLM inference | $0.01 |
| GET /ai/translate | Translation | $0.01 |
| GET /airport/status | Flights & travel | $0.01 |
| GET /arb/balance | On-chain data | $0.01 |
| GET /arb/block | On-chain data | $0.01 |
| GET /arb/call | On-chain data | $0.01 |
| GET /arb/estimate-gas | On-chain data | $0.01 |
| GET /arb/gas | On-chain data | $0.01 |
| GET /arb/gas-history | On-chain data | $0.01 |
| GET /arb/logs | On-chain data | $0.01 |
| GET /arb/receipt | On-chain data | $0.01 |
| GET /arb/storage | On-chain data | $0.01 |
| GET /arb/token-balance | On-chain data | $0.01 |
| GET /base-tx | On-chain data | $0.003 |
| GET /base/balance | On-chain data | $0.01 |
| GET /base/block | On-chain data | $0.01 |
| GET /base/call | On-chain data | $0.01 |
| GET /base/estimate-gas | On-chain data | $0.01 |
| GET /base/gas-history | On-chain data | $0.01 |
| GET /base/logs | On-chain data | $0.01 |
| GET /base/receipt | On-chain data | $0.01 |
| GET /base/storage | On-chain data | $0.01 |
| GET /base/token-balance | On-chain data | $0.01 |
| GET /btc-fees | On-chain data | $0.002 |
| GET /btc-price | Crypto prices & market data | $0.01 |
| GET /company | People & company enrichment | $0.01 |
| GET /date/zodiac | People & company enrichment | $0.01 |
| GET /dictionary | Translation | $0.01 |
| GET /eth/balance | On-chain data | $0.01 |
| GET /eth/block | On-chain data | $0.01 |
| GET /eth/call | On-chain data | $0.01 |
| GET /eth/estimate-gas | On-chain data | $0.01 |
| GET /eth/gas | On-chain data | $0.01 |
| GET /eth/gas-history | On-chain data | $0.01 |
| GET /eth/logs | On-chain data | $0.01 |
| GET /eth/nonce | On-chain data | $0.01 |
| GET /eth/receipt | On-chain data | $0.01 |
| GET /eth/storage | On-chain data | $0.01 |
| GET /eth/token-balance | On-chain data | $0.01 |
| GET /eth/tx | On-chain data | $0.01 |
| GET /evm/decode-calldata | On-chain data | $0.01 |
| GET /extract | Web page extraction | $0.002 |
| GET /extract/rendered | Web page extraction | $0.01 |
| GET /fx | Stocks & forex | $0.01 |
| GET /geocode | Weather | $0.01 |
| GET /geocode/reverse | Maps & places | $0.01 |
| GET /html/to-text | Web page extraction | $0.01 |
| GET /hyperliquid/candles | Crypto prices & market data | $0.01 |
How to buy
This endpoint takes pay-per-call payments over x402. Call it once without paying to see its live price; any x402 client then pays and retries automatically.
curl -i -X GET "https://agent-data-tools.revmesh2074.workers.dev/robots"
# HTTP/1.1 402 Payment Required: price, asset and payTo are in the responseEstimated calls per day
Paid calls to agent-data-tools.revmesh2074.workers.dev's tracked endpoints each day, estimated from the 30-day totals the x402 catalogues report. Each bar is the change in that 30-day total since the day before, plus the calls that left the 30-day window (taken as an average day). It is an estimate: Hagglyai does not measure or pay for these calls.
Usage available for 55 of 88 tracked endpoints, 173 paid calls in the last 30 days. Last updated 7 hours ago. Unknown usage is left out, not counted as zero.
Compared with other providers
agent-data-tools.revmesh2074.workers.dev ranks 32nd of 286 in Web page extraction by its latest reported 30-day paid calls. Coverage matters: a provider whose endpoints report more of their usage can rank higher.
| # | Provider | Reported 30-day calls | Coverage |
|---|---|---|---|
| 1 | stableenrich.dev | 30,700 | 15 of 31 endpoints |
| 2 | win.oneshotagent.com | 14,617 | 13 of 26 endpoints |
| 3 | agent.pocket.network | 6,466 | 14 of 25 endpoints |
| 4 | x402.orthogonal.com | 5,540 | 202 of 267 endpoints |
| 5 | agent402.tools | 3,644 | 146 of 232 endpoints |
| 6 | api.clawfetch.ai | 2,812 | 4 of 8 endpoints |
| 7 | api.kadec0.xyz | 1,487 | 12 of 12 endpoints |
| 8 | api.losbeto.xyz | 1,432 | 21 of 109 endpoints |
| 32 | agent-data-tools.revmesh2074.workers.dev (this one) | 173 | 55 of 88 endpoints |