Agent Extraction API
GET https://freetier-agent-api.pages.dev/api/robots-check
Listed in PayAI
Check whether a URL may be crawled per the site's robots.txt for a given user-agent. Returns allowed, matched rule, crawl-delay, sitemaps.
Other endpoints from freetier-agent-api.pages.dev
The same service's other paid functions that Hagglyai tracks, each with its own price. Functions outside Hagglyai's categories are not listed.
| Endpoint | Category | Price per call |
|---|---|---|
| GET /api/currency | Stocks & forex | $0.002 |
| GET /api/llm-cost | LLM inference | $0.005 |
| GET /api/page-rag | Web page extraction | $0.001 |
How to buy
This endpoint takes pay-per-call payments over x402. Call it once without paying to see its live price; any x402 client then pays and retries automatically.
curl -i -X GET "https://freetier-agent-api.pages.dev/api/robots-check"
# HTTP/1.1 402 Payment Required: price, asset and payTo are in the responseEstimated calls per day
Paid calls to freetier-agent-api.pages.dev's tracked endpoints each day, estimated from the 30-day totals the x402 catalogues report. Each bar is the change in that 30-day total since the day before, plus the calls that left the 30-day window (taken as an average day). It is an estimate: Hagglyai does not measure or pay for these calls.
No usage reported for any of 4 tracked endpoints. Unknown usage is left out, not counted as zero.
Compared with other providers
Coverage matters: a provider whose endpoints report more of their usage can rank higher.
| # | Provider | Reported 30-day calls | Coverage |
|---|---|---|---|
| 1 | stableenrich.dev | 42,340 | 15 of 31 endpoints |
| 2 | win.oneshotagent.com | 13,383 | 14 of 27 endpoints |
| 3 | agent.pocket.network | 7,244 | 14 of 25 endpoints |
| 4 | x402.orthogonal.com | 5,503 | 202 of 268 endpoints |
| 5 | market.datapackvibe.com | 5,236 | 3365 of 6278 endpoints |
| 6 | agent402.tools | 3,805 | 146 of 232 endpoints |
| 7 | api.clawfetch.ai | 2,973 | 4 of 8 endpoints |
| 8 | api.kadec0.xyz | 1,630 | 12 of 12 endpoints |