Robots.txt & Sitemap
GET https://bess-x402.onrender.com/v1/web/robots
Listed in Coinbase x402 Bazaar
A site's robots.txt rules per user agent, whether AI crawlers (GPTBot, ClaudeBot, Google-Extended, CCBot, PerplexityBot) are blocked, and the URLs listed in its sitemap. Use before crawling a site or for SEO audits.
Other endpoints from bess-x402.onrender.com
The same service's other paid functions that Hagglyai tracks, each with its own price. Functions outside Hagglyai's categories are not listed.
| Endpoint | Category | Price per call |
|---|---|---|
| GET /v1/apps/search | Web search | $0.005 |
| GET /v1/crypto/balance | On-chain data | $0.003 |
| GET /v1/crypto/btc-fees | On-chain data | $0.002 |
| GET /v1/crypto/candles | Crypto prices & market data | $0.002 |
| GET /v1/crypto/coin | Crypto prices & market data | $0.003 |
| GET /v1/crypto/dex-pairs | Crypto prices & market data | $0.003 |
| GET /v1/crypto/news | News | $0.003 |
| GET /v1/crypto/price | Crypto prices & market data | $0.002 |
| GET /v1/crypto/price-history | Crypto prices & market data | $0.003 |
| GET /v1/crypto/protocols | Crypto prices & market data | $0.003 |
| GET /v1/crypto/solana-balance | On-chain data | $0.003 |
| GET /v1/crypto/ticker | Crypto prices & market data | $0.002 |
| GET /v1/crypto/top-coins | Crypto prices & market data | $0.002 |
| GET /v1/crypto/trending | Crypto prices & market data | $0.002 |
| GET /v1/finance/fx | Stocks & forex | $0.002 |
| GET /v1/geo/weather | Weather | $0.002 |
| GET /v1/lookup/food | Products & prices | $0.002 |
| GET /v1/news/hackernews | News | $0.002 |
| GET /v1/predictions/kalshi/search | Crypto prices & market data | $0.003 |
| GET /v1/text/language | Translation | $0.002 |
| GET /v1/us-stocks/filings | Stocks & forex | $0.003 |
| GET /v1/us-stocks/full-text-search | Stocks & forex | $0.004 |
| GET /v1/us-stocks/fundamentals | Stocks & forex | $0.005 |
| GET /v1/us-stocks/insiders | Stocks & forex | $0.005 |
| GET /v1/us-stocks/profile | Stocks & forex | $0.003 |
| GET /v1/us-stocks/search | People & company enrichment | $0.002 |
| GET /v1/util/holidays | Stocks & forex | $0.002 |
| GET /v1/web/contacts | People & company enrichment | $0.005 |
| GET /v1/web/links | Web page extraction | $0.002 |
| GET /v1/web/markdown | Web page extraction | $0.005 |
How to buy
This endpoint takes pay-per-call payments over x402. Call it once without paying to see its live price; any x402 client then pays and retries automatically.
curl -i -X GET "https://bess-x402.onrender.com/v1/web/robots"
# HTTP/1.1 402 Payment Required: price, asset and payTo are in the responseEstimated calls per day
Paid calls to bess-x402.onrender.com's tracked endpoints each day, estimated from the 30-day totals the x402 catalogues report. Each bar is the change in that 30-day total since the day before, plus the calls that left the 30-day window (taken as an average day). It is an estimate: Hagglyai does not measure or pay for these calls.
Usage available for 10 of 31 tracked endpoints, 10 paid calls in the last 30 days. Last updated 2 hours ago. Unknown usage is left out, not counted as zero.
Compared with other providers
bess-x402.onrender.com ranks 136th of 267 in Web page extraction by its latest reported 30-day paid calls. Coverage matters: a provider whose endpoints report more of their usage can rank higher.
| # | Provider | Reported 30-day calls | Coverage |
|---|---|---|---|
| 1 | stableenrich.dev | 29,486 | 15 of 31 endpoints |
| 2 | win.oneshotagent.com | 15,070 | 13 of 29 endpoints |
| 3 | x402.orthogonal.com | 5,363 | 194 of 256 endpoints |
| 4 | agent.pocket.network | 5,160 | 14 of 25 endpoints |
| 5 | agent402.tools | 3,745 | 147 of 231 endpoints |
| 6 | x402.ottoai.services | 1,508 | 28 of 32 endpoints |
| 7 | api.kadec0.xyz | 1,432 | 12 of 12 endpoints |
| 8 | api.losbeto.xyz | 1,338 | 21 of 108 endpoints |
| 136 | bess-x402.onrender.com (this one) | 10 | 10 of 31 endpoints |