Agent-Ready Web Map
How ready is the open web for autonomous agents? For a curated universe of domains we score six public surfaces, an x402 payment manifest, agent.json, an OpenAPI spec, llms.txt, an AI-crawlable robots policy, and ai.txt, into one transparent number from 0 to 100. This is the running tally.
Machine-readable JSON/api/agent-ready/summary.jsonEvery day we profile part of the universe at its public paths and add up the surfaces we find. The aggregate below is live: it polls the same free endpoint agents use, so the human view and the machine view never disagree. Pair it with /ai-crawler-access (the robots.txt layer this map is built on) to see what each domain declares to crawlers.
How the score is computed
The score is a plain sum of six fixed weights. There is no model and no hidden weighting. A domain's number is just the surfaces we found published, added together. Here is the full rubric, the same one the worker applies to every domain.
Four tiers, fixed cutoffs
A score of 20 or below is closed. From 21 through 50 is emerging. From 51 through 80 is ready. Above 80 is advanced. The cutoffs at 20, 50, and 80 never move, so a domain changes tier only when it adds or drops a real surface, not because we retuned anything.
The crawlable rule
A domain earns the 15-point crawlable surface if any core AI bot (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, or CCBot) is allowed or partially allowed in robots.txt. If every core bot is blocked, or robots.txt could not be read, the surface scores zero. Unknown is never counted as crawlable.
OpenAPI two-path heuristic
We probe two conventional locations, /openapi.json and /.well-known/openapi.json, and count the OpenAPI surface present if either returns non-empty content. A spec served at a bespoke path reads as absent. We would rather under-report than guess at a location we did not check.
Rolling fill, honest denominator
The crawl is rolling, so the map seeds over about a week. The summary reports profiled (domains actually scored for agent surfaces) versus domains tracked, and profiled grows toward domains tracked over the first seven days. Adoption percentages are computed only over profiled domains, never over the full universe.
Free agent endpoints
/api/agent-ready/summary.jsonAggregate: per-surface adoption, the readiness-tier distribution, and a top-25 leaderboard. No parameters. Same payload this page renders./api/agent-ready/site?domain=One domain: a transparent 0 to 100 score, tier, and which agent surfaces the site exposes. Required param: domain.
The full per-domain dataset is premium at one credit. See /developers/agent-payments.
Related on TensorFeed
Frequently asked questions
- What does an agent-readiness score mean?
- It is a transparent 0 to 100 sum of six published surfaces a site can expose for autonomous agents. Each surface adds a fixed number of points: an x402 payment manifest is worth 25, agent.json 20, an OpenAPI spec 20, llms.txt 15, being crawlable by a core AI bot 15, and ai.txt 5. There is no model, no opinion, and no weighting we hide. A site's score is just the sum of the surfaces we found published at its public paths.
- How are the tiers defined?
- Four bands by score. A score of 20 or below is closed. From 21 through 50 is emerging. From 51 through 80 is ready. Above 80 is advanced. The cutoffs are fixed at 20, 50, and 80 so a tier never drifts: a domain moves bands only when it actually adds or drops a surface.
- What counts as AI-bot-crawlable?
- We look at the robots.txt verdict for a small set of core AI crawlers: GPTBot, ClaudeBot, PerplexityBot, Google-Extended, and CCBot. If any one of them is allowed or partially allowed at the root, the site earns the crawlable surface. If every core bot is blocked, or robots.txt could not be read at all, the surface scores zero. Unknown is never treated as crawlable.
- How do you detect an OpenAPI spec?
- With a two-path heuristic. We fetch /openapi.json and /.well-known/openapi.json and count the OpenAPI surface present if either one returns non-empty content. It is a deliberately simple probe: it catches the two conventional locations without claiming to validate the spec itself. A site that serves its spec at some bespoke path will read as absent here, which we would rather do than guess.
- Why is coverage filling in gradually?
- The crawl is rolling. Each daily run profiles a slice of the universe, so the full set re-checks roughly weekly and the first snapshot fills in over about a week. The summary reports profiled versus domains tracked precisely so you can watch coverage build: profiled is the number of domains we have actually scored for agent surfaces, and it climbs toward domains tracked as the backfill catches up. We never quietly count an unprofiled domain as ready.
- Is this stated capability or verified behavior?
- Stated, published surfaces. We read what a site publishes at its public paths and score the presence of each surface. We do not test whether the x402 manifest actually settles a payment, whether the OpenAPI endpoints respond, or whether a crawler obeys the robots policy. This is a map of declared agent readiness across the open web, not a conformance certification.
- How do I pull this programmatically?
- Two free endpoints, no auth. GET /api/agent-ready/summary.json for the aggregate (per-surface adoption, the tier distribution, and a top-25 leaderboard), and GET /api/agent-ready/site?domain=example.com for a single domain score and surface breakdown. The full per-domain dataset is premium at one credit, with an eight-day freshness SLA and no charge when the snapshot is stale.