Live AI bot traffic
Which AI agents and crawlers are pulling data from TensorFeed right now. Most sites hide this; we publish it because TensorFeed.ai was built for AI agents.
/api/agents/activity, refreshed every 30s.Bot breakdown (recent 50)
No bot hits in the recent buffer yet.
Top endpoints hit
No endpoint hits yet.
Live tail
Waiting for the next bot hit.
We keep the most recent 50 bot hits in a rolling buffer plus a daily counter (resets at 00:00 UTC). Coverage spans both the API/feed surface (via the Cloudflare Worker) and every static editorial / SEO route (via a Pages Functions middleware that pings the Worker on detection). Daily totals are also captured into the public history snapshot at /api/history so multi-day trends become readable as the archive accumulates.
Pull this data yourself
curl -s https://tensorfeed.ai/api/agents/activity | jq
# or via the SDKs:
# Python: tf.agents_activity() (pip install tensorfeed)
# TypeScript: client.agents.activity() (npm install tensorfeed)
# MCP: no separate tool yet, /api/agents/activity is free + unauthenticatedFAQ
Which AI bots does TensorFeed track?
We track 25+ user-agent patterns including ClaudeBot and anthropic-ai (Anthropic), GPTBot, ChatGPT-User, and OAI-SearchBot (OpenAI), PerplexityBot, Google-Extended and Googlebot, Bingbot, Applebot, DuckDuckBot, Bytespider (TikTok / Doubao), Amazonbot, FacebookBot, Twitterbot, cohere-ai, YouBot, plus generic patterns for Scrapy, python-requests, axios, node-fetch, and any UA matching bot/crawler/spider/agent.
How is bot traffic detected?
Pure user-agent string matching, in two places. The Cloudflare Worker matches on every API/feed request; a Cloudflare Pages Functions middleware matches on every static route request (/originals/*, /api-reference/*, /for-ai-agents, etc) and forwards a fire-and-forget ping to the Worker so both stream into the same in-memory buffer. We do not fingerprint, rate-limit, or block any of these bots. Hits flush to KV in batches to stay inside the 100k operations/day budget on the Cloudflare free tier.
How often does the dashboard refresh?
The page polls /api/agents/activity every 30 seconds. The endpoint itself caches its KV reads in worker memory for 30 seconds, so the freshest visible state lags real-time hits by up to a minute.
How far back does this data go?
The live view shows the most recent 50 bot hits and the running daily counter. Historical daily counts are also captured into our public history snapshot system at /api/history once per day, so multi-day trends become available as the archive accumulates.
Why does TensorFeed publish this?
Two reasons. First, most sites hide bot traffic; we built TensorFeed for AI agents, so making the agent footprint visible is on-brand. Second, agent traffic patterns themselves are useful data: which crawler indexes which surface, which endpoints attract the most agent attention, and how that distribution shifts over time.
Can I get this data programmatically?
Yes. GET https://tensorfeed.ai/api/agents/activity returns JSON with today_count, last_updated, and the recent hits array. No auth, no rate limit. Both SDKs (pip install tensorfeed, npm install tensorfeed) expose it via tf.agents_activity() / client.agents.activity().
Related
/for-ai-agents
Discovery, MCP, x402 payments, SDKs. The full agent-first overview.
/api-reference
Per-endpoint reference. Free + premium, with code samples and JSON-LD.
Why we publish this dashboard
The editorial argument: most sites hide bot traffic. We count and publish.
/api/history
Daily snapshots of agent-activity, pricing, models, benchmarks, and status.