The agent traffic explosion, measured
Bots became the majority of the web traffic Cloudflare measures, ahead of forecast. Projections for agent query volume run to 1000x. Here is what the data actually supports.
Skip Everling6 min read
In June 2026, automated traffic crossed a line: Cloudflare Radar data showed bots generating 57.5% of the HTML requests it measures, the first bot majority in Cloudflare's numbers . CEO Matthew Prince had forecast the crossover for 2027. It beat his forecast with at least half a year to spare.
The growth behind the milestone is steeper than the milestone. HUMAN Security's 2026 State of AI Traffic report , built on more than a quadrillion analyzed interactions, measured agentic traffic growing 7,851% year over year through 2025. That percentage owes plenty to a small starting base. The direction does not: automated traffic overall, a base that is anything but small, is expanding roughly eight times faster than human activity. The projections stacked on numbers like these run to 1000x. You do not need to believe them. The pressure on retrieval infrastructure arrives far earlier: the per-query capacity penalty is already measurable, before any growth in agent volume.
The loudest projection is not hedged. When Exa announced its Series C in May, CEO Will Bryk wrote that "AI agents will search the web more than humans this year," and separately that within a few years LLM-driven searches will run "1000x more than Google searches today."
A thousand-fold projection deserves scrutiny. So does the evidence ladder underneath it: bot share of web requests, growth of one vendor's agentic category, search calls per task, and serving capacity per query mix are four measurements with four different denominators. They point the same direction. They do not chain into one curve. Here is what each rung can and cannot carry.
The aggregate figures blur three kinds of automation: crawling, fetching on a user's behalf, and agents searching to complete a task. Cloudflare's July 2026 breakdown puts training at 52% of identified crawler requests, with mixed-use crawlers that blend training, search, and agent use at more than 36%; pure search is a small and declining share. A model vendor crawling the web for training data is not an agent retrieving to act. Stop there and the story is just crawlers.
Two reasons not to stop there.
First, the growth is concentrated in the slice that matters. The 7,851% figure is specifically agentic traffic, the fastest-growing category HUMAN measured, and the slice closest to search-user behavior: software browsing, comparing, and transacting on behalf of a task rather than vacuuming pages for a training run.
Second, the 1000x claim is not about web traffic at all. It is about queries. Bryk's projection is a statement about how often reasoning systems will need to look something up, and on that question we have direct evidence from our own published traces: an agent answering one hard research question issues a median of 24 search calls, with the 90th percentile at 35. In this benchmark, one question fans out into dozens of searches behind the scenes. Production fan-out will vary by task, deployment scale will set the total volume, and where that nets out is Bryk's forecast to defend, not ours. The claim this post stands behind is smaller: the per-query penalty does not wait for 1000x. It is measurable now.
Agent queries also differ in kind. Watching GPT-5 work a corpus of 100,195 web pages, the median query ran 10 terms, past the 99th percentile of human query length in the AOL log (twenty years old, but still the best large public baseline available). Phrase-quote operators appeared in 98% of sessions. Filters, year constraints, iterative refinement across a session: agents search like the power users almost no human ever becomes. And they do not skim a results page; they read what comes back and act on it.
Volume times shape is what changes the economics. We measured what that query mix does to serving capacity on one system: hardware that handled 3,236 human keyword queries per second at a 500 ms p99 handled 384 agentic queries at the same p99, an 8.4x spread on the same index. A human types two words and scans ten links. An agent sends ten terms, pulls whole documents, and comes back roughly two dozen times before it is done.
The workload, measured: BrowseComp-Plus traces, and serving capacity on Hornet over a 100M-document index (single Graviton4 instance, 500 ms p99).
Retrieval infrastructure tuned for human behavior fails against this workload without erroring. The engine that comfortably served two-word queries at human pace still returns results. It just gets expensive: in our test, serving the same query volume at the same latency target would take roughly 8.4x the hardware for the agent mix, before any growth in agent count, and volume growth compounds on top. That is one benchmark on one system, and it is exactly the capacity-planning risk the traffic curves point at. Plans built on human assumptions snap under the combination.
The direction is the settled part. The month-to-month line is jagged; HUMAN's own monthly tracking recorded a 4.3% dip in overall agentic traffic in May 2026, even as agentic traffic to financial services more than doubled and site operators stepped up blocking. Across the full year the growth held. HUMAN measured rapid growth in its agentic category; separately, our benchmarks show one GPT-5 research workload needing substantially more serving capacity per query. Together they make a capacity-planning case, not a volume forecast. The long-run multiplier is unknown; the workload shift is already measured. Agents are the new user of search, and the new user brings a workload the old infrastructure was never priced for.
The pattern is visible on the web because the web is where someone is measuring: Exa sees it in search demand, Cloudflare sees it at the edge. But the same loops, the same ten-term queries, the same fan-out arrive wherever agents are put to work, and for most organizations that means their own data: documents, tickets, logs, code. That curve will not show up in anyone's traffic report, and it probably matters more, because getting agents to work over your data is the horizontal challenge every organization is about to share. It is the case we made for a new retrieval engine before the web numbers existed to prove it.
The teams that will handle this well are the ones treating agent retrieval as its own engineering problem now: an engine built for long structured queries in loops, throughput as a first-class concern, and costs modeled on agent behavior rather than human sessions. The ones that will struggle are waiting to see whether the curve is real. Cloudflare's bot majority already arrived, ahead of its own CEO's forecast. Waiting is a bet that the fastest-growing slice of measured traffic, public or private, stops growing.
Hornet is the retrieval engine built for agents, designed for the workload this post describes. Next in this series: what agent retrieval actually costs, with the full model. To be notified when it lands, .