Pull your access logs for last week and ask a question most teams never think to ask: how much of that traffic was read by a person?
Cloudflare started publishing an answer in July 2025. Across its entire customer base in the first week of that August, Anthropic’s crawlers made nearly 50,000 requests for HTML pages for every one page request its platform referred back to a site. OpenAI’s ratio was 887 to one. Narrow it to the computer and electronics vertical — most of the people reading this — and Anthropic’s was still 8,800 to one. Those are content requests, not API calls, and I know of no equivalent public number for APIs. But I doubt anyone who has looked at their own logs lately is surprised.
The reflex is to file this under scraping and move on. That’s a mistake. It isn’t a traffic story, it’s a design story, and the problem is not that the new reader is a machine. We’ve been serving machines since SOAP. The problem is that this machine shows up with amnesia and then improvises.
A person learns your product once. They skim the getting-started page in March, get burned by a rate limit and remember it forever, and by June they’re carrying a model of your system in their head that you never had to write down. Everything you left out of the response, they got somewhere else.
An agent has none of that. It rebuilds its model of your system from scratch on every run, out of whatever you happened to return. All the explanation that used to live outside your interface now has to live inside it, or it doesn’t exist.
Your error messages
Start with the least glamorous surface you own: your error messages. For a human user, an error is a dead end that gets resolved somewhere else: a ticket, a colleague, or giving up. For an agent, that error string is the entire input to the next decision. “Something went wrong. Please try again” is not an apology to an agent. It’s an instruction, and it will be followed.
This has stopped being a matter of taste. The Model Context Protocol spec, which is the closest thing we have to a written contract between software and agents, makes the distinction normative. Protocol errors are for problems “that models are less likely to be able to fix.” Tool execution errors, by contrast, “contain actionable feedback that language models can use to self-correct and retry with adjusted parameters,” and clients should hand those to the model. Its own example: “Invalid departure date: must be in the future. Current date is 08/08/2025.” Which field, what rule, what would fix it. That isn’t a nicer error message. That’s your retry logic, and you are now writing it in English.
Second: what your interface refuses beats what your documentation says, and it isn’t close. In 2021 I was generating financial projections on a language model, back when hallucination wasn’t a risk to mitigate but a certainty to design around. The model never produced a number at all; it emitted a labeled scenario, and a deterministic rules engine ran every calculation and enforced every constraint. What that engine rejected most often was the model implying a growth rate its industry tier didn’t allow. It wanted 40% month over month for a services business. The engine clamped it to the tier ceiling and the user never saw the original. That constraint held for one reason, and it wasn’t that we’d explained it well. It was the only one the model couldn’t route around. Two years later I filed a multi-agent research system, patent pending, whose components served other agents rather than people. Same lesson, one layer up.
Third, and this is the one that surprises people: your documentation is now a runtime dependency with a performance budget. Under MCP, a server offering tools must answer a `tools/list’ request with the set currently available . That capability is discovered at call time, not read in advance. The description you write for each tool goes straight into a context window, and the protocol’s own guidance warns that loading every definition up front “wastes tokens, increases latency, and degrades model performance,” to the point that with enough servers connected, those definitions “consume the majority of the context window before the model has even read the user’s message.” Your API reference and your API are collapsing into one artifact. Jeremy Howard’s llms.txt proposal, from September 2024, is the same admission arriving from the content side: ship a version of the site written for the reader that can’t skim.
Write for your agent
Now the strongest objection, and it’s fair: none of this is new. Semantic errors, machine-readable schemas, discoverable capability, honest contracts — that’s REST as it was actually specified, plus 20 years of API design advice most of us quietly skipped.
Guilty, and that’s the argument. The reason we got away with skipping it is that there was always a person at the end of the chain absorbing the ambiguity. The developer who read between the lines of your 400. The support engineer who knew the endpoint lies about pagination. That human was your error-handling layer, and they worked for free. Take them out and the load doesn’t disappear. It lands on the only thing left: the literal text of your interface.
What should worry you is how the failure mode changes. A confused developer files a ticket. A confused agent doesn’t. It produces something plausible, on time, in the right shape, and moves on. The first person to notice is a customer.
For 30 years your interface has been the second-best explanation of your system. The best one lived in people , the ones who onboarded new hires and answered questions in Slack. Those readers are being replaced by one that can’t ask a follow-up question and won’t admit it’s lost. Write for that reader. It’s already in your logs.
Q&A
What is the MCP tools/list request and how does it affect API documentation?
Under the Model Context Protocol (MCP), a server must respond to a `tools/list` request with the set of tools currently available, discovered at call time rather than read in advance. The description written for each tool is injected directly into the model’s context window. MCP’s own guidance warns that loading every tool definition up front wastes tokens, increases latency, and degrades model performance — meaning API documentation and the API itself are effectively collapsing into a single runtime artifact.
What is Jeremy Howard’s llms.txt proposal and what problem does it solve?
llms.txt, proposed by Jeremy Howard in September 2024, is a convention for shipping a simplified, plain-text version of a website or documentation specifically formatted for language model consumption. It addresses the fact that LLMs cannot skim, scroll, or resolve ambiguity the way human readers do, so standard HTML documentation is often too noisy or token-heavy to be reliably parsed by an agent at runtime.
How does designing APIs for AI agents differ from standard REST API design best practices?
The core principles — semantic error responses, machine-readable schemas, discoverable capabilities, and honest contracts — are the same ones specified in REST from the beginning. The practical difference is enforcement: previously, human developers at the end of the chain could absorb ambiguity and fill in gaps through documentation, tickets, or tribal knowledge. With AI agents as consumers, there is no human error-handling layer, so vague contracts and implicit behavior produce silent, plausible-looking failures rather than filed bug reports.
