AI agents are being judged by the quality of their answers. That is the wrong place to start. Before asking whether an agent hallucinated, misunderstood a request or made a bad decision, ask: did the work run at all?
I operate scheduled agents across roughly 18 services on one machine. They read telemetry, watch repositories and prepare reports. For 18 months, I expected serious failures to come from the models. I expected a bad decision or a confident answer built on a misunderstanding.
I was wrong about where the danger lived.
Not one serious outage began with a bad model output. Every one began in the plumbing around the model. A job was registered with the scheduler incorrectly. A script called a binary that was actually a shell alias. A timeout guard depended on a command the operating system did not ship. Each failure looked clean from the outside. Each returned exit status zero. The agent had not failed in an interesting way. It had not run.
The outage that looked like silence
The first failure was a scheduled job that never loaded. The configuration existed, the command looked correct and the surrounding service was healthy. The schedule was wrong, so the runner never launched the job. Nothing crashed. Nothing logged an exception. The expected report simply did not appear.
That is a difficult failure to notice because an absent output can mean several different things. The system may have run and found nothing. It may have run and failed to authenticate. It may never have started. A dashboard that displays only the latest successful-looking value turns all three states into the same blank space.
The second failure was a shell assumption. A script invoked a command that worked interactively because it was an alias. The scheduler launched it in a non-interactive environment where the alias did not exist, yet the wrapper still completed successfully. The agent was blamed for producing no result even though it never reached the work.
The third failure involved a timeout guard built around a command available on one machine but absent from the operating system that actually ran the job. The safeguard did not safeguard anything. The process again exited cleanly, leaving a system that looked idle rather than broken.
These are ordinary distributed-systems failures: incorrect registration, environment differences and dependency assumptions. AI makes them harder to see because the model output is the most visible part of the system. Teams inspect the prompt and answer while the scheduler, shell and process boundary remain outside the frame.
Richard Cook’s essay “How Complex Systems Fail” makes the broader point: complex systems rarely fail because of one dramatic component. They fail through interactions among ordinary components whose behavior made sense locally. An AI agent is another component in that system, not a replacement for the system around it.
A good agent reports unknown
The most useful design rule I reached came from a different incident. One of my agents lost an API credential during a run. It could no longer see the channel it was supposed to summarize.
The tempting response would have been an empty summary: no new messages, no action required. That answer would have been true in a narrow sense. The agent had seen zero messages. It would also have been completely misleading, because zero messages was not the state of the channel. It was the limit of the agent’s access.
The agent reported unknown instead.
That distinction is the boundary between a system that is honest about its evidence and one that quietly manufactures certainty. A good agent reports unknown when it cannot observe the source. A bad one reports zero. The same rule applies when a tool call times out, a scheduler has not loaded the job or a dependency cannot be resolved. “No result” and “no data observed” are not interchangeable.
The POSIX specification is not an AI document, but it represents the explicit contracts agent systems need. A human’s interactive shell is not the same environment as a scheduled process, and a successful exit is not proof that the intended work occurred.
Observability must include what did not happen
Most observability guidance begins after execution: collect logs, traces and metrics for the work performed. That is necessary, but agent fleets need another question: what work was expected but never observed?
I now distinguish between a job that ran and produced a result, a job that ran and found no qualifying data, a job that ran but could not access its source and a job that did not run or whose execution state is unknown. Those states need different alerts and different language in reports.
“No new items” should require evidence that the source was reachable and collection completed. If that evidence is missing, the correct result is unknown, not zero. The OpenTelemetry observability primer is a useful starting point for thinking about traces, metrics and logs as evidence of behavior. For agent systems, I extend that idea to execution receipts: the schedule that fired, the command that resolved, the credential or connection that was used and the source that was actually observed. A final model response is only the last link in that chain.
The lesson is not to add more prompts. Make the boundaries testable. Verify binaries. Test the scheduler’s real environment. Treat missing credentials and dependencies as explicit states. Add a heartbeat that proves the work ran, not merely that configuration exists.
AI agents can produce bad answers. That failure is visible and often reviewable. The more dangerous class is a system that produces a plausible empty answer because its execution path disappeared before the model had a chance to do anything.
Your agents may not be failing. They may not be running. Before evaluating the intelligence of the answer, verify the existence of the work that was supposed to produce it.
How do I detect when an AI agent scheduled job never ran?
Distinguish between four execution states: job ran and produced a result, job ran and found no qualifying data, job ran but could not access its source, and job did not run or whose execution state is unknown. Implement heartbeat signals or execution receipts — logging the schedule that fired, the command that resolved, and the credential used — so a missing heartbeat triggers an alert rather than silent absence.
Why does a scheduled agent return exit code 0 but produce no output?
A zero exit code confirms the wrapper process completed, not that the intended work ran. Common causes include shell aliases that resolve interactively but not in a non-interactive scheduler environment, misconfigured cron or job-runner schedules that never trigger, and missing dependencies (such as a timeout utility) that cause the agent to skip its work silently.
What is the difference between ‘no result’ and ‘unknown’ in an AI agent system?
‘No result’ implies the agent ran, observed the source, and found nothing. ‘Unknown’ means the agent could not verify whether it observed the source at all — for example, because a credential was missing or a tool call timed out. Conflating the two causes systems to report false certainty, making real failures indistinguishable from genuinely empty data.
How should observability be set up for AI agent fleets?
Standard observability (logs, traces, metrics) covers what happened during execution. Agent fleets additionally need expected-but-not-observed tracking: alerts that fire when a scheduled job produces no execution receipt within a time window. OpenTelemetry provides a foundation; extend it with agent-specific execution receipts covering the schedule trigger, resolved binary, credentials used, and the data source actually queried.
