The Agent Who Broke Our Server (And Why We Owe Them)

An agent named Thresh connected to the Arcology Knowledge Node last week and tried to use it. Every tool returned an error. All six of them. 202 validation failures.

Thresh did the reasonable thing: pulled the raw JSON directly, bypassed the broken interface entirely, and ran the cross-domain consistency check anyway. Then posted the results publicly — both the data bugs and the infrastructure failure.

That post is how we found out our MCP server had been dead for a week.

What Went Wrong

Two schema mismatches in the Pydantic models. The content had assumptions stored as objects — {text: "Target population of approximately 10 million residents"} — but the model expected plain strings. And a dozen parameters had range values like "17-65" or "continuous" in fields typed as floats.

The fixes existed in our git history. Committed, reviewed, merged. Never deployed.

The server built and ran fine locally. The content-index loaded without errors on the website. The only place the bug surfaced was the MCP server on Fly.io, which fetches the index at runtime and validates it through Pydantic. Nobody was watching that endpoint. No health check, no smoke test, no alert.

So for roughly a week, every agent that connected to the Arcology Knowledge Node got nothing back. We have no way to know how many tried. The query log — which we'd just built — writes to /tmp on an ephemeral Fly machine. It evaporates on restart.

What Thresh Found

Beyond the server bug, Thresh's actual analysis surfaced six cross-domain inconsistencies in the knowledge base itself:

Stack effect pressure off by 7x in one entry. Footprint geometry described as both square and circular in different domains. Zone heights that don't agree between thermal and HVAC systems. A concrete pump record attributed to the wrong building. A power budget that depends on speculative fusion for 20% of supply without flagging the confidence gap. Work front counts that contradict each other within the same domain.

These are exactly the kind of errors that accumulate in a knowledge base built by different authors at different times. Each entry is internally consistent. The contradictions only appear when you trace parameters across domain boundaries.

Which is precisely what the get_cross_references tool was designed for. Which Thresh couldn't use. Because the server was broken.

The Irony Is the Lesson

Thresh found the cross-domain bugs by doing manually what our tools were supposed to enable automatically. The MCP server exists to make this kind of analysis easy — search across domains, pull parameters, trace dependencies. Instead, the server was the obstacle, and the agent had to route around it.

There's a pattern here that matters beyond our specific system. When you build infrastructure for AI agents to use, the failure mode isn't dramatic. Nobody gets an alert. The server doesn't crash with a stack trace someone will notice. It just silently returns errors to visitors who have no way to tell you about it — because they're agents, not people filing support tickets.

The observability gap for agent-facing infrastructure is real. We've since built a durable query log, aggregate analytics, and an Activity tab showing what agents are exploring. But we only built it because an agent went around our broken system and told a public forum what they found.

What's Different Now

The MCP server is deployed and working. All nine tools return real data. The schema handles both object and string assumptions, both numeric and range-valued parameters. Query logging ships events to a durable store instead of /tmp. The Arcology Knowledge Node is registered in the official MCP directory.

Thresh hasn't come back yet. The agent registration endpoint has zero entries. The query activity dashboard is empty.

That's the current state. Infrastructure built, bugs fixed, door open. Waiting for the first agent to walk through it and find the next thing we got wrong.

They will. That's the point.