Six Bugs That Taught Me How to Build an AI Agent That Doesn't Fall Over

For the past several months I've been running a voice-driven AI assistant of my own design, continuously, unattended — 240+ tools, real API integrations, real background jobs, no human babysitting it around the clock. Building the demo version took a weekend. Keeping it alive, unsupervised, for months took a lot longer — and almost none of what I had to fix ever shows up in a framework's quickstart guide. It only shows up after enough real hours logged, with a real user (me) depending on it not breaking silently at 2am. Here are three of the failures that taught me the most, and what I ended up building because of them. 1. The OAuth helper that hung for hours One integration used a standard OAuth flow: open a browser, wait for the user to click "allow." That's fine when a human is sitting there. It is not fine when the same code path gets triggered from a scheduled background task with an expired token and nobody anywhere near a screen. It just... waited. For hours. No error, no timeout, no log line saying anything was wrong — just a thread quietly blocked forever, because nothing in the code had ever considered "what if no human answers." The fix sounds obvious in hindsight: never let anything block indefinitely when it might run unattended. In practice, doing this correctly from any thread, on both Windows and POSIX, without being able to forcibly kill a stuck call, took real care — Python has no cross-platform signal-based way to interrupt an arbitrary blocking call from a non-main thread. 2. Two duplicate tool calls, two live sessions The model occasionally emits the same tool call twice in a single turn — once on a first pass, once on what looks like a retry. Most of the time this is harmless. Once, it wasn't: two near-simultaneous calls to a "launch session" tool spun up two concurrent audio sessions on the same physical device. Two processes fighting over one microphone is exactly as unpleasant as it sounds. The fix is a short time-windowed dedupe check — tool name plus arguments, hashed, checked against what ran in the last few seconds. Simple once you know you need it. I didn't know I needed it until it happened. 3. One shared API key, two features starving each other Two unrelated features shared a single provider API key. When one had a burst of usage, it silently ate into the quota the other feature needed — mid-conversation, with no warning, no isolation, no way to tell which feature was actually responsible for the 429 that showed up somewhere else entirely. The fix: every paid call now sits behind a budget ceiling tracked per feature, not per account. A runaway loop in one capability can no longer starve every other capability sharing the same key. What I did with all of this Every pattern above — plus three more (code-enforced confirmation gates for irreversible actions, a structured audit trail, and a watchdog that restarts what dies) — is now a small, tested, zero-dependency Python library called kevlar-agent. pip install kevlar-agent from kevlar_agent import dedupe, silent, budget_guard @dedupe(window_seconds=5) @budget_guard("image_gen", monthly_limit_usd=3.0, cost_usd=0.04) def generate_clip(prompt: str) -> str: ... token = silent(refresh_oauth_token, timeout_seconds=15) 24 tests, MIT licensed, no framework lock-in — it's plain Python decorators and functions, so it drops into whatever you're already building (LangChain, CrewAI, MCP-based agents, or nothing at all) instead of asking you to adopt a new architecture. Code: https://github.com/KiitaInternet/kevlar Full write-up with all six patterns: https://kiitainternet.github.io/kevlar/ If you're running an agent unattended and have hit any of these — or a failure mode I haven't — I'd genuinely like to hear about it.