I audited my AI coding setup: a security hook that never ran, and 11 dead agents

For months I believed my Claude Code setup was fine. Hooks, agents, rules, automations. Everything looked healthy, because nothing threw errors.

Then I audited the whole thing for the first time since building it. Roughly half of it was either dead or faking it.

The worst find: a security hook that had never run. Not once. Since the day I installed it.

The security hook that never fired

I wrote a hook to stop secrets (API keys, tokens) from leaking into tool output. The code checked a field called tool_result. The actual field in the payload is named tool_response.

One word. The match never succeeded, so the hook silently did nothing on every single call, for months. No error, no warning. A guard that never fires looks exactly like a guard with nothing to catch.

Fixing the typo would not have been enough, though. The hook ran after the tool executed. By the time it could mask anything, the model had already read the output, secret included. Post-hoc masking is too late by design, not by accident.

So I rebuilt it to block before execution: intercept the tool call, refuse it if it touches sensitive files. Then I fed it 32 test inputs, normal ones and ones with planted secrets, and watched it pass and block correctly. Only after that did I get to say “it works”. For the months before, I had been trusting a safety device that did not exist.

Eleven agents that were not there

I keep agent definitions as markdown files with YAML frontmatter. At some point I added a one-line comment above the frontmatter in a batch of files. Something like # Part of ....

That single line broke frontmatter parsing. The system stopped recognizing those files as agents.

Eleven agents, including my architect, planner, and code reviewer, were not registered at all. The definition files sat there looking perfectly fine. No error, no warning. They were simply missing from the roster, and I kept believing I was “using” them.

The smaller rot

The rest of the audit was the same story in smaller portions.

A work log had grown to 20MB because a sync script had gone missing. It was being read in full at every session start, burning a second or two each time.

A hook had timeout: 5000. The unit was seconds. I had configured an 83-minute timeout.

A routing table had 23 entries pointing at agents that do not exist in my environment. They were copied from someone else’s config and never pruned.

And one find surprised me: upgrading the model made the rot worse. The new model follows instructions more literally. Pressure phrases I had written to bully the old model (“ALWAYS do X immediately”) started causing overtriggering: excessive tool calls, double-checking trivial things. If you swap models and keep your config unchanged, the config drifts out of tune with the model.

“It exists” is not “it works”

Every failure above shares one property: silence. A comment line, a field name, a time unit. None of them threw. The system looked healthy, and I kept assuming it was.

“No errors” is not evidence of “working”. Dead configs do not scream. If you never open them up, you never even find the bodies.

The fix is not sophisticated. Treat config the way you treat code, and audit it:

• Security hooks: feed them planted secrets and watch them actually block.
• Agents: confirm they actually get invoked in a session.
• Anything that publishes or writes: read the result back and verify it.

I skipped that last one once. It cost me a batch of wiped post titles.

Limitations

• This is one person’s setup, audited once. I do not know whether other setups rot at the same rate.
• The audit was manual. I have not settled on a cadence or automation for making it regular.
• The specific failures are mine. Different tools and versions will rot in different places.

This is an English adaptation of a post I originally wrote in Korean (June 2026). The incidents and numbers are unchanged.

---

Author: Jason Hwang

I write about running AI coding tools and one-person automation at cd4761.blogspot.com. Frontend dev for 8 years. These days I spend more time checking whether the things I built still work than building new ones.

Tags: #ClaudeCode #ConfigRot #AICoding #Hooks #Agents #DevAutomation