Three tools told me everything was fine
none of them had actually checked

I keep a written defect log for the tooling I build my own work on, because chat logs don't survive and memory does. Reading back through it this month, the same shape kept showing up: a tool reports success, and the success field is simply not connected to the thing that would tell you it failed. Not a crash. Not an error. A green field, sitting next to the exact number that contradicts it.

The link checker that hid six broken links

On August 6, 2026, I ran docs_check_links against this repo, on tapps-mcp version 3.12.69. It came back success: true, score: 78, broken_links: []. Clean, by every field that's supposed to matter. The same response also carried total_available_broken_links: 6. The tool had counted six broken links and reported zero of them. Re-running with broken_only=true surfaced the actual six, inside AGENTS.md, .github/copilot-instructions.md, and a Linear-issue skill file. Nothing in the first response distinguished "no broken links" from "six broken links, list withheld." I read success and moved on, which is exactly what the field is there to make you do.

The completion gate that graded a different repo

The second one is worse, because it's the tool I'm supposed to trust last. On a branch named docs/linkedin-publish-automation, at commit e5c06b13, I called tapps_checklist and got back complete: true with git_context reporting branch master, head SHA c69ef8a6. That SHA wasn't a stale copy of my repo. It was tapps-mcp's own HEAD, the exact commit its own release directory is named after. The tool resolves git_context from its installation path, not from the project root I gave it. So the one verdict meant to certify the pipeline had run over my changes had never seen my branch, and re-running it wouldn't fix that. It's a misattribution, not staleness.

The memory save that dropped the one field I asked for

The third is the quietest failure of the three. I called the memory save tool with memory_group="linkedin-publish-research", the field that keeps related entries from being merged into each other. The response came back status: "saved", memory_group: null, with a hint telling me to "pass memory_group on brain memory_save," which is the exact parameter I had already passed. One side of the bridge names the field memory_group; the other side expects group. The mismatch means it's silently discarded as an unrecognized argument, and the response reports success on a save that dropped the one thing I'd asked it to isolate.

The same shape, three more times

Once I started looking for it, this stopped being one bug and became a pattern with a name. A stale MCP client served fifteen tools when there should have been sixteen, and every signal short of an actual tool count looked fine. A health check returned ready: true for six hours and forty-five minutes after an OAuth token had already expired, because the canary meant to catch that was disabled. A dry_run passed for a workflow node whose dry-run path stubs out the exact thing being tested, so the check was structurally incapable of catching the bug it existed to catch.

The question that actually catches it

Every one of these has the same test, and it's the only one that would have caught all six: what would this check have returned if the thing it's checking had actually failed? If the answer is "the same thing," it isn't a check. It's a status field that always says yes. That's the standard I hold my own gates to now, and it's why the defect log stays open after the fix ships, not just until the field turns green.

This is the discipline I try to build into every gate I write for my own agents and for the systems I help other teams stand up. Not because a passing check feels good, but because a check that never exercises the failure path is worse than no check at all. It tells you not to look again.

Sources: drawn from this project's own internal defect log, kept in git because a cross-project fix needs the maintainer's sign-off and the findings need to survive somewhere other than a chat transcript. docs/UPSTREAM-TOOL-DEFECTS.md holds the three reproduced defects (docs_check_links, tapps_checklist, the memory-group parameter mismatch). docs/LINKEDIN-POST-AGENT-DESIGN.md holds the recurring-pattern note on the stale MCP client, the OAuth health check, and the stubbed dry run.