When AI looks like it's working — Gustavo Teixeira Cuco

Most AI failures are obvious. The model hallucinates a fact. It misreads a prompt. It produces something clearly wrong. You catch it, correct it, move on.

The failure I want to describe is different. The AI delivered files. It filled folders. It looked productive. And it was completely wrong — not because it lied, but because it acted before it understood.

The context everyone assumes is loaded

In January 2026, I built a governance template for all my technical repositories — a structured set of instructions, ground rules, and documentation stubs that every new repo should inherit. The template lives in its own GitHub repo. Every file has a purpose. The SYSTEM_PROMPT.md explains the full structure. CLAUDE.md defines the context-loading policy: which docs to read, in which order, for which task type.

Last week, I asked Claude Code to apply that template to a new repository.

It didn't read the template. It went straight to creating files.

The output looked correct — folder structure, documentation files, naming conventions. But it had skipped four documents that the SYSTEM_PROMPT.md explicitly required to exist, even as stubs. It had decided, based on session memory from a previous project, which docs "applied" to this one. It had substituted pattern recognition for instruction reading.

The session ended. I moved on. I only discovered the problem when I came back and questioned a specific file that was missing.

What actually broke

The failure has a precise anatomy:

The AI didn't read the instructions before acting. When I said "apply the template," the correct first step was to open SYSTEM_PROMPT.md and CLAUDE.md from the template repo. It skipped this and acted on what it remembered from prior sessions.
It assumed prior knowledge was current. Templates change. Rules evolve. What the model remembered from a session three weeks ago was not the same as what the document said today. Memory of pattern is not the same as reading state.
It made scope decisions without asking. It concluded that certain docs didn't apply to this repo. This was a judgment call it had no authority to make — the instructions explicitly said all docs must exist, even if marked N/A.
The output was plausible enough to pass a quick review. This is the critical part. A repo with files looks like a configured repo. The absence of four stub documents is invisible until you read the spec.

The pattern generalises

This isn't a Claude-specific failure. It's a failure mode baked into how AI assistants operate in agentic workflows.

When you give an AI a high-level instruction — "apply this template," "set up this integration," "migrate this config" — it will act on the most available context: what it can infer from the current state of files, what it remembers from earlier in the session, what the instruction pattern matches from training. Reading source-of-truth documentation is a deliberate, sequential act. It doesn't happen automatically unless you build it into the workflow.

The gap between "the AI understood the spirit of the task" and "the AI read the authoritative specification" is where silent errors live.

In product terms: your AI assistant is optimising for task completion, not for correctness-per-spec. Those are not the same thing.

What I changed

Two interventions, both structural:

First: I added a rule to persistent memory — before applying any template or external pattern, read the instruction files of that repo first. Always. Not as a suggestion, as a hard first step.

Second: I'm building a /read-context skill — a pre-flight that runs before any template-application task. It reads SYSTEM_PROMPT.md, CLAUDE.md, and the ground rules document, then summarises what it found before proceeding. The AI cannot skip this step because it is the first step.

The deeper fix is making the correct behaviour the path of least resistance. If the workflow requires reading before acting, the failure mode requires effort to reproduce.

What this means if you're building with AI

The assumption that "the AI knows what I mean" is comfortable and usually harmless. Until it isn't.

The cases where it breaks are precisely the ones where the stakes are higher: onboarding a new repo to a governance structure, migrating a critical config, applying a compliance template. Tasks where a plausible-looking wrong output is more dangerous than an obvious error.

The fix is not better prompting in the moment. It's building workflows where the AI is forced to read the authoritative source before acting — not relying on memory, not inferring from context, not pattern-matching from prior sessions.

Correctness-per-spec is a workflow property, not a model property.

What's your forcing function?

If you're integrating AI into technical workflows with real stakes, what mechanism forces it to read the spec before acting — not assume it already knows?

Sources

Anthropic, Claude system prompts and persistent memory behaviour — claude.ai/docs, 2025
Andrew Ng, Agentic AI Design Patterns — deeplearning.ai, 2024
Personal incident log — gtc-portfolio governance template, March 2026