I Kept Teaching the Same Goldfish

When this happened: January-February 2026. Mostly Cursor and Claude Code on Windows.

This is where we started. Our stack looks different today. The models will improve. The architectural mistake in this story will still be available to anyone who wants to repeat it.

Table of Contents

A quick terminology note
Where AI coding agents actually break
Review stopped being review
We tried to out-prompt the problem
The goldfish came back mid-task
Prompts are not memory
The folder of things I was tired of saying
No artifact, no approval
The economics changed
What we had actually built

Early in 2026, we started putting AI coding agents on real client work: mobile apps, backends, cross-layer changes, old callers nobody wanted to break, and tests that had to fit patterns already living in the repo.

The speed was obscene. Work that could occupy a developer for a day appeared in minutes.

We leaned in and hit a wall.

The agent produced code that looked right and failed in ways that did not announce themselves. It called a method that did not exist in our SDK version. It inverted a condition on one input path. It wrote a test that passed forever while proving nothing. Worse, it used our naming and style, so the bad code looked native.

My first diagnosis was obvious and wrong: I needed a better prompt.

I added architecture notes, file paths, test conventions, edge cases, and instructions to run the build before declaring victory. Soon I was writing prompts that looked suspiciously like full engineering specifications. If I already knew every file, dependency, constraint, and failure mode well enough to specify the implementation, I had done most of the difficult thinking myself. The agent reduced the typing while the cognitive load stayed with me.

Two persistence failures were hiding underneath the apparent prompting problem:

1. Reusable engineering behavior: architecture rules, test standards, review requirements, and the definition of done.
2. Task-specific state: the current objective, decisions already made, files changed, checks completed, work remaining, and known blockers.

Agent-written code turned ordinary review into a manual correctness audit across files, dependencies, tests, and error paths

Both lived in chat. Chat was the wrong database for both.

A quick terminology note

I need six terms for the rest of this series:

• Model: the underlying language model.
• Host: the environment that gives the model an interface and tools, such as Cursor or Claude Code.
• Agent: the model plus instructions, context, tools, and an execution loop.
• Skill: a version-controlled set of reusable instructions or workflows loaded when relevant.
• Task state: project-specific facts stored outside the conversation so work can continue across sessions, hosts, and context loss.
• Artifact checkpoint: a required file or output that proves a workflow stage happened. A cheerful sentence in chat does not count. We wanted evidence on disk.

These distinctions matter. A model can change while the host stays the same. A task can continue after the model or host changes. And a context window is not memory; it is the finite working set available for the current model call.

If the process depends on something the agent once heard in chat, the process depends on luck.

Where AI coding agents actually break

Give an agent a clear input, a clear output, and limited side effects, and it can be excellent. Rename a function. Write a parser for a fully specified format. Generate a fixture. Explain a stack trace. Fine.

The trouble starts when the task has a blast radius:

• a change crosses presentation, domain, and data layers
• several existing callers depend on the method being changed
• tests must match project-specific patterns the agent has not inspected
• state must stay consistent after mutations
• an SDK or internal API must be used exactly as implemented, not approximately
• success depends on files the agent may never choose to open

The same failure classes kept coming back:

• SDK methods invented for the version we did not have
• conditions inverted without a syntax error
• mutable values fetched once and accidentally frozen
• exceptions swallowed while success was reported
• hardcoded placeholders presented as finished implementations
• tests that could not fail

None of this looked like garbage. Garbage would have been easier. The code compiled often enough, matched the local style, and told a convincing story. That made it dangerous.

Review stopped being review

A normal code review begins with a useful assumption: another engineer understood the language, inspected the relevant code, and would flag genuine uncertainty. The reviewer can focus on the approach, structure, trade-offs, and architectural fit.

That assumption disappeared with agent-written code.

For cross-layer changes, review became a correctness audit. We were no longer asking only whether an abstraction was sensible. We were checking whether the method existed, whether every affected layer had been inspected, whether the test could detect the defect named in its title, and whether an error path had quietly become a success path.

The agent did not say, "I am unsure whether this method exists." It simply called the method with impeccable confidence.

So I became the architect, reviewer, QA engineer, debugger, and institutional memory for code I had not written. That combination was more exhausting than implementing the feature myself. When you write code, your mental model grows with it. Here I had to reconstruct somebody else's plausible-looking mental model and find the places where no model existed at all.

The work was faster. I am not going to pretend otherwise. By the end of the day, though, I was cooked.

We tried to out-prompt the problem

Our first fixes were reasonable. They were also partial.

Attempt one: repeat the important rules at the start of every conversation

Read the architecture document. Follow existing test patterns. Ask before assuming. Run the build. Do not claim completion while work remains.

The agent would comply for a while and gradually drift. We also used Cursor for some tasks and Claude Code for others. Different hosts and models exposed different blind spots, which meant the instructions had to survive more than one environment.

Attempt two: put the instructions in Cursor rule files

Better. The host could inject them instead of making us paste them manually. But the files lived on one machine, worked in one host, and forked easily into several slightly different versions of "how we work."

Attempt three: point multiple rule folders at one shared directory with Windows junctions

Correct instinct, brittle implementation. We eventually replaced the junctions with sync scripts.

Another paragraph in the prompt would only restart the cycle. We needed one canonical, version-controlled source that every supported host could load.

It took us an embarrassing amount of time to admit this because prompt tweaking feels productive. You change a sentence, run the agent again, and sometimes the result improves. That local improvement hides the structural failure: nothing durable has changed.

The goldfish came back mid-task

The agent usually got better after enough correction. I pasted paths. I showed it the method it should have read. I rejected an invented abstraction. I made it rewrite a meaningless test.

Eventually, the result became acceptable. Sometimes it became genuinely better than what I had planned: a cleaner abstraction, an edge case I had missed, or documentation that would otherwise have remained six months out of date.

The next day, I opened a new chat.

Dumb again. Back to step one.

The cost was not retyping instructions. The cost was being the memory for every architectural decision, rejected approach, testing convention, and project-specific constraint. Every conversation began with archaeology.

Long conversations did not make this reliable. The model only receives what the host sends on a given call. As the working context fills, older material may be truncated, summarized, compacted, or simply become harder for the model to use. The exact mechanism depends on the host. The result we experienced was the same: a decision established earlier in the session could stop influencing the work.

Reusable behavior changes slowly and applies across projects; task-specific state changes constantly and belongs to one job

Move to another PC? Start explaining where everything lives.

Hand the task to a colleague? More archaeology.

Switch models to control cost? Enjoy onboarding a new amnesiac contractor halfway through the job.

The phrase "the model forgot" is convenient but slightly misleading. The durable information had never been stored where the workflow could reliably retrieve it. We had confused temporary context with persistent memory. As a result, we blamed the context window for behaving exactly as designed.

Prompts are not memory

Once we saw the problem this way, the solution changed.

We stopped trying to fit the entire company brain into the opening prompt and moved durable information outside the conversation.

Architecture rules, testing standards, review requirements, naming conventions, and quality gates applied across many jobs. They changed slowly. The current objective, decisions, modified files, validation status, and blockers belonged to one job and changed constantly. Shoving both into the same giant prompt had made a mess of them.

Skills handled behavior: how work should be performed regardless of the current feature.

State files handled continuity: what was true about one task at one point in time.

Mixing them creates problems in both directions. Behavior changes slowly and should apply across many tasks. State changes constantly and belongs to one job. Put task history into a global skill and it becomes stale baggage. Put engineering rules into a chat summary and they evaporate when the conversation does.

The folder of things I was tired of saying

The first repository was embarrassingly simple: a folder of Markdown files containing things we were sick of explaining. There was no framework or clever CLI yet.

When critical context disappears, developers must repeat earlier instructions and audit another plausible implementation from scratch

The structure changed later. The principle survived: important context had to exist somewhere an agent could re-read on another machine, in another session, through another host.

The first skill we formalized was code-guidelines

Its rules came from actual audit failures. We skipped the decorative best-practice list.

The full file grew to 185 lines across ten sections. What mattered was the shape of each rule: name a defect class, explain the constraint, and give us something concrete to check.

One of those rules came from a state mutation

The agent fetched initial data once, injected it into a state manager, and produced code whose types and dependency structure looked perfectly reasonable. After a mutation, the manager kept using the frozen value. The defect appeared only when state changed.

So the lesson became a rule: do not fetch mutable data at startup and inject it as permanent state. We also added a regression test that mutated the source and confirmed the manager could see the new value.

We were recording institutional scar tissue and attaching a test to it. Calling that a style preference would miss the point.

No artifact, no approval

The companion workflow was assess-and-fix.

We kept seeing the same absurd failure. The agent would announce that an assessment was complete. I would look for the report, fail to find it, and ask where it had been saved. Only at that point would the agent admit that no file had been created.

Confident summary in chat. Required artifact: missing.

So we changed the definition of done. Before the assessment could pass, ASSESSMENT_REPORT.md had to exist on disk. Later, assess-and-fix was absorbed into a broader job framework. The rule survived:

No artifact, no approval.

An agent can know a rule, claim to have followed it, and still leave no evidence that it did. Instructions made behavior more consistent. Artifact checkpoints made completion testable. We needed both.

This gave us the first hint that a library of prompts would never be enough. We needed execution stages, state, required outputs, checks, and approval gates. We did not call it orchestration yet. We were trying to stop an agent from congratulating itself for a report it had never written.

The framework came later. By this point, we had discovered why we would need one.

The economics changed

Eventually, this work reached a real client project.

The client was a news outlet with a mobile app. The request sounded simple: record a video in the app and upload it.

For a news organization, the video may be the story. If a journalist captures an event, the network drops halfway through the upload, and the footage disappears, "please try again" is useless. The event is over.

The actual feature went well beyond a basic upload endpoint. It included:

• idempotent, chunked uploads with exact chunk-level progress
• retries that continued from the failed chunk
• state preserved in memory and in a local database
• immediate storage of captured video in the iOS sandbox
• protection against duplicate uploads after successful completion
• recovery from network interruption and application failure

The work crossed mobile and server code, persistence, networking, recovery logic, and test coverage.

Before the pilot, we had estimated the same scope at 240-300 hours for fully manual delivery. The client could not fit that into the budget. We offered a production pilot of our internal workflow. We told them that much of the implementation would be AI-written and described the risks. They agreed.

The feature took roughly 40-50 hours, including implementation, structured review, and validation. Anyone picturing five hours of prompting followed by a victory lap has the wrong picture. The review and verification work are part of the number.

One case cannot prove that every software project becomes five times cheaper or isolate every variable. Treating it as a universal productivity benchmark would be dishonest. The estimate method, quality controls, and post-release results deserve their own case study.

It proved something narrower and more useful:

A controlled AI-assisted workflow made a production feature feasible when the conventional estimate put it outside the client's budget.

The feature existed because the economics changed.

What we had actually built

Important engineering knowledge needs durable storage outside the conversation, where an agent can retrieve it after context loss

At the start, I blamed prompt quality. I was fixing the wrong layer.

We had placed durable engineering knowledge and live task state inside a temporary conversation, and we expected that conversation to behave like infrastructure.

The first useful system was an external memory structure:

• version-controlled skills for reusable behavior
• persistent files for task-specific state
• artifact checkpoints for proof
• human approval where judgment still mattered

Model errors continued, so the system made them harder to hide.

Once we understood the gap, the project changed direction. A prompt library can tell an agent what good work looks like. Preserving live task state, requiring artifacts, running gates, coordinating stages, and proving that the work happened require a larger system.

We had no grand plan to build an agent orchestration framework. We were fixing one infuriating failure after another. Much later, we could look back at the skills, state files, artifacts, and gates and see the outline of one.

Next: Why "write clean code" gives an agent almost nothing to work with. What happened when we turned vague quality expectations into rules an agent could follow and a reviewer could verify.