The Human Is the Opening
Photograph a printout of a photo. Print the photo. Photograph the printout again.
Do that ten times. The image is mush.
What's interesting is that no one did anything wrong. Every individual step was executed correctly. The camera worked, the printer worked, the paper was right. And yet the result is unusable, because each copy could only draw from the previous copy, and each round lost a little.
This is not an analogy for what happens when an AI loop runs without a human. It is the same mechanism.
The Mechanism Has a Name
In analog technology it's called generation loss. In machine learning it's called model collapse: models trained on their own synthetic output degrade, and they do so in a predictable order. The edges of the distribution disappear first, the unusual and the specific, then the middle hollows out.
Worth keeping in mind the next time someone dismisses the argument as a metaphor. It isn't a metaphor. It is a literal description of what happens to information in a closed system.
The Objection Worth Taking Seriously
There is a strong objection to saying that AI cannot add anything new.
It goes like this: humans only do the same thing. All creativity is recombination. No thought arises from nothing. Everyone builds on what they've seen and heard. If the model can't create something new, neither can you.
That objection is correct. Which is why the formulation "AI can't create anything new" is wrong. It proves too much, and you lose the argument, rightly so.
The dividing line is not between recombination and creation. It runs between a closed system and an opening.
A camera photographing its own printouts is closed. It has no channel to anything outside itself, so degradation is guaranteed. A human has channels the camera lacks. The senses. An experiment that returns a different answer than expected. A real user who gets angry. An invoice that was wrong and a customer who calls. A decision that turned out to cost money.
That is information that didn't exist in the history. It comes from outside. And it only comes in through someone who is there.
Recombination is still valuable. Synthesis and compression are real work. But recombination is bounded by the amount of information it starts from. It can never exceed what is already in the material.
Why the Debt Compounds
The danger of an unsupervised loop is not that errors accumulate. It's that they compound.
Every unvalidated automated step builds on the previous unvalidated step. Noise in round N is treated as signal in round N+1 and as established fact in round N+2. After enough rounds, the error is no longer an error in the system's view. It is a premise.
That's why "let it work while I sleep" doesn't save time. It defers a growing debt to the point where no one can see where it entered.
Review is amortization. Skip the payments and the debt doesn't grow linearly. It grows exponentially.
Cloning Is the Same Thing One Level Up
This doesn't only happen inside the model. It happens in how people work.
Agent courses are being sold with the explicit promise that you clone ready-made systems instead of starting from scratch, then sell the cloned workflows on to your own clients. Hundreds of people run the same flow. Not one of them has added a single observation from their own reality.
The system behaves exactly as the mechanism predicts. Information is redistributed, nothing is added, and the specific disappears first. What made the original sharp was someone's actual knowledge of an actual problem. That doesn't survive the cloning. Only the form does.
A cloned agent is not a shortcut to someone else's judgment. It is a copy of their form without their reasons.
And starting from scratch is not the problem to be engineered away. That's where the opening is. The person who starts from scratch is forced to look at their own problem. The person who starts from a clone never has to, and doesn't notice they've opted out.
What This Means for Who Should Be Sitting There
If the human is the opening, it matters which human.
The opening is worth something to the extent that it lets in genuine signal. A senior with domain knowledge sees that the answer looks plausible but contradicts how things actually work for this particular client. That is new information entering the system. A junior who approves because it looks right lets nothing in. The loop is still closed. It just has a human sitting next to it.
This is the same as saying you can't validate what you don't understand, stated from the other direction. Validation is not a rubber stamp. It is the moment when reality gets to come in.
One Gate, Two Flows
I've written about why the human stays in the loop even as the model approaches near-perfect accuracy: someone has to bear the consequence, which a machine cannot bear no matter how capable it becomes.
This is the second reason. It is independent of the first. Even for someone who doesn't care at all about accountability, a closed system gets worse.
The same gate carries both. New information flowing in, responsibility flowing out. Close it and both flows stop simultaneously. The first loss shows up in quality long before the second shows up in accountability.
That makes human in the loop something other than a brake. It is not quality assurance that costs time. It is the only place where anything new enters the system at all.
See also: Faultless Is Not the Same as Accountable (series 60) on the other flow through the same gate, and You Can't Validate What You Don't Understand on why the opening requires a senior.