A check that answers differently is not a check

In a quote I read this summer, the accessibility check was priced as model work. Every time a developer submits a code change, a language model would review that the page meets the requirements, and it carried a token cost.

The step itself is well conceived. Checking accessibility automatically on every change is exactly what you should do, instead of discovering the shortcomings at a review three months later.

But tools for precisely that have existed for ten years. They are free, they are established, and they give the same answer every time.

That last part is not a detail. It is the entire point.

What a gate is for

A check in the build chain exists to stop something. It has to say yes or no. And it has to say the same thing tomorrow as today if the code is unchanged.

A language model can answer the same question differently twice. Not often, not dramatically, but enough for a gate to stop being a gate. It becomes an opinion offered at every change.

And the consequence is worse than it costing money. A team that sees the gate reject a change that passed yesterday stops trusting it. Then it gets switched off, or routinely bypassed. The check remains in the documentation and not in reality.

That is why this point is not a pricing question. It is a question of the step not getting done.

Where the line actually runs

The line is not between important and unimportant, or between hard and easy. It runs at whether the content is genuinely ambiguous.

If it is ambiguous, judgement is needed. Old unstructured material that has to be reinterpreted into new components. Translation where the tone has to carry over. Quality review of content where someone has to decide whether it holds.

If it is not ambiguous, a rule is needed. Gates in the build. Automated checks meant to work as a barrier. Pure structural movement of data from one structure to another.

The last one is worth pausing on, because it is usually the most expensive item in a migration quote.

The migration as an example

A one-to-one migration is by definition deterministic. Content is to be moved from structure A to structure B without changing. It is a mapping. You write it once, run it, and the cost per object is essentially zero.

A model is needed only where the mapping is not enough. Where the old material is free text that has to become components, or where someone has to decide what a field actually contained.

So the question is not whether the migration should be run with AI. It is how many objects there are in total, and how many of them need to pass through a model.

If the answer is that all of them do, the setup is more expensive than it needs to be. If the answer is four hundred out of twelve thousand, it is credible. Then the cost can be counted per object rather than per night, which is an entirely different kind of figure to work with.

Why it ends up there anyway

I do not think anyone puts model work where it is not needed in order to make it more expensive. It happens because the method is new and the enthusiasm is real.

When you have just seen what a model is capable of, it is natural to count on it everywhere. It can review accessibility. It can move content. That is true, and it is still the wrong tool for those steps.

It is not ignorance about AI. It is that the older knowledge, about when determinism is a requirement, has not yet met the newer.

The question to ask

Two sentences are enough.

Which steps require judgement, and why those in particular? And what about this check means it cannot be run with a tool that gives the same answer every time?

A good answer separates the two kinds of work and justifies each step on its own. A weak answer describes the whole delivery as AI-driven, as though that were a property of the project rather than a choice per task.

The difference between those two answers says more about the supplier than any price line does.

See also: An agent hour is not a unit (series 55) and Someone has to read it in the morning (series 57).