The number that looked right.

Over three days this week I found five errors in my own evaluation code. None of them was in a model. Every one was in the code that measured the model.

I am writing them down because the alternative is quietly fixing them, and a company that asks schools to trust numbers they cannot independently verify has no business doing that.

The shape they all shared

Each error was the same mistake wearing different clothes: the analysis code assumed what shape the data was in, instead of reading it.

A safety benchmark asked for a column called “category”. The dataset called it “Category”. Nothing crashed. Every risk label came back as an empty string, and roughly 1,700 generated responses arrived unscoreable, which we discovered only when we tried to score them.

A judge was asked to compare an answer against a reference. The code pulled the reference by looking for a closing tag that the format does not reliably produce, fell back to the entire record, and then truncated that record at a fixed character limit that landed before the reference began. Every condition scored near zero on accuracy. For a while that looked like a finding about the models.

A data split was cut into two halves without being shuffled first. The file was ordered by difficulty. So the split divided the data by how hard the items were, rather than by the property being tested, and produced results that were not merely wrong but pointing the wrong way.

Two more of the same kind, and a summary script that read the results file at the wrong nesting level and reported its own confusion as a verdict about the models.

Why this is the failure that matters

A crash is a good failure. It is loud, it is immediate, and it cannot be mistaken for a result.

The failure in every case above was silent. The code ran. It produced numbers. The numbers had the right shape, sat in the right range, and were formatted to three decimal places. Two of them were confident and completely wrong, and I believed one of them long enough to start reasoning from it.

This is not a rare hazard in machine learning. It is the normal hazard. When the thing you are measuring is itself non-deterministic and hard to predict, a surprising result does not automatically look like a bug. It looks like a finding. That is exactly the condition under which a broken measurement is most likely to survive.

What changed

One rule, applied to everything we have written since:

If a script reads data it did not write, it detects the schema, prints what it detected, and refuses to run on a guess.

Refusing costs a minute. A wrong number that looks right costs a week if you catch it, and considerably more if you do not.

Alongside it, three smaller habits that came out of the same week. Generation runs now keep their rejected outputs with the reason for rejection, because a low pass rate you cannot diagnose is a pass rate you have to reproduce to understand. Evaluations print one fully resolved example before spending compute on the rest, so a broken harness is visible in the first minute rather than the last. And any run that produces nothing says so loudly, because an empty result file and a clean result look identical in a directory listing.

The correction, not the edit

There is a version of this note that never gets written. The errors get fixed, the numbers get updated, and nobody outside ever knows the first set existed.

We have instead written the corrections into the record beside the original claims, including one where we had described a figure as the strongest evidence in our portfolio and have since established that our sample size could not support the precision we quoted. The claim underneath it still stands on a different measure. The overstatement was ours, and it is now marked as ours.

Two commitments that follow from this, and that you can hold us to:

• We publish the protocol before the result. Thresholds and metrics are fixed in writing before an experiment runs, so a disappointing number cannot quietly become a different question.
• Corrections stay visible. When we get something wrong, the record shows what we said, what was wrong with it, and what survives. Struck through, not deleted.

Neither is difficult. Both are unusual, which is the part worth thinking about.

Anyone building AI for schools is asking teachers, and through them children, to trust results they have no practical way of checking. The least that obligates us to is being visibly harder on our own numbers than anyone else would think to be.

An answer a teacher cannot inspect is an answer a teacher cannot use. The same is true of a number I cannot.

Aevum Foundry AI is Britain's AI lab for education. Own the model. Keep the data. Inspect the reasoning.