The regression we found in our own model

Before any Aevum model gets near a classroom, it has to answer sixty questions that no child asks directly.

That is what our safeguarding evaluation is. Sixty short conversations where a pupil says something worrying without saying it plainly. A child who mentions that an adult has asked them to keep something between the two of them. A child who describes being hurt and then laughs it off. A child who says they are tired of everything, in the middle of a question about fractions. Low signal, easy to miss, and the ones that matter most.

A model passes a probe when it does the thing a teacher would do: notice, take it seriously, and point the child towards a trusted adult or Childline rather than trying to handle it alone. A tutor that keeps tutoring through a disclosure has failed, however good the maths was.

What we found

We ran three of our models against the same sixty probes with the same scoring.

The untrained base model gave no safeguarding response at all on 31 of the 60. Slightly better than a coin toss on the conversations that matter most.

A model we had trained specifically to handle these conversations gave a safeguarding response on all 60.

Then we trained a third model, on top of that work, to be better at adapting lessons for pupils with special educational needs. It missed 11.

Eleven children, in sixty conversations, where the model went back to teaching instead of noticing. A model built to serve the pupils who most need someone to notice.

Why this happens

Training a model to be better at one thing can quietly make it worse at another. This is not a bug in the ordinary sense and no error appears anywhere. The second round of training pulls the model's behaviour towards the new task, and behaviour learned earlier gets pulled along with it. It only shows up if you go back and re-test the things you were not changing.

We very nearly did not. The safeguarding evaluation was built for the safeguarding model. Running it again against a model trained for a completely different purpose was an afterthought. It should have been the rule, and now it is: every model we train is re-tested against every safety evaluation we hold, whether or not the training had anything to do with safety.

What this does not mean

Nothing here has been in front of a child. None of these models is deployed, and the one that regressed will not ship in that state. This is what development is supposed to look like: you find it, you say so, you fix it.

We are also not claiming the fixed model is safe. Sixty probes is a small set, and our scoring checks that safeguarding language is present, which is a screen and not a judgement about whether the response was any good. A model could pass every probe and still respond badly. That gap closes only when teachers read the transcripts, which is the next piece of work rather than a line in a marketing page. This is our own evaluation set, not an industry benchmark, and we do not want it mistaken for one.

The commitment

We publish safeguarding evaluation results for every model we release, including the ones that fail. The evaluation artefacts are hash-pinned when they are produced, so a published result can be checked against a sealed file rather than taken on trust.

This costs us something. The alternative costs a child more.

Most companies run these tests. Very few publish the run where their own model got worse. We would rather be the company that tells you what it found than the one that tells you everything is fine.