Maestro
Checks and re-routing
Every result is checked by something that did not produce it. When a check fails, only that task moves to a stronger model.
What counts as a check
A check is a test, a review or a rule that a result must pass. It is always run by something that did not produce the result:
| Kind of work | Checked by |
|---|---|
| Code | Tests, plus a review by a different model |
| Copy and content | A review |
| Fixes in a run | Tests, a scan for secrets, file limits and cost, then a review |
| Anything with a rule | The rule itself, such as where data may go or how much a task may spend |
When a check fails
The task is re-routed: it goes to the next model up and tries again. Only the failed task moves. The rest of the work stays where it is, on the models that did it well.
You pay the higher price for that one fix, not the whole job. In the example store build, seventy-nine of 3,559 tasks failed a check and were re-routed, and their retries are inside the $29.10 total.
In the record
Every retry is in the record: the task, the check it failed, the model it moved to, and what the retry cost. In a run, a fix that fails three tries is marked failed and kept in the record, so nothing that was tried is lost.