How patterns are tested

Every number on this site is produced by the test harness in the repository, on a local machine, and committed with the code that produced it.

Setup

Models run locally through Ollama. No hosted API is involved.
Each result file records the machine, chip, memory, operating system, Ollama version, model digest and quantization, run date and the git commit of the harness.
Inputs are synthetic and labelled by hand. They are written to contain the cases that break pipelines in practice: localized number formats, relative dates, missing fields, instructions embedded in the input.
Each input is run under several arms (for example: no control, validation, validation plus repair, constrained decoding) with fixed seeds, at more than one temperature, and repeated.

results/<pattern>.json: the summary shown on the pattern page.
results/raw/<pattern>/<model>.jsonl: every request and reply, so any number can be traced to the outputs behind it.

Results hold for the model, quantization, runtime version and hardware listed. A different combination can behave differently; that is why the harness is published.
The input sets are small and designed to expose failure modes. They show whether a control works and roughly how much it costs, not production error rates.
A pattern without committed results is marked as a draft and is not indexed.