Evals

How we check renders

Every render passes automatic checks before it is stored. Separately, a run renders a golden set of product photos and scores them against five criteria, so a change to a prompt or a model shows up as a number.

Results

What a run scores

Each photo of the golden set is rendered with the live pipeline and scored on five criteria, pass or fail.

Reviewed against the photo
Shape and proportions
The model keeps the shape, proportions and parts of the photographed object, with nothing added or removed.
Automatic check
Eight different views
The eight turnaround views of the sheet are different pictures, and the object is in every cell.
Reviewed against the photo
Material realism
The object looks makeable in its material: stone cut or carved and never bent, wood with the grain along each part, metal tube bending in smooth arcs.
Reviewed against the photo
Colour and finish
Colour, texture and finish (polished, brushed, matte) match the photo.
Automatic check
A valid file
The GLB opens, is complete and passes the file check.

Checked on every render

These checks already run in production. A render that fails one is not delivered, and its credits are refunded.

  • Photo: at least 400 px on the short side, under 12,000 px on the long side, no wider than 4:1 and not a flat colour. It is rejected before any model runs, so it costs nothing.
  • Sheet: at least 768 px, the object visible in at least 7 of the 9 cells, and no more than 3 of the 8 turnaround views repeated.
  • 3D file: a complete GLB (binary glTF, version 2) of more than 20 KB.