LLM → CAD code · verification
Does LLM-written CAD code run, and does the part come out the right size?
claude-opus-5-5 writes CadQuery programs for 151 prompts from the public preview of
Text2CAD-Bench (CC BY 4.0). Two verifiers check every program,
and one round of verifier feedback goes back to the model. This page shows the committed runs; it runs no model and no CAD code.
How a program is checked
Every generated program goes through the same five steps.
01
Prompt
A step-by-step (pro) or geometric (geo) description of one part.
02
Generate
The model writes a CadQuery program that assigns the part to
result.03
Execution check
Runs in a separate process, imports only cadquery and math, must leave a valid solid.
04
Size check
Bounding box vs the size the text implies, when two separate readings of the text agree (2% or 0.5 mm).
05
One repair round
Each failure goes back once with the verifier's message.
Results
Counted in your browser from the committed verdicts; the same numbers as the README table.
Explore the parts
Each dot is one check: execution, then size. Pick a part to see its prompt, both programs and the solid they build.
drag to rotate · scroll to zoom
What these numbers do and do not show
Read before quoting a number.
Overall size onlyThe size check compares the bounding box. A part with the right box and a missing hole passes it.
Repair saw the answer boxSize feedback quoted the expected box, so a size match after repair is measured by the check that guided it. A run failure fixed after repair is the stronger signal.
Noise floorThe same 151 prompts generated twice differ by 2 run failures and 1 size mismatch; the repair effect is well outside that.
Broken symbols in the input