LLM benchmark: 2D CAD → 3D model code
Loading…
The 2D drawings and what the model produced
Each card is one input drawing (DXF). The three rows are the three prompt strategies; the note explains, from reading the generated OpenSCAD, why a run failed.
Method
- 6 DXF drawings (easy to hard: holes, nested loops, bulge arcs, rectangular windows), each with an extrusion depth.
- 3 prompt strategies: raw-dxf (DXF text), structured (parsed entity JSON), fewshot (structured plus one worked example).
- The model writes OpenSCAD; OpenSCAD renders it to STL; volume and bounding box are compared with ground truth from this app's own DXF pipeline.
- Pass = renders, volume within 5%, bounding box within 2% of the largest dimension. Temperature 0, fixed seed, one run per cell.
- Executed on a GitHub-hosted CPU runner via Ollama. Source:
bench/in the repository.