The benchmark is about to be used as the objective for LLM-designed
engines, and several gaps would have rewarded the wrong behavior:
- Ranking was utilization-first, so dropping awkward parts raised the
score. Rank valid > fully placed > cost > plates, where cost is
salvage-credited sheet area plus a largest-sheet penalty per unplaced
part; placing a part is never scored worse than omitting it.
- Salvage rate was ignored in scoring; cost now uses EstimateNetArea,
recomputed from job geometry rather than trusted from the engine.
- Rotation constraints were never validated. Add RotationPolicy.Allows
(shared with NestJobPlacementValidator) and check every placement.
- Returned sheets were trusted, so an engine could loosen spacing or
invent a size. Sheets must now match offered stock.
- Part-in-part placements were flagged as overlaps; spacing now accounts
for cutouts, with an X-sorted sweep to prune distant pairs.
- Summary averaged per-job percentages; it now sums areas and cost.
- --spacing and sheet sizes parsed with the current culture.
- Warn when .nest jobs offer only their original sheet sizes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Benchmark jobs could only come from .nest files. A JSON manifest now lists
DXF files with quantities (plus sheet sizes, spacing, edge spacing, quadrant
and per-part allowRotation), imported through CadImporter. DXF paths resolve
relative to the manifest; sheet sizes are required from the manifest or
--sheet-sizes and are read in the DXFs' own units. Folder scans pick up
*.nest and *.manifest.json, and invalid manifests fail loudly.
BenchmarkRunner now runs (job x engine) solves concurrently, capped by
--parallel N (CLI default 3; --parallel 1 is sequential). Results are written
by index so report order is unchanged. Concurrent solves compete for cores,
so Time(ms) is only clean at --parallel 1; the run prints a note when N > 1.
Also fixes --output for manifest jobs, which tried to read the manifest as a
.nest to copy metadata from.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
aj
