This repository has been archived on 2026-09-29. You can view files and clone it. You cannot open issues or pull requests or push a commit.
Files
OpenNest-Engines/_Template/BENCH-RULES.md
T
aj d13ee28653 docs(template): retire the model competition and open engines to each other
Engines now target job lanes; a change lands when it beats the lane's current
best on the benchmark. Reading and borrowing from other engines is allowed with
credit, replacing the isolation/disqualification rule.
2026-09-29 20:49:16 -04:00

3.7 KiB

Bench rules

These rules apply to every model building or optimizing an OpenNest engine. Engines are no longer a competition between models: each engine targets a job lane (rectangles, circles, rings, irregular), and a change lands only when it beats that lane's current best on the benchmark. README.md in this folder covers what counts as an engine; this file covers how the run works.

Your workspace

  • You are working in a copy of OpenNest. Your engine is Engines/OpenNest.Engine.__NAME__/.
  • Other engines in OpenNest-Engines are open to read, compare against and borrow from. Credit borrowed ideas in your report. Do not edit another engine's folder in the same change.
  • Do not edit OpenNest.Core, OpenNest.Engine, OpenNest.Benchmark or anything else outside your engine folder, except for throwaway tools under tools/ and scratch work under bench-local/ (see below). If shared code has a bug that blocks you, work around it in your engine and describe the bug in your report.

Version control

  • At the start, if this copy is not already a git repository, run git init in the copy's root and commit everything as Initial bench copy before changing anything.
  • Commit every time the engine reaches a working state (it builds and the tests pass), with a message that says what changed and why, plus any benchmark numbers you measured.
  • When an experiment makes things worse, go back to the last good commit rather than patching forward. Do not rewrite or squash history; the commit log is part of the record of your run.
  • Do not add a git remote or push anywhere.

Real parts

  • Real production drawings are available read-only at /mnt/rogers/PEP Drawings/Archive (on hermes.lan) or Y:\Archive (on Windows); both are the same archive. Use them to build realistic benchmark jobs.
  • Never modify, move or delete anything in the archive.
  • Do not copy DXF files into your engine folder, and do not commit archive paths, file names or part numbers inside your engine folder. The engine folder may be published; these drawings are customer property. Keep manifests, results and notes that reference the archive under bench-local/ in the copy's root instead.
  • Acceptance also uses jobs you did not tune on. Tune for real parts in general, not for the specific drawings you tested with.

Tests and scoring

  • Engine.Testing/ is the shared, read-only test kit supplied beside engine folders in bench copies. Reference it for shapes, job construction, layout assertions and inherited engine contract tests; do not copy or edit it during an engine optimization run.
  • Placement must be deterministic: no clocks, unseeded randomness or environment variables may influence placement. Budgets count work. Wall time may stop work only through the host's cancellation token. Diagnostics must not affect placement decisions.
  • Keep the starter tests in tests/ and keep them passing. Add tests; do not weaken, skip or delete existing ones. If you believe an existing test is wrong, leave it and explain why in your report.
  • Engines are scored by OpenNest.Benchmark. Every layout goes through NestValidator; an invalid layout places nothing and pays the unplaced-part penalty, so validity comes before utilization.
  • Benchmark with --parallel 1 whenever you report timing.

Your report

When you finish, update README.md in your engine folder to replace the template text with:

  • the algorithm and why you chose it,
  • what you tried that did not work,
  • benchmark results (synthetic jobs only in the README; real-part results go in bench-local/),
  • any shared-code bugs or improvements you found, with measured numbers, so they can be upstreamed.