ajandClaude Opus 5.5
ffd0187fbb
Rules for the runs were being made up mid-run (git for rollback, access to the real-part drawing archive), so each model started under a different rule set. BENCH-RULES.md writes them down in one place, stamped into every new engine, so all models work under the same rules: workspace limits and no searching for other engines, git init plus commit-per-working-state, the archive as read-only with nothing customer-identifying kept in the (publishable) engine folder, tests may only be added to, and what the final report must cover. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
61 lines
3.0 KiB
Markdown
61 lines
3.0 KiB
Markdown
# Bench rules
|
|
|
|
These rules apply to every model building or optimizing an OpenNest engine, so results can be
|
|
compared fairly. `README.md` in this folder covers what counts as an independent engine; this
|
|
file covers how the run itself works.
|
|
|
|
## Your workspace
|
|
|
|
- You are working in a copy of OpenNest. Your engine is `Engines/OpenNest.Engine.__NAME__/`.
|
|
- Work only inside this copy. Do not search the rest of the filesystem, other checkouts of
|
|
OpenNest, the network, or any git remote for other nesting engines or earlier versions of
|
|
this project. Using another engine's code or results in any form is disqualifying.
|
|
- Do not edit `OpenNest.Core`, `OpenNest.Engine`, `OpenNest.Benchmark` or anything else outside
|
|
your engine folder, except for throwaway tools under `tools/` and scratch work under
|
|
`bench-local/` (see below). If shared code has a bug that blocks you, work around it in your
|
|
engine and describe the bug in your report.
|
|
|
|
## Version control
|
|
|
|
- At the start, if this copy is not already a git repository, run `git init` in the copy's root
|
|
and commit everything as `Initial bench copy` before changing anything.
|
|
- Commit every time the engine reaches a working state (it builds and the tests pass), with a
|
|
message that says what changed and why, plus any benchmark numbers you measured.
|
|
- When an experiment makes things worse, go back to the last good commit rather than
|
|
patching forward. Do not rewrite or squash history; the commit log is part of the record
|
|
of your run.
|
|
- Do not add a git remote or push anywhere.
|
|
|
|
## Real parts
|
|
|
|
- Real production drawings are available read-only at `/mnt/rogers/PEP Drawings/Archive`.
|
|
Use them to build realistic benchmark jobs.
|
|
- Never modify, move or delete anything in the archive.
|
|
- Do not copy DXF files into your engine folder, and do not commit archive paths, file names or
|
|
part numbers inside your engine folder. The engine folder may be published; these drawings
|
|
are customer property. Keep manifests, results and notes that reference the archive under
|
|
`bench-local/` in the copy's root instead.
|
|
- Final scoring also uses jobs you will not see. Tune for real parts in general, not for the
|
|
specific drawings you tested with.
|
|
|
|
## Tests and scoring
|
|
|
|
- Keep the starter tests in `tests/` and keep them passing. Add tests; do not weaken,
|
|
skip or delete existing ones. If you believe an existing test is wrong, leave it and explain
|
|
why in your report.
|
|
- Engines are scored by `OpenNest.Benchmark`. Every layout goes through `NestValidator`; an
|
|
invalid layout places nothing and pays the unplaced-part penalty, so validity comes before
|
|
utilization.
|
|
- Benchmark with `--parallel 1` whenever you report timing.
|
|
|
|
## Your report
|
|
|
|
When you finish, update `README.md` in your engine folder to replace the template text with:
|
|
|
|
- the algorithm and why you chose it,
|
|
- what you tried that did not work,
|
|
- benchmark results (synthetic jobs only in the README; real-part results go in
|
|
`bench-local/`),
|
|
- any shared-code bugs or improvements you found, with measured numbers, so they can be
|
|
upstreamed.
|