Files
OpenNest/docs/performance/fill-verification.md
T
ajandClaude Opus 5.5 683ece4d3c perf(engine): run pair grid overlap checks only where the ranking needs them
FillLinear.FillDeferred returns a grid whose final overlap check has not
run; Resolve runs it and returns exactly what Fill returns, and Fill is
FillDeferred(...).Resolve(). PairFiller sorts a candidate's grids with
CompareByCount, which answers from the counts a pending grid can still
have (its tiling or its bounding-box fallback) and runs the check only
when they could order the pair differently; the early abort and the
remnant pass resolve the grids they use. Every answer equals the checked
comparison, so List.Sort makes the same moves, ties included, and
layouts are bit-identical. A check that throws during the sort surfaces
unwrapped. On the bracket fill this skips about 35% of the pair grids'
final checks by work.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 01:36:19 -05:00

90 lines
14 KiB
Markdown

# Fill performance verification
Opt-in synthetic measurements (`OpenNest.Tests/Fill/FillPerformanceTests.cs`):
```bash
OPENNEST_RUN_FILL_PERF=1 dotnet test OpenNest.Tests/OpenNest.Tests.csproj -c Release \
--filter 'Category=FillPerformance' --logger 'console;verbosity=detailed'
```
Only the exact value `1` enables these tests; otherwise they skip. In PowerShell, set `$env:OPENNEST_RUN_FILL_PERF = '1'` before the same `dotnet test` command, then run `Remove-Item Env:OPENNEST_RUN_FILL_PERF` afterward.
The category covers comparer, group-pattern, rotated-pattern, extents-column, feature-extraction, no-model angle, and FillLinear offset-geometry workloads; individual filters match benchmark method names in `FillPerformanceTests.cs`. Overlap checks are measured separately by `OverlapCheck_ReportsPolygonPairsAndGridChecks` in `OpenNest.Tests/Fill/OverlapCheckPerformanceTests.cs` (same category). Keep harness, inputs, warmups and batches identical before/after; exclude setup/assertions from timing. Comparer/extents allocations are synchronous and current-thread only; parallel group fills omit allocation totals. No timing CI gates or whole-job speedup claims. Keep raw results and run-specific reports outside source control; this guide documents the reusable verification procedure.
### Quantity-limited private fills
Irregular's private block proposals opt into quantity-limited Row/Column stripe fills.
`FillLinear.MaxParts` is a soft row limit: the first primary row stays complete, and
perpendicular copies stop after reaching the requested count. The final fill still
trims its winner to the exact quantity, keeping or dropping interlocked pairs whole
(`ShrinkFiller.TrimToCount`; a pair is split only for an odd count with no unpaired part
left). Existing overlap fallbacks and block spacing
certification remain active; fallback tiling can exceed the soft limit. Nonpositive
limits remain uncapped. Ordinary Lattice-strategy fills and the zero-quantity cutout lattice
keep their full-area behavior.
This can change which private block candidate wins; it is not a promise of identical
Irregular layouts. Verify fulfillment, bounds, spacing and rotation policy through
the full pipeline, rather than treating a faster canceled run as acceptance.
```sh
dotnet test OpenNest.Tests/OpenNest.Tests.csproj -c Release \
--filter 'FullyQualifiedName~FillLinearMaxPartsTests|FullyQualifiedName~StripeFillerTests|FullyQualifiedName~EngineRefactorSmokeTests'
dotnet test OpenNest.Engine.Tests/OpenNest.Engine.Tests.csproj -c Release \
--filter 'FullyQualifiedName~IrregularBlockTests|FullyQualifiedName~CutoutLatticeFillTests'
python3 scripts/check-synthetic-nests.py
```
For a supplied slow job, retain a bounded before/after trace, rebuilt probe and input
hashes, and a completion run with all demand placed and zero independent validation
violations. Bound cancellation and the outer process watchdog separately. Do not
extend a known unchanged timeout just to obtain a baseline duration.
### FillLinear unchanged-row validation baseline
```bash
# Both configurations must characterize unchanged production; unset perf flag skips.
dotnet test OpenNest.Tests/OpenNest.Tests.csproj -c Debug \
--filter 'FullyQualifiedName~FillLinearValidation' --logger 'trx;LogFileName=linear-validation-debug.trx'
dotnet test OpenNest.Tests/OpenNest.Tests.csproj -c Release \
--filter 'FullyQualifiedName~FillLinearValidation' --logger 'trx;LogFileName=linear-validation-release.trx'
OPENNEST_RUN_FILL_PERF=1 dotnet test OpenNest.Tests/OpenNest.Tests.csproj -c Release \
--filter 'FullyQualifiedName~FillLinearValidation_ReportsStripeAndControls' \
--logger 'console;verbosity=detailed' --logger 'trx;LogFileName=linear-validation-perf.trx'
```
`FillLinearValidationPerformanceTests.cs` is self-contained against base APIs: byte-copy and hash the same file in both trees. It times complete production `Fill` calls (not frozen code or overlap-only helpers): horizontal/vertical stripes, full grid, partial-only additions, and single-seed stripe. The closed concave fixture gives counts **8 / 20 / 36 / 30 / 8**; grid is row 8 + 28 additions, partial-only is row 20 + 10 (no full row). Vertical-stripe and partial-only each gain a part of the copy before the pattern (`FillLinear.LeadingParts` moves a tiling into the room past its far end to fit them); the frozen-oracle comparisons and work-count assertions below turn it off and keep the earlier **8 / 19 / 36 / 29 / 8**, and every oracle fixture also checks that the default fill holds at least the frozen count, inside the area and without overlaps. Two 100-call warmups and seven 200-call batches per mode alternate forward/reverse mode order. Retain every raw tick/time/current-thread allocation row, including warmups; setup/assertions/output are excluded, count/last-result consumption is identical. No timing or reduced-work assertions belong in this baseline harness.
`PreStep3FillLinear.cs` freezes production `FillLinear` at **1b23ad79f25d77fdd745bfea029de9dff2a91f6d**, separately from the older `LegacyFillLinear`. Reverse only the documented type/constructor rename, namespace/visibility/import additions, nullable directive and import formatting; retain a zero-diff comparison against `git show <base>:OpenNest.Engine/Fill/FillLinear.cs` and SHA-256 hashes. Do not refactor the oracle or use it for before timings. `FillLinearValidationReuseTests` compares ordered IEEE-754 poses/bounds, CNC values/program sharing, drawing references and immutable inputs through both public `Fill` overloads, including concurrent calls on one filler. Valid fixtures assert closed positive-area material and no overlap. The separate invalid overlapping-seed case intentionally remains invalid; in Debug its listener confirms Step1 and Step2 fallback pair `(0,1)` with six parts at each step (zero perpendicular additions). It makes no claim that bbox fallback repairs the seeds.
Keep validation-work counter assertions Debug-only in `FillCacheCollection`, reset counters in `finally`, and retain the invalid fallback and single-seed/partial-only controls. Since the S1 guard shipped (re-delivered 2026-09-30 on `dbcf7de`), `UnchangedRow_ReducesStep2OverlapWork` asserts the reduced Step 2 work (horizontal stripe 8→4 exact / 4→2 preparations / 16→8 triangulations; vertical stripe 36→18 / 4→2 / 38→19) and `GuardedControls_KeepEagerStep2Work` plus `PerpOnly_KeepsPerpOnlyCheckWork` assert single-seed (7/1/8), Horizontal full-grid (36/4/44, matching the 2026-09-27 table), partial-only (36/4/38) and PerpOnly exact-call (H=9/V=3) work stay at base on the current tree. The work-count methods compile entirely within DEBUG — Release counter properties read zero, and empty passing shells would misreport coverage. The invalid fallback control additionally pins production's exact-call total at 2 so a wrongly skipped Step 2 check cannot hide behind a layout match. Whole-job exact preservation needs a separately repeated serial oracle (fresh process, `DOTNET_PROCESSOR_COUNT=1`, verified one-worker thread-pool cap, solve on that worker); ordinary parallel layouts can differ on the same tree, and the serial gate does not replace concurrent differential tests.
Debug behavior/skipped-work checks:
```bash
dotnet test OpenNest.Tests/OpenNest.Tests.csproj -c Debug \
--filter 'FullyQualifiedName~DefaultFillComparerWorkTests|FullyQualifiedName~FillHelpersTests|FullyQualifiedName~FillExtentsTests|FullyQualifiedName~StrategyOverlapTests|FullyQualifiedName~FillLinearGeometryReuseTests|FullyQualifiedName~CollisionOverlapOnlyTests|FullyQualifiedName~PartOverlapCheckerTests'
```
`PerfCounters.FillScoreComputations`, `PartBoundaryPreparations`, `PartBoundsUpdates`, `OffsetPerimeterEntities`, `FeatureBitmaskCells`, `CrossingPointScans`, `OverlapPolygonPreparations`, and `PolygonTriangulations` increments compile away in Release: zero Release counters prove nothing. `OverlapPolygonPreparations` counts overlap-preparation starts (material extraction), not completed polygons: `Part.Intersects` counts both parts on every call, `PartOverlapChecker` counts once per distinct `Program`. `PolygonTriangulations` counts `Collision` triangulations; the checker triangulates a part at most once per check, and only after a bounding-box hit. `HasOverlap`, `Check` and the checker clip `CollisionTriangle`s (each ear's open vertices and box sides, without a `Polygon` per triangle) in reused buffers with the polygon path's arithmetic; `CollisionOverlapOnlyTests` holds them bitwise to `Collision.Triangulate` and their verdicts to `LegacyCollision`. On a 30-copy fill of an 80-vertex arc-cornered bracket this cut a 15 s Lattice fill to about 4 s with identical poses: per-triangle `Polygon` entities (an interlocked ID each) and per-clip lists dominated. Serialize counter assertions in `FillCacheCollection` and reset in `finally`. Keep `OpenNest.Tests/Fill/LegacyFillExtents.cs`, `OpenNest.Tests/Fill/LegacyFillLinear.cs`, `OpenNest.Tests/Geometry/LegacyCollision.cs`, and `OpenNest.Tests/Fill/LegacyPartOverlap.cs` frozen for differential tests (never route them through production helpers), not production or before timings; measure the actual baseline production code.
Predictor regression checks: `dotnet test OpenNest.Tests/OpenNest.Tests.csproj -c Release --filter "FullyQualifiedName~AngleCandidateBuilderTests|FullyQualifiedName~AnglePredictorTests|FullyQualifiedName~FeatureExtractorTests"` (repeat in Debug for bitmap counters). `IrregularAngles_ReportsWarmNoModelPath` measures the public builder with a missing model and skips when a model is installed; never remove real model files to benchmark. `FeatureExtraction_ReportsFullAndScalarOnly` measures extraction separately.
Predictor availability uses the same one-attempt session initialization as inference. Publish completion only after assignment or definitive failure; concurrent callers must wait for the outcome. The builder skips extraction when unavailable and requests scalar-only features when available. Tests use isolated loaders/prediction doubles, not evidence of real ONNX inference.
Whole-job before/after comparisons use `OpenNest.Benchmark` with a `*.manifest.json` corpus and `--parallel 1`. Retain input hashes, exact commands, raw results, validity, fulfillment and cost for both revisions outside source control. Record baseline failures rather than treating them as regressions or tuning the corpus around them; overlapping timing ranges are inconclusive, not proof of unchanged performance.
### Deferred grid checks and stripe convergence reuse
Both leave every layout bit-identical; they only skip work whose outcome is never used or already known. `FillLinear.FillDeferred` returns a `DeferredGrid` whose final overlap check (Step 2, or the perpendicular-only row) has not run; `Resolve` runs it, and `Fill` is `FillDeferred(...).Resolve()`, so eager callers do the same work as before. Step 1's check stays eager because it decides how the grid is built. While pending, a grid's count is either its tiling's or its bounding-box fallback's (`FallbackCount`, which can be lower or higher). `PairFiller.EvaluateCandidate` sorts with `CompareByCount`, which answers each comparison from those counts and runs a check only when they could order the pair differently; the early abort and Phase 2 resolve the grids they use. Every answer equals the checked comparison, so `List.Sort` (unstable) makes the same moves, ties included. Other `FillLinear` callers stay eager: a stripe trajectory feeds every row's verdict into its next angle, and the Linear, remnant and pattern sweeps rank with a pluggable `IFillComparer` that offers no count bound.
`StripeConvergenceCache` lives in the pipeline's `FillContext.SharedState` and reaches pair remnant strips through `PairFiller`. It keeps each stripe convergence by pair, drawing, axis, variant, span, spacing and row cap, the pair's turned bounds by angle, and each estimate row by angle (one thread tiles a row while others wait). Row and Column converge identical layouts, remnant strips along a full side repeat spans, and trajectories revisit angles; each value is a pure function of its key. `DeferredGridCheckTests` holds both to the eager results.
## Entity identity and hot-test configuration
Use `-c Release` for routine cross-platform test runs, matching CI. Continue running counter-based regressions explicitly in Debug; do not treat fewer Release test cases as lost coverage without checking the DEBUG-only methods and individual skipped outcomes.
Entity IDs are opaque persistence keys, not security tokens or layout ordering keys. New entities use one process-random GUID prefix and an interlocked 48-bit sequence; clones receive fresh IDs. The sequence uses little-endian bytes independent of host byte order and throws on exhaustion rather than wrapping. Explicit restored/imported IDs remain authoritative, including suppressed source-entity IDs. `EntityIdentityTests` covers mixed entity types, concurrent creation, clone identity, serialization and the real `.nest` writer/reader.
For an identity-generation performance change, retain an untouched before tree, use identical harness bytes for both trees and compare full serialized `NestJobResult` SHA-256 hashes for deterministic, mixed curved/concave and zero-spacing jobs. Independently require zero layout violations. Verify each tree repeats its own hashes before using before/after equality as an oracle. If parallel output is unstable, use fresh processes with `DOTNET_PROCESSOR_COUNT=1`, verify a one-worker thread-pool cap and solve on that worker; this serial gate does not replace concurrent identity regressions. Measure solve time separately from validation and output serialization, interleave before/after processes and retain every batch. Re-profile the Release deterministic job to confirm per-entity `Guid.NewGuid` work is removed before claiming the measured gain.