Files
OpenNest/docs/performance/fill-verification.md
T
ajandClaude Opus 5.5 683ece4d3c perf(engine): run pair grid overlap checks only where the ranking needs them
FillLinear.FillDeferred returns a grid whose final overlap check has not
run; Resolve runs it and returns exactly what Fill returns, and Fill is
FillDeferred(...).Resolve(). PairFiller sorts a candidate's grids with
CompareByCount, which answers from the counts a pending grid can still
have (its tiling or its bounding-box fallback) and runs the check only
when they could order the pair differently; the early abort and the
remnant pass resolve the grids they use. Every answer equals the checked
comparison, so List.Sort makes the same moves, ties included, and
layouts are bit-identical. A check that throws during the sort surfaces
unwrapped. On the bracket fill this skips about 35% of the pair grids'
final checks by work.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 01:36:19 -05:00

14 KiB

Fill performance verification

Opt-in synthetic measurements (OpenNest.Tests/Fill/FillPerformanceTests.cs):

OPENNEST_RUN_FILL_PERF=1 dotnet test OpenNest.Tests/OpenNest.Tests.csproj -c Release \
  --filter 'Category=FillPerformance' --logger 'console;verbosity=detailed'

Only the exact value 1 enables these tests; otherwise they skip. In PowerShell, set $env:OPENNEST_RUN_FILL_PERF = '1' before the same dotnet test command, then run Remove-Item Env:OPENNEST_RUN_FILL_PERF afterward.

The category covers comparer, group-pattern, rotated-pattern, extents-column, feature-extraction, no-model angle, and FillLinear offset-geometry workloads; individual filters match benchmark method names in FillPerformanceTests.cs. Overlap checks are measured separately by OverlapCheck_ReportsPolygonPairsAndGridChecks in OpenNest.Tests/Fill/OverlapCheckPerformanceTests.cs (same category). Keep harness, inputs, warmups and batches identical before/after; exclude setup/assertions from timing. Comparer/extents allocations are synchronous and current-thread only; parallel group fills omit allocation totals. No timing CI gates or whole-job speedup claims. Keep raw results and run-specific reports outside source control; this guide documents the reusable verification procedure.

Quantity-limited private fills

Irregular's private block proposals opt into quantity-limited Row/Column stripe fills. FillLinear.MaxParts is a soft row limit: the first primary row stays complete, and perpendicular copies stop after reaching the requested count. The final fill still trims its winner to the exact quantity, keeping or dropping interlocked pairs whole (ShrinkFiller.TrimToCount; a pair is split only for an odd count with no unpaired part left). Existing overlap fallbacks and block spacing certification remain active; fallback tiling can exceed the soft limit. Nonpositive limits remain uncapped. Ordinary Lattice-strategy fills and the zero-quantity cutout lattice keep their full-area behavior.

This can change which private block candidate wins; it is not a promise of identical Irregular layouts. Verify fulfillment, bounds, spacing and rotation policy through the full pipeline, rather than treating a faster canceled run as acceptance.

dotnet test OpenNest.Tests/OpenNest.Tests.csproj -c Release \
  --filter 'FullyQualifiedName~FillLinearMaxPartsTests|FullyQualifiedName~StripeFillerTests|FullyQualifiedName~EngineRefactorSmokeTests'
dotnet test OpenNest.Engine.Tests/OpenNest.Engine.Tests.csproj -c Release \
  --filter 'FullyQualifiedName~IrregularBlockTests|FullyQualifiedName~CutoutLatticeFillTests'
python3 scripts/check-synthetic-nests.py

For a supplied slow job, retain a bounded before/after trace, rebuilt probe and input hashes, and a completion run with all demand placed and zero independent validation violations. Bound cancellation and the outer process watchdog separately. Do not extend a known unchanged timeout just to obtain a baseline duration.

FillLinear unchanged-row validation baseline

# Both configurations must characterize unchanged production; unset perf flag skips.
dotnet test OpenNest.Tests/OpenNest.Tests.csproj -c Debug \
  --filter 'FullyQualifiedName~FillLinearValidation' --logger 'trx;LogFileName=linear-validation-debug.trx'
dotnet test OpenNest.Tests/OpenNest.Tests.csproj -c Release \
  --filter 'FullyQualifiedName~FillLinearValidation' --logger 'trx;LogFileName=linear-validation-release.trx'
OPENNEST_RUN_FILL_PERF=1 dotnet test OpenNest.Tests/OpenNest.Tests.csproj -c Release \
  --filter 'FullyQualifiedName~FillLinearValidation_ReportsStripeAndControls' \
  --logger 'console;verbosity=detailed' --logger 'trx;LogFileName=linear-validation-perf.trx'

FillLinearValidationPerformanceTests.cs is self-contained against base APIs: byte-copy and hash the same file in both trees. It times complete production Fill calls (not frozen code or overlap-only helpers): horizontal/vertical stripes, full grid, partial-only additions, and single-seed stripe. The closed concave fixture gives counts 8 / 20 / 36 / 30 / 8; grid is row 8 + 28 additions, partial-only is row 20 + 10 (no full row). Vertical-stripe and partial-only each gain a part of the copy before the pattern (FillLinear.LeadingParts moves a tiling into the room past its far end to fit them); the frozen-oracle comparisons and work-count assertions below turn it off and keep the earlier 8 / 19 / 36 / 29 / 8, and every oracle fixture also checks that the default fill holds at least the frozen count, inside the area and without overlaps. Two 100-call warmups and seven 200-call batches per mode alternate forward/reverse mode order. Retain every raw tick/time/current-thread allocation row, including warmups; setup/assertions/output are excluded, count/last-result consumption is identical. No timing or reduced-work assertions belong in this baseline harness.

PreStep3FillLinear.cs freezes production FillLinear at 1b23ad79f2, separately from the older LegacyFillLinear. Reverse only the documented type/constructor rename, namespace/visibility/import additions, nullable directive and import formatting; retain a zero-diff comparison against git show <base>:OpenNest.Engine/Fill/FillLinear.cs and SHA-256 hashes. Do not refactor the oracle or use it for before timings. FillLinearValidationReuseTests compares ordered IEEE-754 poses/bounds, CNC values/program sharing, drawing references and immutable inputs through both public Fill overloads, including concurrent calls on one filler. Valid fixtures assert closed positive-area material and no overlap. The separate invalid overlapping-seed case intentionally remains invalid; in Debug its listener confirms Step1 and Step2 fallback pair (0,1) with six parts at each step (zero perpendicular additions). It makes no claim that bbox fallback repairs the seeds.

Keep validation-work counter assertions Debug-only in FillCacheCollection, reset counters in finally, and retain the invalid fallback and single-seed/partial-only controls. Since the S1 guard shipped (re-delivered 2026-09-30 on dbcf7de), UnchangedRow_ReducesStep2OverlapWork asserts the reduced Step 2 work (horizontal stripe 8→4 exact / 4→2 preparations / 16→8 triangulations; vertical stripe 36→18 / 4→2 / 38→19) and GuardedControls_KeepEagerStep2Work plus PerpOnly_KeepsPerpOnlyCheckWork assert single-seed (7/1/8), Horizontal full-grid (36/4/44, matching the 2026-09-27 table), partial-only (36/4/38) and PerpOnly exact-call (H=9/V=3) work stay at base on the current tree. The work-count methods compile entirely within DEBUG — Release counter properties read zero, and empty passing shells would misreport coverage. The invalid fallback control additionally pins production's exact-call total at 2 so a wrongly skipped Step 2 check cannot hide behind a layout match. Whole-job exact preservation needs a separately repeated serial oracle (fresh process, DOTNET_PROCESSOR_COUNT=1, verified one-worker thread-pool cap, solve on that worker); ordinary parallel layouts can differ on the same tree, and the serial gate does not replace concurrent differential tests.

Debug behavior/skipped-work checks:

dotnet test OpenNest.Tests/OpenNest.Tests.csproj -c Debug \
  --filter 'FullyQualifiedName~DefaultFillComparerWorkTests|FullyQualifiedName~FillHelpersTests|FullyQualifiedName~FillExtentsTests|FullyQualifiedName~StrategyOverlapTests|FullyQualifiedName~FillLinearGeometryReuseTests|FullyQualifiedName~CollisionOverlapOnlyTests|FullyQualifiedName~PartOverlapCheckerTests'

PerfCounters.FillScoreComputations, PartBoundaryPreparations, PartBoundsUpdates, OffsetPerimeterEntities, FeatureBitmaskCells, CrossingPointScans, OverlapPolygonPreparations, and PolygonTriangulations increments compile away in Release: zero Release counters prove nothing. OverlapPolygonPreparations counts overlap-preparation starts (material extraction), not completed polygons: Part.Intersects counts both parts on every call, PartOverlapChecker counts once per distinct Program. PolygonTriangulations counts Collision triangulations; the checker triangulates a part at most once per check, and only after a bounding-box hit. HasOverlap, Check and the checker clip CollisionTriangles (each ear's open vertices and box sides, without a Polygon per triangle) in reused buffers with the polygon path's arithmetic; CollisionOverlapOnlyTests holds them bitwise to Collision.Triangulate and their verdicts to LegacyCollision. On a 30-copy fill of an 80-vertex arc-cornered bracket this cut a 15 s Lattice fill to about 4 s with identical poses: per-triangle Polygon entities (an interlocked ID each) and per-clip lists dominated. Serialize counter assertions in FillCacheCollection and reset in finally. Keep OpenNest.Tests/Fill/LegacyFillExtents.cs, OpenNest.Tests/Fill/LegacyFillLinear.cs, OpenNest.Tests/Geometry/LegacyCollision.cs, and OpenNest.Tests/Fill/LegacyPartOverlap.cs frozen for differential tests (never route them through production helpers), not production or before timings; measure the actual baseline production code.

Predictor regression checks: dotnet test OpenNest.Tests/OpenNest.Tests.csproj -c Release --filter "FullyQualifiedName~AngleCandidateBuilderTests|FullyQualifiedName~AnglePredictorTests|FullyQualifiedName~FeatureExtractorTests" (repeat in Debug for bitmap counters). IrregularAngles_ReportsWarmNoModelPath measures the public builder with a missing model and skips when a model is installed; never remove real model files to benchmark. FeatureExtraction_ReportsFullAndScalarOnly measures extraction separately.

Predictor availability uses the same one-attempt session initialization as inference. Publish completion only after assignment or definitive failure; concurrent callers must wait for the outcome. The builder skips extraction when unavailable and requests scalar-only features when available. Tests use isolated loaders/prediction doubles, not evidence of real ONNX inference.

Whole-job before/after comparisons use OpenNest.Benchmark with a *.manifest.json corpus and --parallel 1. Retain input hashes, exact commands, raw results, validity, fulfillment and cost for both revisions outside source control. Record baseline failures rather than treating them as regressions or tuning the corpus around them; overlapping timing ranges are inconclusive, not proof of unchanged performance.

Deferred grid checks and stripe convergence reuse

Both leave every layout bit-identical; they only skip work whose outcome is never used or already known. FillLinear.FillDeferred returns a DeferredGrid whose final overlap check (Step 2, or the perpendicular-only row) has not run; Resolve runs it, and Fill is FillDeferred(...).Resolve(), so eager callers do the same work as before. Step 1's check stays eager because it decides how the grid is built. While pending, a grid's count is either its tiling's or its bounding-box fallback's (FallbackCount, which can be lower or higher). PairFiller.EvaluateCandidate sorts with CompareByCount, which answers each comparison from those counts and runs a check only when they could order the pair differently; the early abort and Phase 2 resolve the grids they use. Every answer equals the checked comparison, so List.Sort (unstable) makes the same moves, ties included. Other FillLinear callers stay eager: a stripe trajectory feeds every row's verdict into its next angle, and the Linear, remnant and pattern sweeps rank with a pluggable IFillComparer that offers no count bound.

StripeConvergenceCache lives in the pipeline's FillContext.SharedState and reaches pair remnant strips through PairFiller. It keeps each stripe convergence by pair, drawing, axis, variant, span, spacing and row cap, the pair's turned bounds by angle, and each estimate row by angle (one thread tiles a row while others wait). Row and Column converge identical layouts, remnant strips along a full side repeat spans, and trajectories revisit angles; each value is a pure function of its key. DeferredGridCheckTests holds both to the eager results.

Entity identity and hot-test configuration

Use -c Release for routine cross-platform test runs, matching CI. Continue running counter-based regressions explicitly in Debug; do not treat fewer Release test cases as lost coverage without checking the DEBUG-only methods and individual skipped outcomes.

Entity IDs are opaque persistence keys, not security tokens or layout ordering keys. New entities use one process-random GUID prefix and an interlocked 48-bit sequence; clones receive fresh IDs. The sequence uses little-endian bytes independent of host byte order and throws on exhaustion rather than wrapping. Explicit restored/imported IDs remain authoritative, including suppressed source-entity IDs. EntityIdentityTests covers mixed entity types, concurrent creation, clone identity, serialization and the real .nest writer/reader.

For an identity-generation performance change, retain an untouched before tree, use identical harness bytes for both trees and compare full serialized NestJobResult SHA-256 hashes for deterministic, mixed curved/concave and zero-spacing jobs. Independently require zero layout violations. Verify each tree repeats its own hashes before using before/after equality as an oracle. If parallel output is unstable, use fresh processes with DOTNET_PROCESSOR_COUNT=1, verify a one-worker thread-pool cap and solve on that worker; this serial gate does not replace concurrent identity regressions. Measure solve time separately from validation and output serialization, interleave before/after processes and retain every batch. Re-profile the Release deterministic job to confirm per-entity Guid.NewGuid work is removed before claiming the measured gain.