Files
OpenNest/docs/performance/fill-verification.md
T
aj 5e9eaf8a4c perf(fill): reuse validation for an unchanged row
Skip FillGrid's Step 2 overlap check only when Step 1 checked this exact
row clean and the perpendicular tiling appended zero parts, so gridResult
still holds the same Part objects in the same order and poses. The local
rowIsVerified flag is cleared by every Step 1 skip or bbox fallback, and
any nonzero append (including partial-copy parts) keeps the eager check.

Debug work assertions pin the reduced overlap work (horizontal stripe
8->4 exact calls, 4->2 preparations, 16->8 triangulations; vertical
stripe 36->18, 4->2, 38->19) and keep single-seed, Horizontal full-grid,
partial-only and PerpOnly work at base; the invalid overlapping-seed
control pins both fallback stages plus the retained exact-call total.
Three mutants (forced eager check, dropped flag, complete-rows-only)
are caught by their designated controls. Serial whole-job layout is
byte-identical to the frozen 62b5a8d0 oracle before and after.
2026-09-30 17:01:11 -04:00

7.7 KiB

Fill performance verification

Opt-in synthetic measurements (OpenNest.Tests/Fill/FillPerformanceTests.cs):

OPENNEST_RUN_FILL_PERF=1 dotnet test OpenNest.Tests/OpenNest.Tests.csproj -c Release \
  --filter 'Category=FillPerformance' --logger 'console;verbosity=detailed'

Only the exact value 1 enables these tests; otherwise they skip. In PowerShell, set $env:OPENNEST_RUN_FILL_PERF = '1' before the same dotnet test command, then run Remove-Item Env:OPENNEST_RUN_FILL_PERF afterward.

The category covers comparer, group-pattern, rotated-pattern, extents-column, feature-extraction, no-model angle, and FillLinear offset-geometry workloads; individual filters match benchmark method names in FillPerformanceTests.cs. Overlap checks are measured separately by OverlapCheck_ReportsPolygonPairsAndGridChecks in OpenNest.Tests/Fill/OverlapCheckPerformanceTests.cs (same category). Keep harness, inputs, warmups and batches identical before/after; exclude setup/assertions from timing. Comparer/extents allocations are synchronous and current-thread only; parallel group fills omit allocation totals. No timing CI gates or whole-job speedup claims. Keep raw results and run-specific reports outside source control; this guide documents the reusable verification procedure.

FillLinear unchanged-row validation baseline

# Both configurations must characterize unchanged production; unset perf flag skips.
dotnet test OpenNest.Tests/OpenNest.Tests.csproj -c Debug \
  --filter 'FullyQualifiedName~FillLinearValidation' --logger 'trx;LogFileName=linear-validation-debug.trx'
dotnet test OpenNest.Tests/OpenNest.Tests.csproj -c Release \
  --filter 'FullyQualifiedName~FillLinearValidation' --logger 'trx;LogFileName=linear-validation-release.trx'
OPENNEST_RUN_FILL_PERF=1 dotnet test OpenNest.Tests/OpenNest.Tests.csproj -c Release \
  --filter 'FullyQualifiedName~FillLinearValidation_ReportsStripeAndControls' \
  --logger 'console;verbosity=detailed' --logger 'trx;LogFileName=linear-validation-perf.trx'

FillLinearValidationPerformanceTests.cs is self-contained against base APIs: byte-copy and hash the same file in both trees. It times complete production Fill calls (not frozen code or overlap-only helpers): horizontal/vertical stripes, full grid, partial-only additions, and single-seed stripe. The closed concave fixture gives counts 8 / 19 / 36 / 29 / 8; grid is row 8 + 28 additions, partial-only is row 19 + 10 (no full row). Two 100-call warmups and seven 200-call batches per mode alternate forward/reverse mode order. Retain every raw tick/time/current-thread allocation row, including warmups; setup/assertions/output are excluded, count/last-result consumption is identical. No timing or reduced-work assertions belong in this baseline harness.

PreStep3FillLinear.cs freezes production FillLinear at 1b23ad79f2, separately from the older LegacyFillLinear. Reverse only the documented type/constructor rename, namespace/visibility/import additions, nullable directive and import formatting; retain a zero-diff comparison against git show <base>:OpenNest.Engine/Fill/FillLinear.cs and SHA-256 hashes. Do not refactor the oracle or use it for before timings. FillLinearValidationReuseTests compares ordered IEEE-754 poses/bounds, CNC values/program sharing, drawing references and immutable inputs through both public Fill overloads, including concurrent calls on one filler. Valid fixtures assert closed positive-area material and no overlap. The separate invalid overlapping-seed case intentionally remains invalid; in Debug its listener confirms Step1 and Step2 fallback pair (0,1) with six parts at each step (zero perpendicular additions). It makes no claim that bbox fallback repairs the seeds.

Keep validation-work counter assertions Debug-only in FillCacheCollection, reset counters in finally, and retain the invalid fallback and single-seed/partial-only controls. Since the S1 guard shipped (re-delivered 2026-09-30 on dbcf7de), UnchangedRow_ReducesStep2OverlapWork asserts the reduced Step 2 work (horizontal stripe 8→4 exact / 4→2 preparations / 16→8 triangulations; vertical stripe 36→18 / 4→2 / 38→19) and GuardedControls_KeepEagerStep2Work plus PerpOnly_KeepsPerpOnlyCheckWork assert single-seed (7/1/8), Horizontal full-grid (36/4/44, matching the 2026-09-27 table), partial-only (36/4/38) and PerpOnly exact-call (H=9/V=3) work stay at base on the current tree. The work-count methods compile entirely within DEBUG — Release counter properties read zero, and empty passing shells would misreport coverage. The invalid fallback control additionally pins production's exact-call total at 2 so a wrongly skipped Step 2 check cannot hide behind a layout match. Whole-job exact preservation needs a separately repeated serial oracle (fresh process, DOTNET_PROCESSOR_COUNT=1, verified one-worker thread-pool cap, solve on that worker); ordinary parallel layouts can differ on the same tree, and the serial gate does not replace concurrent differential tests.

Debug behavior/skipped-work checks:

dotnet test OpenNest.Tests/OpenNest.Tests.csproj -c Debug \
  --filter 'FullyQualifiedName~DefaultFillComparerWorkTests|FullyQualifiedName~FillHelpersTests|FullyQualifiedName~FillExtentsTests|FullyQualifiedName~StrategyOverlapTests|FullyQualifiedName~FillLinearGeometryReuseTests|FullyQualifiedName~CollisionOverlapOnlyTests|FullyQualifiedName~PartOverlapCheckerTests'

PerfCounters.FillScoreComputations, PartBoundaryPreparations, PartBoundsUpdates, OffsetPerimeterEntities, FeatureBitmaskCells, CrossingPointScans, OverlapPolygonPreparations, and PolygonTriangulations increments compile away in Release: zero Release counters prove nothing. OverlapPolygonPreparations counts overlap-preparation starts (material extraction), not completed polygons: Part.Intersects counts both parts on every call, PartOverlapChecker counts once per distinct Program. PolygonTriangulations counts Collision triangulations; the checker triangulates a part at most once per check, and only after a bounding-box hit. Serialize counter assertions in FillCacheCollection and reset in finally. Keep OpenNest.Tests/Fill/LegacyFillExtents.cs, OpenNest.Tests/Fill/LegacyFillLinear.cs, OpenNest.Tests/Geometry/LegacyCollision.cs, and OpenNest.Tests/Fill/LegacyPartOverlap.cs frozen for differential tests (never route them through production helpers), not production or before timings; measure the actual baseline production code.

Predictor regression checks: dotnet test OpenNest.Tests/OpenNest.Tests.csproj -c Release --filter "FullyQualifiedName~AngleCandidateBuilderTests|FullyQualifiedName~AnglePredictorTests|FullyQualifiedName~FeatureExtractorTests" (repeat in Debug for bitmap counters). IrregularAngles_ReportsWarmNoModelPath measures the public builder with a missing model and skips when a model is installed; never remove real model files to benchmark. FeatureExtraction_ReportsFullAndScalarOnly measures extraction separately.

Predictor availability uses the same one-attempt session initialization as inference. Publish completion only after assignment or definitive failure; concurrent callers must wait for the outcome. The builder skips extraction when unavailable and requests scalar-only features when available. Tests use isolated loaders/prediction doubles, not evidence of real ONNX inference.

Whole-job before/after comparisons use OpenNest.Benchmark with a *.manifest.json corpus and --parallel 1. Retain input hashes, exact commands, raw results, validity, fulfillment and cost for both revisions outside source control. Record baseline failures rather than treating them as regressions or tuning the corpus around them; overlapping timing ranges are inconclusive, not proof of unchanged performance.