diff --git a/AGENTS.md b/AGENTS.md
index 239ab797..220fcdc8 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -30,7 +30,7 @@ dotnet test OpenNest.FrontEnd.Tests/OpenNest.FrontEnd.Tests.csproj -c Release
dotnet test OpenNest.WinForms.Tests/OpenNest.WinForms.Tests.csproj
```
-Only Plan Cutting changes need the [problem-nest corpus](docs/problem-nests.md) (`Category=ProblemNests`, opt-in via `OPENNEST_PROBLEM_NESTS_SERVER`); never run it for nesting engines, fills, UI, posts or file formats. Limit it to the areas the change touches with `OPENNEST_PROBLEM_NESTS_CHECKS` (table in that doc); no plate may rank below its baseline. Keep desktop-dependent tests in `OpenNest.WinForms.Tests`, never add a WinForms reference to `OpenNest.Tests`. Optional CHR fixtures use local `OpenNest.Tests/test-config.json` and skip when absent. On Linux, build Windows projects with `-p:EnableWindowsTargeting=true`; this is not Windows runtime verification. The headless console builds independently with `dotnet build OpenNest.Console/OpenNest.Console.csproj`.
+Only Plan Cutting changes need the [problem-nest corpus](docs/problem-nests.md) (`Category=ProblemNests`, opt-in via `OPENNEST_PROBLEM_NESTS_SERVER`); never run it for nesting engines, fills, UI, posts or file formats. Limit it to the areas the change touches with `OPENNEST_PROBLEM_NESTS_CHECKS` (table in that doc); `part-order` and `entries` plan every nest. No plate may rank below its baseline, and none may join or climb the near-budget list compared with master's run. Keep desktop-dependent tests in `OpenNest.WinForms.Tests`, never add a WinForms reference to `OpenNest.Tests`. Optional CHR fixtures use local `OpenNest.Tests/test-config.json` and skip when absent. On Linux, build Windows projects with `-p:EnableWindowsTargeting=true`; this is not Windows runtime verification. The headless console builds independently with `dotnet build OpenNest.Console/OpenNest.Console.csproj`.
Releases: follow [the release procedure](docs/releasing.md) and `scripts/Publish-Windows.ps1`; workflow artifacts are candidates, not published releases. GitHub (`ajisaacs/OpenNest`) is the primary repository; Gitea is a read-only backup mirror.
diff --git a/OpenNest.Tests/CuttingPlanning/ProblemNestRegressionTests.cs b/OpenNest.Tests/CuttingPlanning/ProblemNestRegressionTests.cs
index 568b2791..2b4538d6 100644
--- a/OpenNest.Tests/CuttingPlanning/ProblemNestRegressionTests.cs
+++ b/OpenNest.Tests/CuttingPlanning/ProblemNestRegressionTests.cs
@@ -14,7 +14,10 @@ namespace OpenNest.Tests.CuttingPlanning;
/// worse than its recorded baseline. A nest with a cut-off baseline is planned a second time with
/// its applied lead-ins and cut-offs cleared and default scrap cut-offs added. A "checks:" comment
/// line names the planner areas a nest guards; a run limited to some areas skips nests that guard
-/// none of them, and a nest without the line is always planned. Opt-in and private:
+/// none of them, and a nest without the line is always planned. A run touching the part order or
+/// entries plans every nest, since those change the search on every plate. Each plate's share of
+/// its expansion budget is reported, and plates near it are listed last: they are the ones a change
+/// to the search order tips into . Opt-in and private:
/// the nests and their baselines stay on the server, never in the repository. See docs/problem-nests.md.
///
[Trait("Category", "ProblemNests")]
@@ -31,6 +34,15 @@ public partial class ProblemNestRegressionTests(ITestOutputHelper output)
/// The planner areas a nest can guard; docs/problem-nests.md says which changes touch each.
internal static readonly string[] Areas = ["part-order", "entries", "contours", "geometry-checks", "cut-offs", "grid"];
+
+ ///
+ /// Areas whose changes reorder the search on every plate, not just the nests tagged with them:
+ /// a run naming one plans the whole corpus.
+ ///
+ internal static readonly string[] WholeCorpusAreas = ["part-order", "entries"];
+
+ /// A plate whose strict search spent at least this share of its budget is near it.
+ internal const double NearBudget = 0.8;
// A corpus is a handful of nests; more means the variable names a production archive.
internal const int MaximumNests = 200;
@@ -44,9 +56,14 @@ public partial class ProblemNestRegressionTests(ITestOutputHelper output)
var limit = TimeSpan.FromSeconds(int.TryParse(Environment.GetEnvironmentVariable(TimeoutVariable), out var seconds)
&& seconds > 0 ? seconds : 120);
Assert.True(records.Count > 0, $"No nests on {server}.");
- var wanted = Checks(Environment.GetEnvironmentVariable(ChecksVariable));
+ var named = Checks(Environment.GetEnvironmentVariable(ChecksVariable));
+ var wanted = Wanted(named);
+ if (wanted.Length != named.Length)
+ output.WriteLine($"{string.Join(", ", named.Intersect(WholeCorpusAreas))} change the search on every plate, "
+ + "so every nest is planned.");
var failures = new List();
+ var usage = new List<(string Plate, double Share)>();
foreach (var record in records)
{
var guards = Checks(CommentLine(record.Comments, ChecksLine));
@@ -60,20 +77,32 @@ public partial class ProblemNestRegressionTests(ITestOutputHelper output)
var bytes = await repository.GetFileAsync(record.Id)
?? throw new InvalidOperationException($"'{record.Name}' has no file.");
var baseline = Baseline(record.Comments);
- var observed = PlanAll(record.Name, bytes, false, baseline, limit, failures);
+ var observed = PlanAll(record.Name, bytes, false, baseline, limit, failures, usage);
if (baseline.Count == 0)
failures.Add($"{record.Name} has no baseline. Add this line to its comments: {Line(PlanningLine, observed)}");
if (Environment.GetEnvironmentVariable(CutOffsVariable) == "1")
- PlanAll(record.Name, bytes, true, [], limit, failures);
+ PlanAll(record.Name, bytes, true, [], limit, failures, usage);
else if (Baseline(record.Comments, CutOffPlanningLine) is { Count: > 0 } withCutOffs)
- PlanAll(record.Name, bytes, true, withCutOffs, limit, failures);
+ PlanAll(record.Name, bytes, true, withCutOffs, limit, failures, usage);
}
+ ReportNearBudget(usage);
Assert.True(failures.Count == 0, string.Join(Environment.NewLine, failures));
}
+ // Plates near their budget pass today but are the first a reordered search pushes over it.
+ private void ReportNearBudget(List<(string Plate, double Share)> usage)
+ {
+ var near = usage.Where(u => u.Share >= NearBudget).OrderByDescending(u => u.Share).ToList();
+ output.WriteLine(near.Count == 0
+ ? $"No plate's search used {NearBudget:P0} of its budget."
+ : $"Near the budget ({near.Count}; a change that adds plates here or raises their share is a regression):");
+ foreach (var (plate, share) in near)
+ output.WriteLine($" {plate}: {share:P0}");
+ }
+
// Plans each plate with parts and compares it with the baseline; plate number to outcome.
private SortedDictionary PlanAll(string name, byte[] bytes, bool cutOffs,
- Dictionary baseline, TimeSpan limit, List failures)
+ Dictionary baseline, TimeSpan limit, List failures, List<(string Plate, double Share)> usage)
{
using var stream = new MemoryStream(bytes);
var nest = new NestReader(stream).Read();
@@ -88,11 +117,18 @@ public partial class ProblemNestRegressionTests(ITestOutputHelper output)
var watch = Stopwatch.StartNew();
if (cutOffs)
AddScrapCutOffs(plate, number, nest.Units);
+ var budget = CuttingPlanBatch.PlateBudget(plate.Parts.Count);
var (outcome, planned) = Plan(plate, number, limit);
observed[number] = outcome;
var expected = baseline.GetValueOrDefault(number);
+ var share = BudgetShare(planned, budget);
+ if (share is { } used)
+ usage.Add(($"{label} plate {number}", used));
output.WriteLine($"{label} plate {number}: {outcome} (baseline {expected ?? "none"}), "
- + $"{planned?.Result.Expansions ?? 0} expansions, {watch.Elapsed.TotalSeconds:0.0} s");
+ + $"{planned?.Result.Expansions ?? 0} of {budget} expansions"
+ + (share is { } shown ? $" ({shown:P0})" : "")
+ + (planned?.KeptCurrentOrder == true ? ", new order ran out" : "")
+ + $", {watch.Elapsed.TotalSeconds:0.0} s");
if (expected == null)
continue;
if (Rank(outcome) < Rank(expected))
@@ -183,10 +219,23 @@ public partial class ProblemNestRegressionTests(ITestOutputHelper output)
return areas;
}
+ /// The areas a run plans: none, meaning every nest, when one changes the search on every plate.
+ internal static string[] Wanted(string[] named) => named.Intersect(WholeCorpusAreas).Any() ? [] : named;
+
// Every nest when no area is wanted; otherwise one guarding a wanted area, or one naming none.
internal static bool Selected(IReadOnlyCollection guards, IReadOnlyCollection wanted) =>
wanted.Count == 0 || guards.Count == 0 || guards.Intersect(wanted).Any();
+ ///
+ /// The share of the plate's budget its applied strict search spent; null when no strict search
+ /// found the route (a refusal, the safest-route fallback or the best-effort plan), since then
+ /// there is no margin left to measure.
+ ///
+ internal static double? BudgetShare(CuttingPlanPlateResult planned, int budget) =>
+ planned?.Result is { IsSafestRouteFallback: false, Expansions: > 0 } result && budget > 0
+ && result.Status is CuttingPlanStatus.Ready or CuttingPlanStatus.BestEffort
+ ? (double)result.Expansions / budget : null;
+
// Ready means replayed and overlap-clear; a ready route with overlapping parts is still blocked.
internal static string Outcome(CuttingPlanPlateResult planned) =>
planned.IsReady ? "Ready" : planned.IsRouteReady ? "Overlap" : planned.Result.Status.ToString();
@@ -249,6 +298,14 @@ public class ProblemNestBaselineTests
Assert.Throws(() => ProblemNestRegressionTests.Checks("part-order, lead-ins"));
}
+ [Fact]
+ public void Wanted_PlansEveryNestForAreasThatReorderEveryPlate()
+ {
+ Assert.Equal(["cut-offs", "grid"], ProblemNestRegressionTests.Wanted(["cut-offs", "grid"]));
+ Assert.Empty(ProblemNestRegressionTests.Wanted(["cut-offs", "part-order"]));
+ Assert.Empty(ProblemNestRegressionTests.Wanted(["entries"]));
+ }
+
[Fact]
public void Rank_OrdersReadyAboveBestEffortAboveBudgetAboveBlockedAboveTimeout()
{
diff --git a/docs/problem-nests.md b/docs/problem-nests.md
index d17c7f34..5b3b3f2f 100644
--- a/docs/problem-nests.md
+++ b/docs/problem-nests.md
@@ -9,7 +9,9 @@ posts and file formats never need it.
## Areas
Each nest names the areas it guards in a `checks:` comment line. Pick every area a change touches;
-when a change spans the planner or you cannot tell, run the whole corpus.
+when a change spans the planner or you cannot tell, run the whole corpus. `part-order` and
+`entries` change the order the search tries things on every plate, so naming either plans every
+nest: a plate tagged with another area can still run out of budget after such a change.
| Area | Change touches |
|------|----------------|
@@ -52,6 +54,21 @@ baseline, when a nest has no baseline, or when the baseline's plate numbers diff
nest's. A different outcome at the same or a better rank is only reported. When it is
intended, update the baseline.
+## Budget headroom
+
+The planner's budget counts search expansions, not time, so a plate's outcome is repeatable, but
+a plate that finds its route just inside the budget is fragile: any change to the order the
+search tries parts or entries can push it over, into `NoSolutionWithinBudget`, without touching
+the problem it guards. Each plate's line reports the expansions its applied strict search spent
+against the plate's budget (`12000 of 20000 expansions (60%)`), and whether the new order ran out
+before the current order was kept. Plates without a strict-search route (refusals, the
+safest-route fallback, best-effort plans) report no share.
+
+The run ends with the plates that spent at least 80% of their budget. They pass today and fail
+next. Compare that list with the same run on master: a change that adds plates to it or raises
+their share made the search worse, even when every plate still meets its baseline. Say so in the
+change, and fix it, or explain why the plate's search grew.
+
A nest whose comments also carry a `planning with cut-offs: 1=Ready 2=BestEffort` line is
planned a second time, as Plan Cutting does after **Clear Lead-ins & Cut-offs** and **Add scrap
cut-offs** at their defaults, and each plate is held to that line the same way. Set