test(problem-nests): guard the plate fill engine with a fill area

Fills had no corpus coverage, so a fill-engine change could lose parts on
a real nest without any check noticing. Nests whose checks: line names
fill are now filled plate by plate the way Fill Plate does (parts cleared,
cut-offs kept, the plate's own size, spacing, edges, quadrant and clamps)
and held to a "fill: 1=33" part-count baseline, with an optional strategy
("fill (Strip): 1=33", Lattice by default). Fewer parts, a layout that
fails the layout check, or a timeout fails; more parts are only reported.
A missing or mismatched baseline prints the observed line to paste.

AGENTS.md now sends fill-engine changes to the fill area only, instead of
never running the corpus for fills.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
ajandClaude Opus 5.5 committed 2026-11-18 08:37:09 -05:00
1 parent cacfdcf5dc
commit 48af12c544
3 files changed
+218 -6

No files matched your search

+1 -1
View File
@@ -32,7 +32,7 @@ dotnet test OpenNest.WinForms.Tests/OpenNest.WinForms.Tests.csproj
On Windows, prefer `scripts/Test-Hidden.ps1 <dotnet test args>` for WinForms tests: it runs them on a hidden desktop so no windows appear or take focus; see [hidden-desktop tests](docs/hidden-desktop-tests.md).
Plan Cutting and nesting-engine changes need the [problem-nest corpus](docs/problem-nests.md) (`Category=ProblemNests`, opt-in via `OPENNEST_PROBLEM_NESTS_SERVER`), which holds every nest that ever caused a problem; engine changes run only its `auto-nest` area. Never run it for fills, UI, posts or file formats. Limit it to the areas the change touches with `OPENNEST_PROBLEM_NESTS_CHECKS` (table in that doc); `part-order` and `entries` plan every nest. No plate may rank below its baseline, and none may join or climb the near-budget list compared with master's run. Keep desktop-dependent tests in `OpenNest.WinForms.Tests`, never add a WinForms reference to `OpenNest.Tests`. Optional CHR fixtures use local `OpenNest.Tests/test-config.json` and skip when absent. On Linux, build Windows projects with `-p:EnableWindowsTargeting=true`; this is not Windows runtime verification. The headless console builds independently with `dotnet build OpenNest.Console/OpenNest.Console.csproj`.
Plan Cutting, nesting-engine and fill-engine changes need the [problem-nest corpus](docs/problem-nests.md) (`Category=ProblemNests`, opt-in via `OPENNEST_PROBLEM_NESTS_SERVER`), which holds every nest that ever caused a problem; engine changes run only its `auto-nest` area, fill changes only its `fill` area (a best-fit change runs both). Never run it for UI, posts or file formats. Limit it to the areas the change touches with `OPENNEST_PROBLEM_NESTS_CHECKS` (table in that doc); `part-order` and `entries` plan every nest. No plate may rank below its baseline, and none may join or climb the near-budget list compared with master's run. Keep desktop-dependent tests in `OpenNest.WinForms.Tests`, never add a WinForms reference to `OpenNest.Tests`. Optional CHR fixtures use local `OpenNest.Tests/test-config.json` and skip when absent. On Linux, build Windows projects with `-p:EnableWindowsTargeting=true`; this is not Windows runtime verification. The headless console builds independently with `dotnet build OpenNest.Console/OpenNest.Console.csproj`.
Releases: follow [the release procedure](docs/releasing.md) and `scripts/Publish-Windows.ps1`; workflow artifacts are candidates, not published releases. GitHub (`ajisaacs/OpenNest`) is the primary repository; Gitea is a read-only backup mirror.
@@ -3,7 +3,10 @@ using System.Text.RegularExpressions;
using OpenNest.CNC.CuttingStrategy;
using OpenNest.CutOffs;
using OpenNest.Data;
using OpenNest.Engine;
using OpenNest.Engine.CuttingPlanning;
using OpenNest.Engine.Jobs;
using OpenNest.Engine.Jobs.Placement;
using OpenNest.Engine.Sequencing;
using OpenNest.IO;
using Xunit.Abstractions;
@@ -37,11 +40,14 @@ public partial class ProblemNestRegressionTests(ITestOutputHelper output)
/// <summary>The area of a nest whose problem was Auto Nest's layout, not its cutting plan.</summary>
internal const string AutoNestArea = "auto-nest";
/// <summary>The area of a nest whose problem was a plate fill's layout.</summary>
internal const string FillArea = "fill";
/// <summary>The planner areas a nest can guard; docs/problem-nests.md says which changes touch each.</summary>
internal static readonly string[] PlannerAreas = ["part-order", "entries", "contours", "geometry-checks", "cut-offs", "grid"];
/// <summary>Every area a nest can guard: the planner's and Auto Nest's.</summary>
internal static readonly string[] Areas = [.. PlannerAreas, AutoNestArea];
/// <summary>Every area a nest can guard: the planner's, Auto Nest's and the plate fill's.</summary>
internal static readonly string[] Areas = [.. PlannerAreas, AutoNestArea, FillArea];
/// <summary>
/// Areas whose changes reorder the search on every plate, not just the nests tagged with them:
@@ -175,6 +181,152 @@ public partial class ProblemNestRegressionTests(ITestOutputHelper output)
CommentLine(comments, AutoNestLine) is { } line && Regex.Match(line, @"\d+") is { Success: true } count
? int.Parse(count.Value) : null;
/// <summary>
/// Fills each plate listed in the "fill: 1=33" baseline of every nest that guards
/// <see cref="FillArea"/> with that plate's one drawing, the way Fill Plate does: the plate's
/// parts are cleared and the strategy fills its work area with the plate's own size, spacing,
/// edge spacing, quadrant and clamps. Fails when a plate places fewer parts than its baseline,
/// its layout breaks the layout check or the fill times out; more parts are only reported.
/// </summary>
[SkippableFact]
public async Task ProblemNests_FillNoWorseThanTheirBaselines()
{
var server = Environment.GetEnvironmentVariable(ServerVariable);
Skip.If(string.IsNullOrWhiteSpace(server), $"Set {ServerVariable} to the OpenNest server URL to fill its problem nests.");
var named = Checks(Environment.GetEnvironmentVariable(ChecksVariable));
Skip.If(named.Length != 0 && !named.Contains(FillArea), $"{string.Join(", ", named)} does not name {FillArea}.");
using var repository = new RemoteNestRepository(server!);
var records = await ProblemNests(repository);
var limit = TimeSpan.FromSeconds(int.TryParse(Environment.GetEnvironmentVariable(TimeoutVariable), out var seconds)
&& seconds > 0 ? seconds : 120);
var failures = new List<string>();
var filled = 0;
foreach (var record in records)
{
if (!Checks(CommentLine(record.Comments, ChecksLine)).Contains(FillArea))
continue;
filled++;
var bytes = await repository.GetFileAsync(record.Id)
?? throw new InvalidOperationException($"'{record.Name}' has no file.");
FillAll(record.Name, bytes, FillBaseline(record.Comments), limit, failures);
}
Skip.If(filled == 0, $"No nest on {server} checks {FillArea}.");
Assert.True(failures.Count == 0, string.Join(Environment.NewLine, failures));
}
// Fills the baseline's plates, or every single-drawing plate when there is no baseline.
private void FillAll(string name, byte[] bytes, FillBaselineLine baseline, TimeSpan limit, List<string> failures)
{
using var stream = new MemoryStream(bytes);
var nest = new NestReader(stream).Read();
var strategy = baseline?.Strategy ?? PlateFillService.DefaultStrategy;
var label = strategy == PlateFillService.DefaultStrategy ? name : $"{name} ({strategy})";
// A line naming no plate is no baseline: every single-drawing plate is filled to record one.
var counts = baseline?.Counts ?? new Dictionary<int, int>();
var observed = new SortedDictionary<int, string>();
var mismatched = new List<string>();
for (var index = 0; index < nest.Plates.Count; index++)
{
var number = index + 1;
var listed = counts.ContainsKey(number);
if (counts.Count != 0 && !listed)
continue;
var plate = nest.Plates[index];
var drawings = plate.Parts.Select(p => p.BaseDrawing).Where(d => !d.IsCutOff)
.Distinct<Drawing>(ReferenceEqualityComparer.Instance).ToList();
if (drawings.Count != 1)
{
var why = $"plate {number} holds {drawings.Count} drawings; a fill baseline takes plates holding one";
if (listed)
mismatched.Add(why);
else if (drawings.Count > 1)
output.WriteLine($"{label}: skipped {why}.");
continue;
}
var watch = Stopwatch.StartNew();
var (parts, problem) = Fill(strategy, plate, drawings[0], limit);
var expected = listed ? counts[number] : (int?)null;
output.WriteLine($"{label}: plate {number} {problem ?? $"{parts} parts"} (baseline {expected?.ToString() ?? "none"}), "
+ $"{watch.Elapsed.TotalSeconds:0.0} s");
if (problem != null)
{
failures.Add($"{label} plate {number}: {problem}");
continue;
}
observed[number] = parts.ToString();
if (parts < expected)
failures.Add($"{label} plate {number}: {parts} parts, worse than baseline {expected}.");
else if (parts > expected)
output.WriteLine($" better than {expected}; update the baseline.");
}
foreach (var number in counts.Keys.Where(n => n < 1 || n > nest.Plates.Count))
mismatched.Add($"the nest has no plate {number}");
var line = FillLinePrefix(strategy) + " " + string.Join(" ", observed.Select(p => $"{p.Key}={p.Value}"));
if (mismatched.Count != 0)
failures.Add($"{label}: baseline plates do not match the nest's plates ({string.Join("; ", mismatched)}). "
+ $"Observed: {line}");
else if (counts.Count == 0)
failures.Add(observed.Count == 0
? $"{label} has no fill baseline and no plate holding a single drawing to fill."
: $"{label} has no fill baseline. Add this line to its comments: {line}");
}
// Fill Plate on a cleared plate: the part count, or why the fill cannot be kept.
private static (int Parts, string Problem) Fill(string strategy, Plate plate, Drawing drawing, TimeSpan limit)
{
// Cut-offs stay: Fill Plate fills around the plate's cut-offs.
for (var i = plate.Parts.Count - 1; i >= 0; i--)
if (!plate.Parts[i].BaseDrawing.IsCutOff)
plate.Parts.RemoveAt(i);
using var cancellation = new CancellationTokenSource(limit);
List<Part> parts;
try
{
parts = PlateFillService.FillItem(strategy, plate, new NestItem { Drawing = drawing, Quantity = 0 },
plate.WorkArea(), null, cancellation.Token);
}
catch (OperationCanceledException)
{
return (0, Timeout);
}
if (cancellation.IsCancellationRequested)
return (parts.Count, Timeout);
var violations = NestLayoutCheck.Validate([(plate, parts)],
new Dictionary<Drawing, (string Name, int Quantity)>(ReferenceEqualityComparer.Instance)
{
[drawing] = (drawing.Name, int.MaxValue),
});
return violations.Count != 0
? (parts.Count, $"invalid layout ({parts.Count} parts): " + string.Join(" | ", violations.Take(3)))
: (parts.Count, null);
}
/// <summary>A "fill: 1=33" or "fill (Strip): 1=33" baseline: the strategy and each plate's part count.</summary>
internal sealed record FillBaselineLine(string Strategy, IReadOnlyDictionary<int, int> Counts);
/// <summary>
/// The first "fill: 1=33 3=120" comment line, plate number (counted from 1) to part count, with
/// the strategy named in parentheses ("fill (Strip): 1=33") or Lattice; null when there is none.
/// An unknown strategy throws, so a misspelt line cannot quietly fill with another.
/// </summary>
internal static FillBaselineLine FillBaseline(string comments)
{
var match = (comments ?? "").Split('\n').Select(l => FillLinePattern().Match(l.Trim())).FirstOrDefault(m => m.Success);
if (match == null)
return null;
var strategy = match.Groups["strategy"].Success
? PlateFillService.ResolveStrategy(match.Groups["strategy"].Value.Trim())
: PlateFillService.DefaultStrategy;
return new FillBaselineLine(strategy, FillEntry().Matches(match.Groups["counts"].Value)
.ToDictionary(m => int.Parse(m.Groups[1].Value), m => int.Parse(m.Groups[2].Value)));
}
internal static string FillLinePrefix(string strategy) =>
strategy == PlateFillService.DefaultStrategy ? $"{FillArea}:" : $"{FillArea} ({strategy}):";
// Plates near their budget pass today but are the first a reordered search pushes over it.
private void ReportNearBudget(List<(string Plate, double Share)> usage)
{
@@ -347,6 +499,12 @@ public partial class ProblemNestRegressionTests(ITestOutputHelper output)
[GeneratedRegex(@"(\d+)\s*=\s*(\w+)")]
private static partial Regex BaselineEntry();
[GeneratedRegex(@"^fill\s*(?:\((?<strategy>[^)]*)\))?\s*:(?<counts>.*)$", RegexOptions.IgnoreCase)]
private static partial Regex FillLinePattern();
[GeneratedRegex(@"(\d+)\s*=\s*(\d+)")]
private static partial Regex FillEntry();
}
public class ProblemNestBaselineTests
@@ -393,6 +551,37 @@ public class ProblemNestBaselineTests
Assert.Equal(["auto-nest"], ProblemNestRegressionTests.Checks("auto-nest"));
}
[Fact]
public void FillBaseline_ReadsPlateCountsWithLatticeByDefault()
{
var baseline = ProblemNestRegressionTests.FillBaseline(
"Fill placed 30 before the fix.\r\nchecks: fill\nfill: 1=33 3 = 120\nnotes");
Assert.Equal("Lattice", baseline.Strategy);
Assert.Equal(new Dictionary<int, int> { [1] = 33, [3] = 120 }, baseline.Counts);
Assert.Equal(["fill"], ProblemNestRegressionTests.Checks("fill"));
Assert.Equal("fill:", ProblemNestRegressionTests.FillLinePrefix(baseline.Strategy));
}
[Fact]
public void FillBaseline_ReadsTheStrategyInParentheses()
{
var baseline = ProblemNestRegressionTests.FillBaseline("Fill (strip): 2=48");
Assert.Equal("Strip", baseline.Strategy);
Assert.Equal(new Dictionary<int, int> { [2] = 48 }, baseline.Counts);
Assert.Equal("fill (Strip):", ProblemNestRegressionTests.FillLinePrefix(baseline.Strategy));
Assert.Throws<NotSupportedException>(() => ProblemNestRegressionTests.FillBaseline("fill (Spiral): 1=10"));
}
[Fact]
public void FillBaseline_IgnoresOtherLinesAndIsNullWithoutOne()
{
Assert.Null(ProblemNestRegressionTests.FillBaseline("Fill placed 30.\nchecks: fill\nauto-nest: 2 sheets\nplanning: 1=Ready"));
Assert.Null(ProblemNestRegressionTests.FillBaseline(null));
Assert.Empty(ProblemNestRegressionTests.FillBaseline("fill:").Counts);
}
[Fact]
public void Wanted_PlansEveryNestForAreasThatReorderEveryPlate()
{
+26 -3
View File
@@ -4,8 +4,8 @@ Every real nest that ever caused a problem is kept on a dedicated OpenNest serve
nothing else, not in this repository: they carry customer geometry and names. Each nest's comments
say what went wrong and which commit fixed it, and its `checks:` line names the areas it guards, so
a change runs only the nests that guard what it touches: plan or nest them and confirm none got
worse. Most guard Plan Cutting; `auto-nest` nests guard the nesting engines. Fills, the UI, posts
and file formats never need the corpus.
worse. Most guard Plan Cutting; `auto-nest` nests guard the nesting engines and `fill` nests the
plate fill engine. The UI, posts and file formats never need the corpus.
## Areas
@@ -23,6 +23,7 @@ nest: a plate tagged with another area can still run out of budget after such a
| `cut-offs` | Cut-off segments in the plan (`CutOffInsertion`, `CutOffSegments`, `PlateSequencing` cut-off placement, automatic scrap cut-offs) |
| `grid` | Large plates: the aligned-grid serpentine, or any work done once per part or per pair that would grow on thousands of parts (`CuttingGridOrder`, per-part capture and checks) |
| `auto-nest` | Auto Nest's layout, not its cutting plan: the whole-job nesting engines (`OpenNest.Engine/NestingEngines/`) and what they share. Guarded by sheet count, not by planning |
| `fill` | The plate fill engine: `OpenNest.Engine/Fill/`, `OpenNest.Engine/Strategies/`, `OpenNest.Engine/BestFit/`, `PlateFiller` and `PlateFillService`. Guarded by part count per plate, not by planning. Best fits also feed Auto Nest, so a best-fit change runs `auto-nest` too |
A new planner area goes in `ProblemNestRegressionTests.PlannerAreas` and this table together. A misspelt area,
in a comment or the filter, fails the run rather than skipping nests.
@@ -67,6 +68,26 @@ fewer sheets are only reported. Run it with `OPENNEST_PROBLEM_NESTS_CHECKS=auto-
naming only planner areas skips it, and the planning test skips nests guarding only `auto-nest`
and runs not at all for a filter naming only `auto-nest`.
### Fill nests
`ProblemNests_FillNoWorseThanTheirBaselines` takes the nests whose `checks:` line names `fill`
and fills each plate its `fill: 1=33 3=120` comment line lists, the way **Fill Plate** does: the
plate's parts are removed (its cut-offs stay), and the strategy fills the plate's work area with
the plate's one drawing at unlimited quantity, using the plate's own size, spacing, edge spacing,
quadrant and clamps as saved. Lattice fills unless the line names another strategy in
parentheses: `fill (Strip): 1=33`. Each plate prints one line:
`<nest>: plate 1 33 parts (baseline 33), 61.2 s`.
A plate fails when it places fewer parts than its baseline, its layout breaks the layout check
(bounds, spacing, clamps), or the fill times out (the same per-plate limit as planning: 120 s, or
`OPENNEST_PROBLEM_NESTS_TIMEOUT` seconds). More parts are only reported; update the baseline.
A fill baseline takes plates holding one drawing: a listed plate holding several, or none, or a
plate number the nest lacks, fails and prints the observed line. A nest without the line fills
every plate holding one drawing and fails with the line to paste. A full Lattice fill of a large
plate takes about a minute, so run the fill area alone. Run it with
`OPENNEST_PROBLEM_NESTS_CHECKS=fill`; a filter naming only planner areas or `auto-nest` skips it,
and the planning and Auto Nest tests skip nests guarding only `fill`.
## Budget headroom
The planner's budget counts search expansions, not time, so a plate's outcome is repeatable, but
@@ -100,7 +121,9 @@ Plates use their own cutting parameters. A plate without them uses line lead-ins
and which commit fixed it, if any.
3. Add a baseline line to the comments, with plate numbers counted from the first plate:
`planning: 1=NoSolutionWithinBudget 2=BestEffort`. An Auto Nest problem takes
`auto-nest: 2 sheets` instead, the sheet count Auto Nest reaches today.
`auto-nest: 2 sheets` instead, the sheet count Auto Nest reaches today, and a fill problem
`fill: 1=33`, the part count each listed plate's fill reaches today (`fill (Strip): 1=33` for
a strategy other than Lattice). A fill plate must hold one drawing: the fill uses it.
To get the line, save without one and run the test: the failure prints the observed line.
Add a `checks:` line naming the [areas](#areas) the problem was in, such as `checks: part-order`.
4. Save. Every save keeps the earlier versions, so a baseline change can be reviewed later.