From 5061b41a5d0f89882d303bbbc924168484b6a374 Mon Sep 17 00:00:00 2001 From: AJ Isaacs Date: Sun, 20 Sep 2026 21:43:41 -0400 Subject: [PATCH] feat(benchmark): build jobs from DXF manifests and run solves in parallel Benchmark jobs could only come from .nest files. A JSON manifest now lists DXF files with quantities (plus sheet sizes, spacing, edge spacing, quadrant and per-part allowRotation), imported through CadImporter. DXF paths resolve relative to the manifest; sheet sizes are required from the manifest or --sheet-sizes and are read in the DXFs' own units. Folder scans pick up *.nest and *.manifest.json, and invalid manifests fail loudly. BenchmarkRunner now runs (job x engine) solves concurrently, capped by --parallel N (CLI default 3; --parallel 1 is sequential). Results are written by index so report order is unchanged. Concurrent solves compete for cores, so Time(ms) is only clean at --parallel 1; the run prints a note when N > 1. Also fixes --output for manifest jobs, which tried to read the manifest as a .nest to copy metadata from. Co-Authored-By: Claude Sonnet 5 --- CLAUDE.md | 3 +- OpenNest.Benchmark/BenchmarkRunner.cs | 66 ++++-- OpenNest.Benchmark/DxfManifestLoader.cs | 175 ++++++++++++++ OpenNest.Benchmark/JobLoader.cs | 16 +- OpenNest.Benchmark/Program.cs | 51 ++++- .../Benchmark/BenchmarkRunnerTests.cs | 168 ++++++++++++++ .../Benchmark/JobLoaderManifestTests.cs | 215 ++++++++++++++++++ OpenNest.Tests/OpenNest.Tests.csproj | 1 + README.md | 23 ++ 9 files changed, 690 insertions(+), 28 deletions(-) create mode 100644 OpenNest.Benchmark/DxfManifestLoader.cs create mode 100644 OpenNest.Tests/Benchmark/BenchmarkRunnerTests.cs create mode 100644 OpenNest.Tests/Benchmark/JobLoaderManifestTests.cs diff --git a/CLAUDE.md b/CLAUDE.md index 76abbac..7e5e869 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -82,8 +82,9 @@ Training data collection for ML angle prediction. `TrainingDatabase` stores per- Compares registered `INestingEngine` implementations against each other on real `.nest` files. Each engine solves the whole job — it owns its own multi-plate/size strategy rather than being handed one already-sized plate at a time. Fully generic — it never hardcodes drawing geometry, just reads whatever drawings/quantities/plate settings each input file already has. - `JobLoader` builds `BenchmarkJob`s from a `.nest` file or a folder of them via `NestReader`, using every drawing with `Quantity.Required > 0`. `--sheet-sizes` can sweep a fixed list of plate sizes instead of each file's own. +- `DxfManifestLoader` builds a `BenchmarkJob` from a JSON manifest (`sheetSizes`, `spacing`, `edgeSpacing`, `quadrant`, `parts[] { dxf, quantity, allowRotation }`) instead of a `.nest`, importing each DXF with `CadImporter.ImportDrawing`. DXF paths resolve relative to the manifest; sheet sizes are required (manifest or `--sheet-sizes`, which overrides). `allowRotation: false` locks rotation the same way `NestRunner` does. `JobLoader.Load` routes `*.json` inputs to it, and folder scans pick up `*.nest` plus `*.manifest.json` (plain `*.json` is ignored so `--output` reports are never read as manifests). Invalid manifests throw rather than being skipped. - `BenchmarkJob.BuildNestJob(maxPlates)` converts the job into a `NestJob`: one `NestJobPart` per requested drawing (via `DrawingJobMapper.FromDrawing`) and one `NestPlateStock` per candidate sheet size (unlimited quantity — the engine decides how many of each size it uses). -- `BenchmarkRunner` calls each engine's `INestingEngine.Solve(NestJob)` once per job, under a wall-clock timeout so a runaway or hanging engine can't stall the whole benchmark run, then materializes the result back into legacy `Plate`/`Part` objects via `NestResultMaterializer` for scoring. +- `BenchmarkRunner` fans the (job × engine) pairs out with `Parallel.ForEach` (`NoBuffering`, `MaxDegreeOfParallelism` from `--parallel`, CLI default 3, `Run`'s own default 1) and writes results by index so report order stays job-then-engine. Each solve builds its own `NestJob` snapshot and materialized drawings, so solves share no mutable drawing state. Concurrent solves compete for cores, so `Time(ms)` is only clean at `--parallel 1`. It calls each engine's `INestingEngine.Solve(NestJob)` once per job, under a wall-clock timeout so a runaway or hanging engine can't stall the whole benchmark run, then materializes the result back into legacy `Plate`/`Part` objects via `NestResultMaterializer` for scoring. - `NestValidator` checks the returned layout: every part inside `Plate.WorkArea()`, every pair at least `Plate.PartSpacing` apart (checked geometrically via each part's own world-space polygon, inflated by the spacing — works on arbitrary concave/holed shapes, not just bounding boxes), and no drawing over its requested quantity. An invalid, throwing, or timed-out run scores zero for that job. - Scoring matches `Plate.Utilization()` (placed drawing area / full sheet area, `Plate.Area()`). If an engine placed every requested part, ties are broken by fewer plates used (`Report`'s ranking rule) — using fewer sheets to do the same job wastes less material. - `--engines Name1,Name2` filters to specific registered engines (default: all); `--csv ` writes a flat per-job CSV alongside the console report. diff --git a/OpenNest.Benchmark/BenchmarkRunner.cs b/OpenNest.Benchmark/BenchmarkRunner.cs index 9ffb741..4b06a3a 100644 --- a/OpenNest.Benchmark/BenchmarkRunner.cs +++ b/OpenNest.Benchmark/BenchmarkRunner.cs @@ -1,8 +1,10 @@ using System; +using System.Collections.Concurrent; using System.Collections.Generic; using System.Diagnostics; using System.Linq; using System.Threading; +using System.Threading.Tasks; namespace OpenNest.Benchmark { @@ -28,28 +30,38 @@ namespace OpenNest.Benchmark IReadOnlyList engines, double salvageRate = 0, double minimumSalvageDimension = 0, - string outputDirectory = null + string outputDirectory = null, + int maxParallelism = 1 ) { - var results = new List(jobs.Count * engines.Count); - - foreach (var job in jobs) + var pairs = jobs.SelectMany(job => engines.Select(engine => (Job: job, Engine: engine))) + .ToList(); + var results = new JobResult[pairs.Count]; + var options = new ParallelOptions { - foreach (var engineInfo in engines) - { - results.Add( - RunOne( - job, - engineInfo, - salvageRate, - minimumSalvageDimension, - outputDirectory - ) - ); - } - } + MaxDegreeOfParallelism = System.Math.Max(1, maxParallelism), + }; - return results; + // NoBuffering hands out one pair at a time: solves run for seconds to minutes, + // so chunked partitioning would leave workers idle behind a slow engine. + Parallel.ForEach( + Partitioner.Create( + Enumerable.Range(0, pairs.Count), + EnumerablePartitionerOptions.NoBuffering + ), + options, + i => + results[i] = RunOne( + pairs[i].Job, + pairs[i].Engine, + salvageRate, + minimumSalvageDimension, + outputDirectory + ) + ); + + // Indexed writes keep the report in job-then-engine order whatever finishes first. + return results.ToList(); } private static JobResult RunOne( @@ -101,11 +113,19 @@ namespace OpenNest.Benchmark { System.IO.Directory.CreateDirectory(outputDirectory); // Keep names and job metadata for a useful inspectable output; never modify source. - var source = new OpenNest.IO.NestReader(job.SourceFile).Read(); - materialized.Nest.Name = source.Name; - materialized.Nest.Units = source.Units; - materialized.Nest.Material = source.Material; - materialized.Nest.Thickness = source.Thickness; + // Manifest jobs have no source nest to copy from, so they keep the job's name. + if (job.SourceFile.EndsWith(".nest", StringComparison.OrdinalIgnoreCase)) + { + var source = new OpenNest.IO.NestReader(job.SourceFile).Read(); + materialized.Nest.Name = source.Name; + materialized.Nest.Units = source.Units; + materialized.Nest.Material = source.Material; + materialized.Nest.Thickness = source.Thickness; + } + else + { + materialized.Nest.Name = job.Name; + } materialized.Nest.SalvageRate = salvageRate; foreach (var request in job.Requests) materialized.DrawingsByPartId[request.Drawing.Id.ToString()].Name = request diff --git a/OpenNest.Benchmark/DxfManifestLoader.cs b/OpenNest.Benchmark/DxfManifestLoader.cs new file mode 100644 index 0000000..071b6af --- /dev/null +++ b/OpenNest.Benchmark/DxfManifestLoader.cs @@ -0,0 +1,175 @@ +using System; +using System.Collections.Generic; +using System.IO; +using System.Linq; +using System.Text.Json; +using OpenNest.Geometry; +using OpenNest.IO; + +namespace OpenNest.Benchmark +{ + /// + /// Builds a BenchmarkJob from a JSON manifest that lists DXF files and the + /// quantity of each to nest. DXF paths resolve relative to the manifest. + /// Sheet sizes come from the manifest or the caller's override; unlike a + /// .nest file there is no plate to inherit them from, so a job with none is + /// an error. Sheet sizes must use the same units as the DXFs. + /// + public static class DxfManifestLoader + { + /// Suffix a manifest needs to be picked up when scanning a folder. + public const string FolderSuffix = ".manifest.json"; + + private static readonly JsonSerializerOptions JsonOptions = new() + { + PropertyNameCaseInsensitive = true, + ReadCommentHandling = JsonCommentHandling.Skip, + AllowTrailingCommas = true, + }; + + public static BenchmarkJob Load( + string manifestPath, + IReadOnlyList sheetSizeOverrides = null, + double? partSpacingOverride = null + ) + { + var manifest = ReadManifest(manifestPath); + var baseDir = Path.GetDirectoryName(Path.GetFullPath(manifestPath)); + + if (manifest.Parts == null || manifest.Parts.Count == 0) + throw new InvalidOperationException( + $"Manifest '{manifestPath}' has no parts. Add entries to \"parts\"." + ); + + var sizes = ResolveSheetSizes(manifest, sheetSizeOverrides, manifestPath); + var requests = manifest.Parts.Select(p => BuildRequest(p, baseDir)).ToList(); + + return new BenchmarkJob + { + SourceFile = manifestPath, + CandidateSizes = sizes, + EdgeSpacing = new Spacing(manifest.EdgeSpacing, manifest.EdgeSpacing), + PartSpacing = partSpacingOverride ?? manifest.Spacing, + Quadrant = manifest.Quadrant, + Requests = requests, + }; + } + + private static Manifest ReadManifest(string manifestPath) + { + try + { + return JsonSerializer.Deserialize( + File.ReadAllText(manifestPath), + JsonOptions + ) ?? throw new InvalidOperationException("The manifest is empty."); + } + catch (JsonException ex) + { + throw new InvalidOperationException( + $"Manifest '{manifestPath}' is not valid JSON: {ex.Message}", + ex + ); + } + } + + private static List ResolveSheetSizes( + Manifest manifest, + IReadOnlyList overrides, + string manifestPath + ) + { + if (overrides != null && overrides.Count > 0) + return overrides.ToList(); + + var sizes = new List(); + + foreach (var text in manifest.SheetSizes ?? new List()) + { + if (!Size.TryParse(text, out var size)) + throw new InvalidOperationException( + $"Manifest '{manifestPath}': could not parse sheet size '{text}' (expected e.g. \"48x96\")." + ); + + sizes.Add(size); + } + + if (sizes.Count == 0) + throw new InvalidOperationException( + $"Manifest '{manifestPath}' has no sheet sizes. Set \"sheetSizes\" or pass --sheet-sizes." + ); + + return sizes.Distinct().ToList(); + } + + private static DrawingRequest BuildRequest(ManifestPart part, string baseDir) + { + if (string.IsNullOrWhiteSpace(part.Dxf)) + throw new InvalidOperationException("A manifest part is missing \"dxf\"."); + + if (part.Quantity <= 0) + throw new InvalidOperationException( + $"Manifest part '{part.Dxf}': quantity must be greater than 0 (was {part.Quantity})." + ); + + var dxfPath = Path.GetFullPath(Path.Combine(baseDir, part.Dxf)); + + if (!File.Exists(dxfPath)) + throw new FileNotFoundException($"DXF file not found: {dxfPath}", dxfPath); + + Drawing drawing; + + try + { + drawing = CadImporter.ImportDrawing( + dxfPath, + new CadImportOptions { Quantity = part.Quantity } + ); + } + catch (Exception ex) + { + throw new InvalidOperationException($"Failed to import DXF: {dxfPath}", ex); + } + + if (drawing.Program == null || drawing.Program.Codes.Count == 0) + throw new InvalidOperationException($"Failed to import DXF: {dxfPath}"); + + // A zero legacy step means automatic rotation to DrawingJobMapper, so lock it explicitly. + if (!part.AllowRotation) + { + drawing.Constraints ??= new NestConstraints(); + drawing.Constraints.StepAngle = OpenNest.Math.Angle.TwoPI; + drawing.Constraints.StartAngle = 0; + drawing.Constraints.EndAngle = 0; + } + + var constraints = drawing.Constraints; + + return new DrawingRequest + { + Drawing = drawing, + Quantity = part.Quantity, + Priority = drawing.Priority, + StepAngle = constraints?.StepAngle ?? 0, + RotationStart = constraints?.StartAngle ?? 0, + RotationEnd = constraints?.EndAngle ?? 0, + }; + } + + private class Manifest + { + public List SheetSizes { get; set; } + public double Spacing { get; set; } + public double EdgeSpacing { get; set; } + public int Quadrant { get; set; } = 1; + public List Parts { get; set; } + } + + private class ManifestPart + { + public string Dxf { get; set; } + public int Quantity { get; set; } + public bool AllowRotation { get; set; } = true; + } + } +} diff --git a/OpenNest.Benchmark/JobLoader.cs b/OpenNest.Benchmark/JobLoader.cs index 2346659..d30d32e 100644 --- a/OpenNest.Benchmark/JobLoader.cs +++ b/OpenNest.Benchmark/JobLoader.cs @@ -28,6 +28,13 @@ namespace OpenNest.Benchmark foreach (var file in files) { + if (file.EndsWith(".json", StringComparison.OrdinalIgnoreCase)) + { + // Hand-written manifests fail loudly rather than being skipped like unreadable .nest files. + jobs.Add(DxfManifestLoader.Load(file, sheetSizeOverrides, partSpacingOverride)); + continue; + } + Nest nest; try @@ -79,7 +86,14 @@ namespace OpenNest.Benchmark if (Directory.Exists(inputPath)) { return Directory - .GetFiles(inputPath, "*.nest", SearchOption.AllDirectories) + .EnumerateFiles(inputPath, "*", SearchOption.AllDirectories) + .Where(f => + f.EndsWith(".nest", StringComparison.OrdinalIgnoreCase) + || f.EndsWith( + DxfManifestLoader.FolderSuffix, + StringComparison.OrdinalIgnoreCase + ) + ) .OrderBy(f => f, StringComparer.OrdinalIgnoreCase) .ToList(); } diff --git a/OpenNest.Benchmark/Program.cs b/OpenNest.Benchmark/Program.cs index 5bf5e02..00fafff 100644 --- a/OpenNest.Benchmark/Program.cs +++ b/OpenNest.Benchmark/Program.cs @@ -38,7 +38,7 @@ static class BenchmarkConsole if (jobs.Count == 0) { Console.Error.WriteLine( - "No benchmark jobs found (no .nest files with any drawing quantity > 0)." + "No benchmark jobs found (no .nest files with any drawing quantity > 0, or *.manifest.json files)." ); return 1; } @@ -83,12 +83,23 @@ static class BenchmarkConsole Console.WriteLine($"Engines: {string.Join(", ", engines.Select(e => e.Name))}"); + var solves = jobs.Count * engines.Count; + + if (options.Parallel > 1 && solves > 1) + { + Console.WriteLine( + $"Running up to {options.Parallel} solves at a time; Time(ms) is measured under that " + + "concurrent load. Use --parallel 1 for strictly isolated timings." + ); + } + var results = BenchmarkRunner.Run( jobs, engines, options.SalvageRate, options.MinimumSalvageDimension, - options.OutputDirectory + options.OutputDirectory, + options.Parallel ); Report.PrintDetailed(results); @@ -149,6 +160,15 @@ static class BenchmarkConsole o.OutputDirectory = args[++i]; break; + case "--parallel" when i + 1 < args.Length: + if (int.TryParse(args[++i], out var parallel) && parallel >= 1) + o.Parallel = parallel; + else + Console.Error.WriteLine( + $"Warning: --parallel needs a whole number >= 1, using {o.Parallel}" + ); + break; + case "--help": PrintUsage(); return null; @@ -215,7 +235,25 @@ static class BenchmarkConsole ); Console.Error.WriteLine(); Console.Error.WriteLine("Usage:"); - Console.Error.WriteLine(" OpenNest.Benchmark [options]"); + Console.Error.WriteLine( + " OpenNest.Benchmark [options]" + ); + Console.Error.WriteLine(); + Console.Error.WriteLine( + "A manifest.json builds a job straight from DXF files (paths relative to the manifest):" + ); + Console.Error.WriteLine( + " { \"sheetSizes\": [\"48x96\"], \"spacing\": 0.25, \"edgeSpacing\": 0.25, \"quadrant\": 1," + ); + Console.Error.WriteLine( + " \"parts\": [ { \"dxf\": \"a.dxf\", \"quantity\": 12 }, { \"dxf\": \"b.dxf\", \"quantity\": 4, \"allowRotation\": false } ] }" + ); + Console.Error.WriteLine( + "Sheet sizes must use the same units as the DXFs. A folder is scanned for *.nest and" + ); + Console.Error.WriteLine( + "*.manifest.json files. --sheet-sizes and --spacing override the manifest." + ); Console.Error.WriteLine(); Console.Error.WriteLine("Options:"); Console.Error.WriteLine( @@ -242,6 +280,12 @@ static class BenchmarkConsole Console.Error.WriteLine( " --output Save valid layouts as .nest plus detailed JSON reports" ); + Console.Error.WriteLine( + " --parallel Solves to run at once (default 3; 1 = strictly sequential," + ); + Console.Error.WriteLine( + " which gives the cleanest per-engine timings)" + ); Console.Error.WriteLine(" --help Show this message"); } @@ -255,5 +299,6 @@ static class BenchmarkConsole public string OutputDirectory; public double SalvageRate; public double MinimumSalvageDimension; + public int Parallel = 3; } } diff --git a/OpenNest.Tests/Benchmark/BenchmarkRunnerTests.cs b/OpenNest.Tests/Benchmark/BenchmarkRunnerTests.cs new file mode 100644 index 0000000..1edcf7b --- /dev/null +++ b/OpenNest.Tests/Benchmark/BenchmarkRunnerTests.cs @@ -0,0 +1,168 @@ +using OpenNest.Benchmark; + +namespace OpenNest.Tests.Benchmark; + +public sealed class BenchmarkRunnerTests : IDisposable +{ + private static readonly string SourceDxf = Path.Combine( + "Bending", + "TestData", + "4526 A14 PT11.dxf" + ); + + private readonly string _dir = Path.Combine( + Path.GetTempPath(), + "opennest-runner-" + Guid.NewGuid().ToString("N") + ); + + public BenchmarkRunnerTests() + { + Directory.CreateDirectory(_dir); + File.Copy(SourceDxf, Path.Combine(_dir, "part.dxf")); + } + + public void Dispose() + { + try + { + Directory.Delete(_dir, recursive: true); + } + catch (IOException) { } + } + + private List LoadJob() + { + var manifest = Path.Combine(_dir, "job.json"); + File.WriteAllText( + manifest, + """{ "sheetSizes": ["48x96"], "parts": [ { "dxf": "part.dxf", "quantity": 2 } ] }""" + ); + return JobLoader.Load(manifest); + } + + private static List FakeEngines( + ConcurrencyProbe probe, + params int[] delaysMs + ) => + delaysMs + .Select( + (delay, i) => + new NestingEngineInfo( + $"Engine{i}", + "test double", + () => new SleepingEngine(probe, delay) + ) + ) + .ToList(); + + [Fact] + public void Run_NeverExceedsMaxParallelism_ButReachesIt() + { + var probe = new ConcurrencyProbe(); + + var results = BenchmarkRunner.Run( + LoadJob(), + FakeEngines(probe, 200, 200, 200, 200, 200, 200), + maxParallelism: 2 + ); + + Assert.Equal(6, results.Count); + Assert.Equal(2, probe.Max); + } + + [Fact] + public void Run_WithMaxParallelismOne_RunsOneSolveAtATime() + { + var probe = new ConcurrencyProbe(); + + BenchmarkRunner.Run(LoadJob(), FakeEngines(probe, 50, 50, 50, 50), maxParallelism: 1); + + Assert.Equal(1, probe.Max); + } + + [Fact] + public void Run_KeepsResultsInJobThenEngineOrder_RegardlessOfCompletionOrder() + { + var probe = new ConcurrencyProbe(); + + // The first engine finishes last, so completion order differs from input order. + var results = BenchmarkRunner.Run( + LoadJob(), + FakeEngines(probe, 300, 10, 10, 10), + maxParallelism: 3 + ); + + Assert.Equal( + new[] { "Engine0", "Engine1", "Engine2", "Engine3" }, + results.Select(r => r.EngineName) + ); + } + + [Fact] + public void Run_WithOutputDirectory_WritesManifestJobsWithoutNeedingASourceNest() + { + var probe = new ConcurrencyProbe(); + var output = Path.Combine(_dir, "out"); + + var results = BenchmarkRunner.Run( + LoadJob(), + FakeEngines(probe, 0), + outputDirectory: output + ); + + var result = Assert.Single(results); + Assert.Null(result.Error); + Assert.True(result.Valid); + Assert.True(File.Exists(Path.Combine(output, "job-Engine0.nest"))); + Assert.True(File.Exists(Path.Combine(output, "job-Engine0.json"))); + } + + private sealed class ConcurrencyProbe + { + private int _current; + private int _max; + + public int Max => Volatile.Read(ref _max); + + public void Enter() + { + var now = Interlocked.Increment(ref _current); + int seen; + while ((seen = Volatile.Read(ref _max)) < now) + { + if (Interlocked.CompareExchange(ref _max, now, seen) == seen) + break; + } + } + + public void Exit() => Interlocked.Decrement(ref _current); + } + + /// Places nothing after sleeping, recording how many solves overlap. + private sealed class SleepingEngine(ConcurrencyProbe probe, int delayMs) : INestingEngine + { + public NestJobResult Solve( + NestJob job, + IProgress? progress = null, + CancellationToken token = default + ) + { + probe.Enter(); + try + { + Thread.Sleep(delayMs); + return new NestJobResult( + NestJobStatus.Incomplete, + NestJobStopReason.NoPlacementFound, + Array.Empty(), + job.Parts.Select(p => new PartFulfillment(p.Id, p.Quantity, 0, p.Quantity)), + job.Plates.Select(s => new StockUsage(s.Id, 0, null)) + ); + } + finally + { + probe.Exit(); + } + } + } +} diff --git a/OpenNest.Tests/Benchmark/JobLoaderManifestTests.cs b/OpenNest.Tests/Benchmark/JobLoaderManifestTests.cs new file mode 100644 index 0000000..ed4e86e --- /dev/null +++ b/OpenNest.Tests/Benchmark/JobLoaderManifestTests.cs @@ -0,0 +1,215 @@ +using OpenNest.Benchmark; +using OpenNest.Geometry; + +namespace OpenNest.Tests.Benchmark; + +public sealed class JobLoaderManifestTests : IDisposable +{ + private static readonly string SourceDxf = Path.Combine( + "Bending", + "TestData", + "4526 A14 PT11.dxf" + ); + + private readonly string _dir = Path.Combine( + Path.GetTempPath(), + "opennest-manifest-" + Guid.NewGuid().ToString("N") + ); + + public JobLoaderManifestTests() + { + Directory.CreateDirectory(Path.Combine(_dir, "parts")); + File.Copy(SourceDxf, Path.Combine(_dir, "parts", "bracket.dxf")); + File.Copy(SourceDxf, Path.Combine(_dir, "parts", "plate.dxf")); + } + + public void Dispose() + { + try + { + Directory.Delete(_dir, recursive: true); + } + catch (IOException) { } + } + + private string WriteManifest(string name, string json) + { + var path = Path.Combine(_dir, name); + File.WriteAllText(path, json); + return path; + } + + private const string ValidManifest = """ + { + "sheetSizes": ["48x96", "60x120"], + "spacing": 0.25, + "edgeSpacing": 0.5, + "quadrant": 2, + "parts": [ + { "dxf": "parts/bracket.dxf", "quantity": 12 }, + { "dxf": "parts/plate.dxf", "quantity": 4 } + ] + } + """; + + [Fact] + public void Manifest_ImportsDxfsWithQuantitiesAndJobSettings() + { + var path = WriteManifest("bench.json", ValidManifest); + + var job = Assert.Single(JobLoader.Load(path)); + + Assert.Equal("bench", job.Name); + Assert.Equal(new[] { 12, 4 }, job.Requests.Select(r => r.Quantity)); + Assert.Equal(16, job.TotalRequestedQuantity); + Assert.All(job.Requests, r => Assert.NotEmpty(r.Drawing.Program.Codes)); + Assert.Equal(new[] { new Size(48, 96), new Size(60, 120) }, job.CandidateSizes.ToArray()); + Assert.Equal(0.25, job.PartSpacing); + Assert.Equal(0.5, job.EdgeSpacing.Left); + Assert.Equal(0.5, job.EdgeSpacing.Top); + Assert.Equal(2, job.Quadrant); + } + + [Fact] + public void Manifest_DxfPathsResolveRelativeToTheManifestNotTheWorkingDirectory() + { + var path = WriteManifest("bench.json", ValidManifest); + var elsewhere = Directory.GetCurrentDirectory(); + Assert.NotEqual(_dir, elsewhere); + + var job = Assert.Single(JobLoader.Load(path)); + + Assert.Equal(2, job.Requests.Count); + } + + [Fact] + public void Overrides_ReplaceManifestSheetSizesAndSpacing() + { + var path = WriteManifest("bench.json", ValidManifest); + + var job = Assert.Single(JobLoader.Load(path, new[] { new Size(10, 20) }, 0.75)); + + Assert.Equal(new[] { new Size(10, 20) }, job.CandidateSizes.ToArray()); + Assert.Equal(0.75, job.PartSpacing); + } + + [Fact] + public void SheetSizesFromOverrideAreEnoughWhenManifestOmitsThem() + { + var path = WriteManifest( + "bench.json", + """{ "parts": [ { "dxf": "parts/bracket.dxf", "quantity": 1 } ] }""" + ); + + var job = Assert.Single(JobLoader.Load(path, new[] { new Size(48, 96) })); + + Assert.Equal(new[] { new Size(48, 96) }, job.CandidateSizes.ToArray()); + Assert.Equal(0, job.PartSpacing); + Assert.Equal(1, job.Quadrant); + } + + [Fact] + public void NoSheetSizesAnywhere_ThrowsAndMentionsSheetSizes() + { + var path = WriteManifest( + "bench.json", + """{ "parts": [ { "dxf": "parts/bracket.dxf", "quantity": 1 } ] }""" + ); + + var ex = Assert.Throws(() => JobLoader.Load(path)); + + Assert.Contains("sheet size", ex.Message, StringComparison.OrdinalIgnoreCase); + } + + [Fact] + public void UnparseableSheetSize_ThrowsNamingTheValue() + { + var path = WriteManifest( + "bench.json", + """{ "sheetSizes": ["huge"], "parts": [ { "dxf": "parts/bracket.dxf", "quantity": 1 } ] }""" + ); + + var ex = Assert.Throws(() => JobLoader.Load(path)); + + Assert.Contains("huge", ex.Message); + } + + [Fact] + public void MissingDxf_ThrowsNamingTheEntry() + { + var path = WriteManifest( + "bench.json", + """{ "sheetSizes": ["48x96"], "parts": [ { "dxf": "parts/nope.dxf", "quantity": 1 } ] }""" + ); + + var ex = Assert.Throws(() => JobLoader.Load(path)); + + Assert.Contains("nope.dxf", ex.Message); + } + + [Theory] + [InlineData(0)] + [InlineData(-3)] + public void NonPositiveQuantity_ThrowsNamingTheEntry(int quantity) + { + var path = WriteManifest( + "bench.json", + $$"""{ "sheetSizes": ["48x96"], "parts": [ { "dxf": "parts/bracket.dxf", "quantity": {{quantity}} } ] }""" + ); + + var ex = Assert.Throws(() => JobLoader.Load(path)); + + Assert.Contains("bracket.dxf", ex.Message); + Assert.Contains("quantity", ex.Message, StringComparison.OrdinalIgnoreCase); + } + + [Fact] + public void EmptyPartsList_Throws() + { + var path = WriteManifest("bench.json", """{ "sheetSizes": ["48x96"], "parts": [] }"""); + + var ex = Assert.Throws(() => JobLoader.Load(path)); + + Assert.Contains("parts", ex.Message, StringComparison.OrdinalIgnoreCase); + } + + [Fact] + public void AllowRotationFalse_LocksThatPartsRotationInTheNestJob() + { + var path = WriteManifest( + "bench.json", + """ + { + "sheetSizes": ["48x96"], + "parts": [ + { "dxf": "parts/bracket.dxf", "quantity": 2 }, + { "dxf": "parts/plate.dxf", "quantity": 2, "allowRotation": false } + ] + } + """ + ); + + var job = Assert.Single(JobLoader.Load(path)); + var nestJob = job.BuildNestJob(maxPlates: 4); + + Assert.Equal(RotationPolicyKind.Automatic, nestJob.Parts[0].Rotation.Kind); + + // The legacy lock (step 2π, start = end = 0) reaches the engine as a single-angle sweep at 0. + var locked = nestJob.Parts[1].Rotation; + Assert.Equal(RotationPolicyKind.BoundedSweep, locked.Kind); + Assert.Equal(0, locked.Start); + Assert.Equal(0, locked.End); + } + + [Fact] + public void Folder_LoadsManifestSuffixedFilesAndIgnoresOtherJson() + { + WriteManifest("a.manifest.json", ValidManifest); + WriteManifest("b.manifest.json", ValidManifest); + WriteManifest("results.json", """{ "not": "a manifest" }"""); + + var jobs = JobLoader.Load(_dir); + + Assert.Equal(new[] { "a.manifest", "b.manifest" }, jobs.Select(j => j.Name)); + } +} diff --git a/OpenNest.Tests/OpenNest.Tests.csproj b/OpenNest.Tests/OpenNest.Tests.csproj index 7d4c4df..cf0b302 100644 --- a/OpenNest.Tests/OpenNest.Tests.csproj +++ b/OpenNest.Tests/OpenNest.Tests.csproj @@ -23,6 +23,7 @@ + diff --git a/README.md b/README.md index 34cbb03..276d64d 100644 --- a/README.md +++ b/README.md @@ -205,6 +205,29 @@ dotnet run --project OpenNest.Benchmark/OpenNest.Benchmark.csproj -- job.nest \ --sheet-sizes 48x96,60x96,60x120,72x120,72x144 --engines Default,Astra,Claude --csv results.csv ``` +To benchmark straight from DXF files without building a `.nest` first, pass a JSON manifest listing each DXF and its quantity: + +```json +{ + "sheetSizes": ["48x96", "60x120"], + "spacing": 0.25, + "edgeSpacing": 0.25, + "quadrant": 1, + "parts": [ + { "dxf": "parts/bracket.dxf", "quantity": 12 }, + { "dxf": "parts/plate.dxf", "quantity": 4, "allowRotation": false } + ] +} +``` + +```bash +dotnet run --project OpenNest.Benchmark/OpenNest.Benchmark.csproj -- job.json --csv results.csv +``` + +DXF paths resolve relative to the manifest. Sheet sizes are required (from the manifest or `--sheet-sizes`, which overrides it) and must use the same units as the DXFs; `--spacing` likewise overrides `spacing`. `spacing`, `edgeSpacing` and `quadrant` default to 0, 0 and 1, and rotation is unconstrained unless a part sets `allowRotation: false`. A folder input is scanned for `*.nest` and `*.manifest.json` files. Unlike an unreadable `.nest`, an invalid manifest (missing DXF, bad quantity, no sheet sizes) stops the run with an error naming the problem. + +Each engine-on-job solve is independent, so `--parallel ` (default 3) runs up to `n` at once; `--parallel 1` is strictly sequential. Results and their order in the report are the same either way. The catch is timing: some solves use one core, others spread across many, and when concurrent solves compete for cores `Time(ms)` goes up (a short multi-threaded job showed 2–4× inflation at `--parallel 3`). Scores (utilization, plates, validity) are unaffected, so use `--parallel 1` when comparing speed. The 5-minute per-solve timeout is wall-clock, so contention can also push a slow engine over it. + An engine's layout is rejected (scoring zero for that job) if any part falls outside the work area, any two parts are closer than the required spacing, or a drawing gets more parts placed than requested. A run that doesn't finish within its time budget also scores zero, as a timeout. Custom competitor engines can be added by dropping a DLL implementing `INestingEngine` with a public parameterless constructor into the `Engines/` directory next to the benchmark executable; each one is registered under its own CLR type name. This is a separate plugin contract from the desktop app's `NestEngineRegistry`/`NestEngineBase` (which requires a `(Plate)` constructor) — a `NestEngineBase` plugin dropped into the benchmark's `Engines/` folder is silently skipped, since the benchmark only ever solves whole jobs.