feat(benchmark): build jobs from DXF manifests and run solves in parallel

Benchmark jobs could only come from .nest files. A JSON manifest now lists
DXF files with quantities (plus sheet sizes, spacing, edge spacing, quadrant
and per-part allowRotation), imported through CadImporter. DXF paths resolve
relative to the manifest; sheet sizes are required from the manifest or
--sheet-sizes and are read in the DXFs' own units. Folder scans pick up
*.nest and *.manifest.json, and invalid manifests fail loudly.

BenchmarkRunner now runs (job x engine) solves concurrently, capped by
--parallel N (CLI default 3; --parallel 1 is sequential). Results are written
by index so report order is unchanged. Concurrent solves compete for cores,
so Time(ms) is only clean at --parallel 1; the run prints a note when N > 1.

Also fixes --output for manifest jobs, which tried to read the manifest as a
.nest to copy metadata from.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

CLAUDE.md

@@ -82,8 +82,9 @@ Training data collection for ML angle prediction. `TrainingDatabase` stores per-
 Compares registered `INestingEngine` implementations against each other on real `.nest` files. Each engine solves the whole job — it owns its own multi-plate/size strategy rather than being handed one already-sized plate at a time. Fully generic — it never hardcodes drawing geometry, just reads whatever drawings/quantities/plate settings each input file already has.
 
 - `JobLoader` builds `BenchmarkJob`s from a `.nest` file or a folder of them via `NestReader`, using every drawing with `Quantity.Required > 0`. `--sheet-sizes` can sweep a fixed list of plate sizes instead of each file's own.
+- `DxfManifestLoader` builds a `BenchmarkJob` from a JSON manifest (`sheetSizes`, `spacing`, `edgeSpacing`, `quadrant`, `parts[] { dxf, quantity, allowRotation }`) instead of a `.nest`, importing each DXF with `CadImporter.ImportDrawing`. DXF paths resolve relative to the manifest; sheet sizes are required (manifest or `--sheet-sizes`, which overrides). `allowRotation: false` locks rotation the same way `NestRunner` does. `JobLoader.Load` routes `*.json` inputs to it, and folder scans pick up `*.nest` plus `*.manifest.json` (plain `*.json` is ignored so `--output` reports are never read as manifests). Invalid manifests throw rather than being skipped.
 - `BenchmarkJob.BuildNestJob(maxPlates)` converts the job into a `NestJob`: one `NestJobPart` per requested drawing (via `DrawingJobMapper.FromDrawing`) and one `NestPlateStock` per candidate sheet size (unlimited quantity — the engine decides how many of each size it uses).
-- `BenchmarkRunner` calls each engine's `INestingEngine.Solve(NestJob)` once per job, under a wall-clock timeout so a runaway or hanging engine can't stall the whole benchmark run, then materializes the result back into legacy `Plate`/`Part` objects via `NestResultMaterializer` for scoring.
+- `BenchmarkRunner` fans the (job × engine) pairs out with `Parallel.ForEach` (`NoBuffering`, `MaxDegreeOfParallelism` from `--parallel`, CLI default 3, `Run`'s own default 1) and writes results by index so report order stays job-then-engine. Each solve builds its own `NestJob` snapshot and materialized drawings, so solves share no mutable drawing state. Concurrent solves compete for cores, so `Time(ms)` is only clean at `--parallel 1`. It calls each engine's `INestingEngine.Solve(NestJob)` once per job, under a wall-clock timeout so a runaway or hanging engine can't stall the whole benchmark run, then materializes the result back into legacy `Plate`/`Part` objects via `NestResultMaterializer` for scoring.
 - `NestValidator` checks the returned layout: every part inside `Plate.WorkArea()`, every pair at least `Plate.PartSpacing` apart (checked geometrically via each part's own world-space polygon, inflated by the spacing — works on arbitrary concave/holed shapes, not just bounding boxes), and no drawing over its requested quantity. An invalid, throwing, or timed-out run scores zero for that job.
 - Scoring matches `Plate.Utilization()` (placed drawing area / full sheet area, `Plate.Area()`). If an engine placed every requested part, ties are broken by fewer plates used (`Report`'s ranking rule) — using fewer sheets to do the same job wastes less material.
 - `--engines Name1,Name2` filters to specific registered engines (default: all); `--csv <path>` writes a flat per-job CSV alongside the console report.

OpenNest.Benchmark/BenchmarkRunner.cs

@@ -1,8 +1,10 @@
 using System;
+using System.Collections.Concurrent;
 using System.Collections.Generic;
 using System.Diagnostics;
 using System.Linq;
 using System.Threading;
+using System.Threading.Tasks;
 
 namespace OpenNest.Benchmark
 {
@@ -28,28 +30,38 @@ namespace OpenNest.Benchmark
             IReadOnlyList<NestingEngineInfo> engines,
             double salvageRate = 0,
             double minimumSalvageDimension = 0,
-            string outputDirectory = null
+            string outputDirectory = null,
+            int maxParallelism = 1
         )
         {
-            var results = new List<JobResult>(jobs.Count * engines.Count);
+            var pairs = jobs.SelectMany(job => engines.Select(engine => (Job: job, Engine: engine)))
+                .ToList();
+            var results = new JobResult[pairs.Count];
+            var options = new ParallelOptions
+            {
+                MaxDegreeOfParallelism = System.Math.Max(1, maxParallelism),
+            };
 
-            foreach (var job in jobs)
+            // NoBuffering hands out one pair at a time: solves run for seconds to minutes,
-            {
+            // so chunked partitioning would leave workers idle behind a slow engine.
-                foreach (var engineInfo in engines)
+            Parallel.ForEach(
-                {
+                Partitioner.Create(
-                    results.Add(
+                    Enumerable.Range(0, pairs.Count),
-                        RunOne(
+                    EnumerablePartitionerOptions.NoBuffering
-                            job,
+                ),
-                            engineInfo,
+                options,
+                i =>
+                    results[i] = RunOne(
+                        pairs[i].Job,
+                        pairs[i].Engine,
                         salvageRate,
                         minimumSalvageDimension,
                         outputDirectory
                     )
             );
-                }
-            }
 
-            return results;
+            // Indexed writes keep the report in job-then-engine order whatever finishes first.
+            return results.ToList();
         }
 
         private static JobResult RunOne(
@@ -101,11 +113,19 @@ namespace OpenNest.Benchmark
                 {
                     System.IO.Directory.CreateDirectory(outputDirectory);
                     // Keep names and job metadata for a useful inspectable output; never modify source.
+                    // Manifest jobs have no source nest to copy from, so they keep the job's name.
+                    if (job.SourceFile.EndsWith(".nest", StringComparison.OrdinalIgnoreCase))
+                    {
                         var source = new OpenNest.IO.NestReader(job.SourceFile).Read();
                         materialized.Nest.Name = source.Name;
                         materialized.Nest.Units = source.Units;
                         materialized.Nest.Material = source.Material;
                         materialized.Nest.Thickness = source.Thickness;
+                    }
+                    else
+                    {
+                        materialized.Nest.Name = job.Name;
+                    }
                     materialized.Nest.SalvageRate = salvageRate;
                     foreach (var request in job.Requests)
                         materialized.DrawingsByPartId[request.Drawing.Id.ToString()].Name = request

OpenNest.Benchmark/DxfManifestLoader.cs

@@ -0,0 +1,175 @@
+using System;
+using System.Collections.Generic;
+using System.IO;
+using System.Linq;
+using System.Text.Json;
+using OpenNest.Geometry;
+using OpenNest.IO;
+
+namespace OpenNest.Benchmark
+{
+    /// <summary>
+    /// Builds a BenchmarkJob from a JSON manifest that lists DXF files and the
+    /// quantity of each to nest. DXF paths resolve relative to the manifest.
+    /// Sheet sizes come from the manifest or the caller's override; unlike a
+    /// .nest file there is no plate to inherit them from, so a job with none is
+    /// an error. Sheet sizes must use the same units as the DXFs.
+    /// </summary>
+    public static class DxfManifestLoader
+    {
+        /// <summary>Suffix a manifest needs to be picked up when scanning a folder.</summary>
+        public const string FolderSuffix = ".manifest.json";
+
+        private static readonly JsonSerializerOptions JsonOptions = new()
+        {
+            PropertyNameCaseInsensitive = true,
+            ReadCommentHandling = JsonCommentHandling.Skip,
+            AllowTrailingCommas = true,
+        };
+
+        public static BenchmarkJob Load(
+            string manifestPath,
+            IReadOnlyList<Size> sheetSizeOverrides = null,
+            double? partSpacingOverride = null
+        )
+        {
+            var manifest = ReadManifest(manifestPath);
+            var baseDir = Path.GetDirectoryName(Path.GetFullPath(manifestPath));
+
+            if (manifest.Parts == null || manifest.Parts.Count == 0)
+                throw new InvalidOperationException(
+                    $"Manifest '{manifestPath}' has no parts. Add entries to \"parts\"."
+                );
+
+            var sizes = ResolveSheetSizes(manifest, sheetSizeOverrides, manifestPath);
+            var requests = manifest.Parts.Select(p => BuildRequest(p, baseDir)).ToList();
+
+            return new BenchmarkJob
+            {
+                SourceFile = manifestPath,
+                CandidateSizes = sizes,
+                EdgeSpacing = new Spacing(manifest.EdgeSpacing, manifest.EdgeSpacing),
+                PartSpacing = partSpacingOverride ?? manifest.Spacing,
+                Quadrant = manifest.Quadrant,
+                Requests = requests,
+            };
+        }
+
+        private static Manifest ReadManifest(string manifestPath)
+        {
+            try
+            {
+                return JsonSerializer.Deserialize<Manifest>(
+                        File.ReadAllText(manifestPath),
+                        JsonOptions
+                    ) ?? throw new InvalidOperationException("The manifest is empty.");
+            }
+            catch (JsonException ex)
+            {
+                throw new InvalidOperationException(
+                    $"Manifest '{manifestPath}' is not valid JSON: {ex.Message}",
+                    ex
+                );
+            }
+        }
+
+        private static List<Size> ResolveSheetSizes(
+            Manifest manifest,
+            IReadOnlyList<Size> overrides,
+            string manifestPath
+        )
+        {
+            if (overrides != null && overrides.Count > 0)
+                return overrides.ToList();
+
+            var sizes = new List<Size>();
+
+            foreach (var text in manifest.SheetSizes ?? new List<string>())
+            {
+                if (!Size.TryParse(text, out var size))
+                    throw new InvalidOperationException(
+                        $"Manifest '{manifestPath}': could not parse sheet size '{text}' (expected e.g. \"48x96\")."
+                    );
+
+                sizes.Add(size);
+            }
+
+            if (sizes.Count == 0)
+                throw new InvalidOperationException(
+                    $"Manifest '{manifestPath}' has no sheet sizes. Set \"sheetSizes\" or pass --sheet-sizes."
+                );
+
+            return sizes.Distinct().ToList();
+        }
+
+        private static DrawingRequest BuildRequest(ManifestPart part, string baseDir)
+        {
+            if (string.IsNullOrWhiteSpace(part.Dxf))
+                throw new InvalidOperationException("A manifest part is missing \"dxf\".");
+
+            if (part.Quantity <= 0)
+                throw new InvalidOperationException(
+                    $"Manifest part '{part.Dxf}': quantity must be greater than 0 (was {part.Quantity})."
+                );
+
+            var dxfPath = Path.GetFullPath(Path.Combine(baseDir, part.Dxf));
+
+            if (!File.Exists(dxfPath))
+                throw new FileNotFoundException($"DXF file not found: {dxfPath}", dxfPath);
+
+            Drawing drawing;
+
+            try
+            {
+                drawing = CadImporter.ImportDrawing(
+                    dxfPath,
+                    new CadImportOptions { Quantity = part.Quantity }
+                );
+            }
+            catch (Exception ex)
+            {
+                throw new InvalidOperationException($"Failed to import DXF: {dxfPath}", ex);
+            }
+
+            if (drawing.Program == null || drawing.Program.Codes.Count == 0)
+                throw new InvalidOperationException($"Failed to import DXF: {dxfPath}");
+
+            // A zero legacy step means automatic rotation to DrawingJobMapper, so lock it explicitly.
+            if (!part.AllowRotation)
+            {
+                drawing.Constraints ??= new NestConstraints();
+                drawing.Constraints.StepAngle = OpenNest.Math.Angle.TwoPI;
+                drawing.Constraints.StartAngle = 0;
+                drawing.Constraints.EndAngle = 0;
+            }
+
+            var constraints = drawing.Constraints;
+
+            return new DrawingRequest
+            {
+                Drawing = drawing,
+                Quantity = part.Quantity,
+                Priority = drawing.Priority,
+                StepAngle = constraints?.StepAngle ?? 0,
+                RotationStart = constraints?.StartAngle ?? 0,
+                RotationEnd = constraints?.EndAngle ?? 0,
+            };
+        }
+
+        private class Manifest
+        {
+            public List<string> SheetSizes { get; set; }
+            public double Spacing { get; set; }
+            public double EdgeSpacing { get; set; }
+            public int Quadrant { get; set; } = 1;
+            public List<ManifestPart> Parts { get; set; }
+        }
+
+        private class ManifestPart
+        {
+            public string Dxf { get; set; }
+            public int Quantity { get; set; }
+            public bool AllowRotation { get; set; } = true;
+        }
+    }
+}

OpenNest.Benchmark/JobLoader.cs

@@ -28,6 +28,13 @@ namespace OpenNest.Benchmark
 
             foreach (var file in files)
             {
+                if (file.EndsWith(".json", StringComparison.OrdinalIgnoreCase))
+                {
+                    // Hand-written manifests fail loudly rather than being skipped like unreadable .nest files.
+                    jobs.Add(DxfManifestLoader.Load(file, sheetSizeOverrides, partSpacingOverride));
+                    continue;
+                }
+
                 Nest nest;
 
                 try
@@ -79,7 +86,14 @@ namespace OpenNest.Benchmark
             if (Directory.Exists(inputPath))
             {
                 return Directory
-                    .GetFiles(inputPath, "*.nest", SearchOption.AllDirectories)
+                    .EnumerateFiles(inputPath, "*", SearchOption.AllDirectories)
+                    .Where(f =>
+                        f.EndsWith(".nest", StringComparison.OrdinalIgnoreCase)
+                        || f.EndsWith(
+                            DxfManifestLoader.FolderSuffix,
+                            StringComparison.OrdinalIgnoreCase
+                        )
+                    )
                     .OrderBy(f => f, StringComparer.OrdinalIgnoreCase)
                     .ToList();
             }

OpenNest.Benchmark/Program.cs

@@ -38,7 +38,7 @@ static class BenchmarkConsole
         if (jobs.Count == 0)
         {
             Console.Error.WriteLine(
-                "No benchmark jobs found (no .nest files with any drawing quantity > 0)."
+                "No benchmark jobs found (no .nest files with any drawing quantity > 0, or *.manifest.json files)."
             );
             return 1;
         }
@@ -83,12 +83,23 @@ static class BenchmarkConsole
 
         Console.WriteLine($"Engines: {string.Join(", ", engines.Select(e => e.Name))}");
 
+        var solves = jobs.Count * engines.Count;
+
+        if (options.Parallel > 1 && solves > 1)
+        {
+            Console.WriteLine(
+                $"Running up to {options.Parallel} solves at a time; Time(ms) is measured under that "
+                    + "concurrent load. Use --parallel 1 for strictly isolated timings."
+            );
+        }
+
         var results = BenchmarkRunner.Run(
             jobs,
             engines,
             options.SalvageRate,
             options.MinimumSalvageDimension,
-            options.OutputDirectory
+            options.OutputDirectory,
+            options.Parallel
         );
 
         Report.PrintDetailed(results);
@@ -149,6 +160,15 @@ static class BenchmarkConsole
                     o.OutputDirectory = args[++i];
                     break;
 
+                case "--parallel" when i + 1 < args.Length:
+                    if (int.TryParse(args[++i], out var parallel) && parallel >= 1)
+                        o.Parallel = parallel;
+                    else
+                        Console.Error.WriteLine(
+                            $"Warning: --parallel needs a whole number >= 1, using {o.Parallel}"
+                        );
+                    break;
+
                 case "--help":
                     PrintUsage();
                     return null;
@@ -215,7 +235,25 @@ static class BenchmarkConsole
         );
         Console.Error.WriteLine();
         Console.Error.WriteLine("Usage:");
-        Console.Error.WriteLine("  OpenNest.Benchmark <file.nest | folder> [options]");
+        Console.Error.WriteLine(
+            "  OpenNest.Benchmark <file.nest | manifest.json | folder> [options]"
+        );
+        Console.Error.WriteLine();
+        Console.Error.WriteLine(
+            "A manifest.json builds a job straight from DXF files (paths relative to the manifest):"
+        );
+        Console.Error.WriteLine(
+            "  { \"sheetSizes\": [\"48x96\"], \"spacing\": 0.25, \"edgeSpacing\": 0.25, \"quadrant\": 1,"
+        );
+        Console.Error.WriteLine(
+            "    \"parts\": [ { \"dxf\": \"a.dxf\", \"quantity\": 12 }, { \"dxf\": \"b.dxf\", \"quantity\": 4, \"allowRotation\": false } ] }"
+        );
+        Console.Error.WriteLine(
+            "Sheet sizes must use the same units as the DXFs. A folder is scanned for *.nest and"
+        );
+        Console.Error.WriteLine(
+            "*.manifest.json files. --sheet-sizes and --spacing override the manifest."
+        );
         Console.Error.WriteLine();
         Console.Error.WriteLine("Options:");
         Console.Error.WriteLine(
@@ -242,6 +280,12 @@ static class BenchmarkConsole
         Console.Error.WriteLine(
             "  --output <directory>           Save valid layouts as .nest plus detailed JSON reports"
         );
+        Console.Error.WriteLine(
+            "  --parallel <n>                  Solves to run at once (default 3; 1 = strictly sequential,"
+        );
+        Console.Error.WriteLine(
+            "                                 which gives the cleanest per-engine timings)"
+        );
         Console.Error.WriteLine("  --help                          Show this message");
     }
 
@@ -255,5 +299,6 @@ static class BenchmarkConsole
         public string OutputDirectory;
         public double SalvageRate;
         public double MinimumSalvageDimension;
+        public int Parallel = 3;
     }
 }

OpenNest.Tests/Benchmark/BenchmarkRunnerTests.cs

@@ -0,0 +1,168 @@
+using OpenNest.Benchmark;
+
+namespace OpenNest.Tests.Benchmark;
+
+public sealed class BenchmarkRunnerTests : IDisposable
+{
+    private static readonly string SourceDxf = Path.Combine(
+        "Bending",
+        "TestData",
+        "4526 A14 PT11.dxf"
+    );
+
+    private readonly string _dir = Path.Combine(
+        Path.GetTempPath(),
+        "opennest-runner-" + Guid.NewGuid().ToString("N")
+    );
+
+    public BenchmarkRunnerTests()
+    {
+        Directory.CreateDirectory(_dir);
+        File.Copy(SourceDxf, Path.Combine(_dir, "part.dxf"));
+    }
+
+    public void Dispose()
+    {
+        try
+        {
+            Directory.Delete(_dir, recursive: true);
+        }
+        catch (IOException) { }
+    }
+
+    private List<BenchmarkJob> LoadJob()
+    {
+        var manifest = Path.Combine(_dir, "job.json");
+        File.WriteAllText(
+            manifest,
+            """{ "sheetSizes": ["48x96"], "parts": [ { "dxf": "part.dxf", "quantity": 2 } ] }"""
+        );
+        return JobLoader.Load(manifest);
+    }
+
+    private static List<NestingEngineInfo> FakeEngines(
+        ConcurrencyProbe probe,
+        params int[] delaysMs
+    ) =>
+        delaysMs
+            .Select(
+                (delay, i) =>
+                    new NestingEngineInfo(
+                        $"Engine{i}",
+                        "test double",
+                        () => new SleepingEngine(probe, delay)
+                    )
+            )
+            .ToList();
+
+    [Fact]
+    public void Run_NeverExceedsMaxParallelism_ButReachesIt()
+    {
+        var probe = new ConcurrencyProbe();
+
+        var results = BenchmarkRunner.Run(
+            LoadJob(),
+            FakeEngines(probe, 200, 200, 200, 200, 200, 200),
+            maxParallelism: 2
+        );
+
+        Assert.Equal(6, results.Count);
+        Assert.Equal(2, probe.Max);
+    }
+
+    [Fact]
+    public void Run_WithMaxParallelismOne_RunsOneSolveAtATime()
+    {
+        var probe = new ConcurrencyProbe();
+
+        BenchmarkRunner.Run(LoadJob(), FakeEngines(probe, 50, 50, 50, 50), maxParallelism: 1);
+
+        Assert.Equal(1, probe.Max);
+    }
+
+    [Fact]
+    public void Run_KeepsResultsInJobThenEngineOrder_RegardlessOfCompletionOrder()
+    {
+        var probe = new ConcurrencyProbe();
+
+        // The first engine finishes last, so completion order differs from input order.
+        var results = BenchmarkRunner.Run(
+            LoadJob(),
+            FakeEngines(probe, 300, 10, 10, 10),
+            maxParallelism: 3
+        );
+
+        Assert.Equal(
+            new[] { "Engine0", "Engine1", "Engine2", "Engine3" },
+            results.Select(r => r.EngineName)
+        );
+    }
+
+    [Fact]
+    public void Run_WithOutputDirectory_WritesManifestJobsWithoutNeedingASourceNest()
+    {
+        var probe = new ConcurrencyProbe();
+        var output = Path.Combine(_dir, "out");
+
+        var results = BenchmarkRunner.Run(
+            LoadJob(),
+            FakeEngines(probe, 0),
+            outputDirectory: output
+        );
+
+        var result = Assert.Single(results);
+        Assert.Null(result.Error);
+        Assert.True(result.Valid);
+        Assert.True(File.Exists(Path.Combine(output, "job-Engine0.nest")));
+        Assert.True(File.Exists(Path.Combine(output, "job-Engine0.json")));
+    }
+
+    private sealed class ConcurrencyProbe
+    {
+        private int _current;
+        private int _max;
+
+        public int Max => Volatile.Read(ref _max);
+
+        public void Enter()
+        {
+            var now = Interlocked.Increment(ref _current);
+            int seen;
+            while ((seen = Volatile.Read(ref _max)) < now)
+            {
+                if (Interlocked.CompareExchange(ref _max, now, seen) == seen)
+                    break;
+            }
+        }
+
+        public void Exit() => Interlocked.Decrement(ref _current);
+    }
+
+    /// <summary>Places nothing after sleeping, recording how many solves overlap.</summary>
+    private sealed class SleepingEngine(ConcurrencyProbe probe, int delayMs) : INestingEngine
+    {
+        public NestJobResult Solve(
+            NestJob job,
+            IProgress<NestJobProgress>? progress = null,
+            CancellationToken token = default
+        )
+        {
+            probe.Enter();
+            try
+            {
+                Thread.Sleep(delayMs);
+                return new NestJobResult(
+                    NestJobStatus.Incomplete,
+                    NestJobStopReason.NoPlacementFound,
+                    Array.Empty<NestJobPlateResult>(),
+                    job.Parts.Select(p => new PartFulfillment(p.Id, p.Quantity, 0, p.Quantity)),
+                    job.Plates.Select(s => new StockUsage(s.Id, 0, null))
+                );
+            }
+            finally
+            {
+                probe.Exit();
+            }
+        }
+    }
+}

OpenNest.Tests/Benchmark/JobLoaderManifestTests.cs

@@ -0,0 +1,215 @@
+using OpenNest.Benchmark;
+using OpenNest.Geometry;
+
+namespace OpenNest.Tests.Benchmark;
+
+public sealed class JobLoaderManifestTests : IDisposable
+{
+    private static readonly string SourceDxf = Path.Combine(
+        "Bending",
+        "TestData",
+        "4526 A14 PT11.dxf"
+    );
+
+    private readonly string _dir = Path.Combine(
+        Path.GetTempPath(),
+        "opennest-manifest-" + Guid.NewGuid().ToString("N")
+    );
+
+    public JobLoaderManifestTests()
+    {
+        Directory.CreateDirectory(Path.Combine(_dir, "parts"));
+        File.Copy(SourceDxf, Path.Combine(_dir, "parts", "bracket.dxf"));
+        File.Copy(SourceDxf, Path.Combine(_dir, "parts", "plate.dxf"));
+    }
+
+    public void Dispose()
+    {
+        try
+        {
+            Directory.Delete(_dir, recursive: true);
+        }
+        catch (IOException) { }
+    }
+
+    private string WriteManifest(string name, string json)
+    {
+        var path = Path.Combine(_dir, name);
+        File.WriteAllText(path, json);
+        return path;
+    }
+
+    private const string ValidManifest = """
+        {
+          "sheetSizes": ["48x96", "60x120"],
+          "spacing": 0.25,
+          "edgeSpacing": 0.5,
+          "quadrant": 2,
+          "parts": [
+            { "dxf": "parts/bracket.dxf", "quantity": 12 },
+            { "dxf": "parts/plate.dxf", "quantity": 4 }
+          ]
+        }
+        """;
+
+    [Fact]
+    public void Manifest_ImportsDxfsWithQuantitiesAndJobSettings()
+    {
+        var path = WriteManifest("bench.json", ValidManifest);
+
+        var job = Assert.Single(JobLoader.Load(path));
+
+        Assert.Equal("bench", job.Name);
+        Assert.Equal(new[] { 12, 4 }, job.Requests.Select(r => r.Quantity));
+        Assert.Equal(16, job.TotalRequestedQuantity);
+        Assert.All(job.Requests, r => Assert.NotEmpty(r.Drawing.Program.Codes));
+        Assert.Equal(new[] { new Size(48, 96), new Size(60, 120) }, job.CandidateSizes.ToArray());
+        Assert.Equal(0.25, job.PartSpacing);
+        Assert.Equal(0.5, job.EdgeSpacing.Left);
+        Assert.Equal(0.5, job.EdgeSpacing.Top);
+        Assert.Equal(2, job.Quadrant);
+    }
+
+    [Fact]
+    public void Manifest_DxfPathsResolveRelativeToTheManifestNotTheWorkingDirectory()
+    {
+        var path = WriteManifest("bench.json", ValidManifest);
+        var elsewhere = Directory.GetCurrentDirectory();
+        Assert.NotEqual(_dir, elsewhere);
+
+        var job = Assert.Single(JobLoader.Load(path));
+
+        Assert.Equal(2, job.Requests.Count);
+    }
+
+    [Fact]
+    public void Overrides_ReplaceManifestSheetSizesAndSpacing()
+    {
+        var path = WriteManifest("bench.json", ValidManifest);
+
+        var job = Assert.Single(JobLoader.Load(path, new[] { new Size(10, 20) }, 0.75));
+
+        Assert.Equal(new[] { new Size(10, 20) }, job.CandidateSizes.ToArray());
+        Assert.Equal(0.75, job.PartSpacing);
+    }
+
+    [Fact]
+    public void SheetSizesFromOverrideAreEnoughWhenManifestOmitsThem()
+    {
+        var path = WriteManifest(
+            "bench.json",
+            """{ "parts": [ { "dxf": "parts/bracket.dxf", "quantity": 1 } ] }"""
+        );
+
+        var job = Assert.Single(JobLoader.Load(path, new[] { new Size(48, 96) }));
+
+        Assert.Equal(new[] { new Size(48, 96) }, job.CandidateSizes.ToArray());
+        Assert.Equal(0, job.PartSpacing);
+        Assert.Equal(1, job.Quadrant);
+    }
+
+    [Fact]
+    public void NoSheetSizesAnywhere_ThrowsAndMentionsSheetSizes()
+    {
+        var path = WriteManifest(
+            "bench.json",
+            """{ "parts": [ { "dxf": "parts/bracket.dxf", "quantity": 1 } ] }"""
+        );
+
+        var ex = Assert.Throws<InvalidOperationException>(() => JobLoader.Load(path));
+
+        Assert.Contains("sheet size", ex.Message, StringComparison.OrdinalIgnoreCase);
+    }
+
+    [Fact]
+    public void UnparseableSheetSize_ThrowsNamingTheValue()
+    {
+        var path = WriteManifest(
+            "bench.json",
+            """{ "sheetSizes": ["huge"], "parts": [ { "dxf": "parts/bracket.dxf", "quantity": 1 } ] }"""
+        );
+
+        var ex = Assert.Throws<InvalidOperationException>(() => JobLoader.Load(path));
+
+        Assert.Contains("huge", ex.Message);
+    }
+
+    [Fact]
+    public void MissingDxf_ThrowsNamingTheEntry()
+    {
+        var path = WriteManifest(
+            "bench.json",
+            """{ "sheetSizes": ["48x96"], "parts": [ { "dxf": "parts/nope.dxf", "quantity": 1 } ] }"""
+        );
+
+        var ex = Assert.Throws<FileNotFoundException>(() => JobLoader.Load(path));
+
+        Assert.Contains("nope.dxf", ex.Message);
+    }
+
+    [Theory]
+    [InlineData(0)]
+    [InlineData(-3)]
+    public void NonPositiveQuantity_ThrowsNamingTheEntry(int quantity)
+    {
+        var path = WriteManifest(
+            "bench.json",
+            $$"""{ "sheetSizes": ["48x96"], "parts": [ { "dxf": "parts/bracket.dxf", "quantity": {{quantity}} } ] }"""
+        );
+
+        var ex = Assert.Throws<InvalidOperationException>(() => JobLoader.Load(path));
+
+        Assert.Contains("bracket.dxf", ex.Message);
+        Assert.Contains("quantity", ex.Message, StringComparison.OrdinalIgnoreCase);
+    }
+
+    [Fact]
+    public void EmptyPartsList_Throws()
+    {
+        var path = WriteManifest("bench.json", """{ "sheetSizes": ["48x96"], "parts": [] }""");
+
+        var ex = Assert.Throws<InvalidOperationException>(() => JobLoader.Load(path));
+
+        Assert.Contains("parts", ex.Message, StringComparison.OrdinalIgnoreCase);
+    }
+
+    [Fact]
+    public void AllowRotationFalse_LocksThatPartsRotationInTheNestJob()
+    {
+        var path = WriteManifest(
+            "bench.json",
+            """
+            {
+              "sheetSizes": ["48x96"],
+              "parts": [
+                { "dxf": "parts/bracket.dxf", "quantity": 2 },
+                { "dxf": "parts/plate.dxf", "quantity": 2, "allowRotation": false }
+              ]
+            }
+            """
+        );
+
+        var job = Assert.Single(JobLoader.Load(path));
+        var nestJob = job.BuildNestJob(maxPlates: 4);
+
+        Assert.Equal(RotationPolicyKind.Automatic, nestJob.Parts[0].Rotation.Kind);
+
+        // The legacy lock (step 2π, start = end = 0) reaches the engine as a single-angle sweep at 0.
+        var locked = nestJob.Parts[1].Rotation;
+        Assert.Equal(RotationPolicyKind.BoundedSweep, locked.Kind);
+        Assert.Equal(0, locked.Start);
+        Assert.Equal(0, locked.End);
+    }
+
+    [Fact]
+    public void Folder_LoadsManifestSuffixedFilesAndIgnoresOtherJson()
+    {
+        WriteManifest("a.manifest.json", ValidManifest);
+        WriteManifest("b.manifest.json", ValidManifest);
+        WriteManifest("results.json", """{ "not": "a manifest" }""");
+
+        var jobs = JobLoader.Load(_dir);
+
+        Assert.Equal(new[] { "a.manifest", "b.manifest" }, jobs.Select(j => j.Name));
+    }
+}

OpenNest.Tests/OpenNest.Tests.csproj

@@ -23,6 +23,7 @@
 
   <ItemGroup>
     <ProjectReference Include="..\OpenNest.Api\OpenNest.Api.csproj" />
+    <ProjectReference Include="..\OpenNest.Benchmark\OpenNest.Benchmark.csproj" />
     <ProjectReference Include="..\OpenNest.Data\OpenNest.Data.csproj" />
     <ProjectReference Include="..\OpenNest.Core\OpenNest.Core.csproj" />
     <ProjectReference Include="..\OpenNest.Engine\OpenNest.Engine.csproj" />

README.md

@@ -205,6 +205,29 @@ dotnet run --project OpenNest.Benchmark/OpenNest.Benchmark.csproj -- job.nest \
   --sheet-sizes 48x96,60x96,60x120,72x120,72x144 --engines Default,Astra,Claude --csv results.csv
 ```
 
+To benchmark straight from DXF files without building a `.nest` first, pass a JSON manifest listing each DXF and its quantity:
+
+```json
+{
+  "sheetSizes": ["48x96", "60x120"],
+  "spacing": 0.25,
+  "edgeSpacing": 0.25,
+  "quadrant": 1,
+  "parts": [
+    { "dxf": "parts/bracket.dxf", "quantity": 12 },
+    { "dxf": "parts/plate.dxf", "quantity": 4, "allowRotation": false }
+  ]
+}
+```
+
+```bash
+dotnet run --project OpenNest.Benchmark/OpenNest.Benchmark.csproj -- job.json --csv results.csv
+```
+
+DXF paths resolve relative to the manifest. Sheet sizes are required (from the manifest or `--sheet-sizes`, which overrides it) and must use the same units as the DXFs; `--spacing` likewise overrides `spacing`. `spacing`, `edgeSpacing` and `quadrant` default to 0, 0 and 1, and rotation is unconstrained unless a part sets `allowRotation: false`. A folder input is scanned for `*.nest` and `*.manifest.json` files. Unlike an unreadable `.nest`, an invalid manifest (missing DXF, bad quantity, no sheet sizes) stops the run with an error naming the problem.
+
+Each engine-on-job solve is independent, so `--parallel <n>` (default 3) runs up to `n` at once; `--parallel 1` is strictly sequential. Results and their order in the report are the same either way. The catch is timing: some solves use one core, others spread across many, and when concurrent solves compete for cores `Time(ms)` goes up (a short multi-threaded job showed 2–4× inflation at `--parallel 3`). Scores (utilization, plates, validity) are unaffected, so use `--parallel 1` when comparing speed. The 5-minute per-solve timeout is wall-clock, so contention can also push a slow engine over it.
+
 An engine's layout is rejected (scoring zero for that job) if any part falls outside the work area, any two parts are closer than the required spacing, or a drawing gets more parts placed than requested. A run that doesn't finish within its time budget also scores zero, as a timeout.
 
 Custom competitor engines can be added by dropping a DLL implementing `INestingEngine` with a public parameterless constructor into the `Engines/` directory next to the benchmark executable; each one is registered under its own CLR type name. This is a separate plugin contract from the desktop app's `NestEngineRegistry`/`NestEngineBase` (which requires a `(Plate)` constructor) — a `NestEngineBase` plugin dropped into the benchmark's `Engines/` folder is silently skipped, since the benchmark only ever solves whole jobs.