mirror of
https://github.com/ajisaacs/OpenNest.git
synced 2026-10-12 07:44:21 -04:00
test(server): verify complete revision backup and restore
This commit is contained in:
aj committed
1 parent
a7c8e8541e
commit
0216a9e9e2
6 files changed
+586
-31
No files matched your search
+105
-25
@@ -23,8 +23,8 @@ The desktop app can save nests two ways:
|
||||
first save and updates that same record afterward. Save As creates a new record
|
||||
(a copy). A failed save keeps the previous record association; switching server
|
||||
URLs creates a record on the new server rather than updating an id from the
|
||||
previous one. Delete on the Open dialog's right-click menu permanently removes
|
||||
the highlighted server record after confirmation. A separate **File > Export
|
||||
previous one. Version-aware deletion hides the record and retains every archive
|
||||
and revision for explicit recovery; it never purges data. A separate **File > Export
|
||||
.nest...** command is always available (in both modes) for producing a local
|
||||
file to share or back up. Export does not change the document's file save path
|
||||
or database record association.
|
||||
@@ -62,6 +62,9 @@ with enums serialized as strings (`JsonSerializerDefaults.Web` +
|
||||
| `madeBy` | string | |
|
||||
| `fileSize` | integer | Server-computed; ignored on upload. |
|
||||
| `savedAt` | datetime | Server-computed; ignored on upload. |
|
||||
| `revision` | integer | Current immutable revision; zero on a legacy server without history. |
|
||||
| `deleted` | boolean | Hidden from ordinary reads when true; retained in recovery/backup. |
|
||||
| `restoredFromRevision` | nullable integer | Historical source promoted by Restore; not the preceding revision. |
|
||||
|
||||
## Endpoints
|
||||
|
||||
@@ -75,7 +78,20 @@ with enums serialized as strings (`JsonSerializerDefaults.Web` +
|
||||
| POST | `/api/nests` | multipart: `metadata` (JSON `NestRecord`) + `file` (`.nest` bytes) | `NestRecord` with server-assigned `id` when the client sends an empty guid |
|
||||
| PUT | `/api/nests/{id}/file` | multipart: `metadata` + `file` | Updated `NestRecord`, or 404 if `id` is unknown |
|
||||
| PUT | `/api/nests/{id}/metadata` | JSON `NestRecord` | Updated `NestRecord` (archive untouched), or 404 |
|
||||
| DELETE | `/api/nests/{id}` | — | 204, or 404 |
|
||||
| DELETE | `/api/nests/{id}` | — | Updated hidden `NestRecord`, or 404 |
|
||||
| GET | `/api/nests/{id}/revisions?offset=0&limit=100` | — | Metadata page, newest first; limit 1–200 |
|
||||
| GET | `/api/nests/{id}/revisions/{revision}/file` | — | Exact immutable archive bytes |
|
||||
| POST | `/api/nests/{id}/revisions/{revision}/restore` | — | New current revision of the same ID, with source metadata/bytes |
|
||||
| GET | `/api/nests/deleted/query` | — | Bounded deleted-record page; same query parameters as ordinary browsing |
|
||||
| GET | `/api/nests/deleted/{id}` / `/{id}/file` / `/{id}/revisions` / `/{id}/revisions/{revision}/file` | — | Recovery equivalents of current/history reads |
|
||||
| POST | `/api/nests/deleted/{id}/recover` | — | New nondeleted revision of the same ID |
|
||||
|
||||
All existing-record mutations require `If-Match: "<current revision>"`.
|
||||
Missing precondition returns 428 (upgrade/reload), malformed precondition 400,
|
||||
stale precondition 409 with current metadata; none mutate data. Ordinary reads
|
||||
of hidden records return 404. Changed file/metadata saves and visibility changes
|
||||
append snapshots atomically; an identical save at the current revision is a no-op.
|
||||
Restore appends a new snapshot, never rewrites its source or intervening history.
|
||||
|
||||
There is no authentication; this is a LAN-only service. Do not expose it
|
||||
outside the shop network without adding one.
|
||||
@@ -122,9 +138,11 @@ HTTP upload/body reading happens outside that lock. `/healthz` executes `SELECT
|
||||
on the live connection; it is a connection check, not a backup, integrity scan,
|
||||
or guarantee of future disk capacity.
|
||||
|
||||
Concurrent edits of the same record remain **last-writer-wins**: there is no
|
||||
optimistic version check or document lock. Coordinate editing with other operators
|
||||
to avoid overwriting their changes.
|
||||
Revision preconditions refuse stale edits; the database monitor is not a document
|
||||
edit lock. After a conflict, explicitly reload the newer revision or Save As a
|
||||
separate nest. After an ambiguous network failure, fetch current metadata before
|
||||
retrying; never blindly replay a write. Legacy desktops may read but must upgrade
|
||||
before updating/deleting. A version-aware client refuses writes to a legacy server.
|
||||
|
||||
## Server integration tests
|
||||
|
||||
@@ -252,21 +270,29 @@ or enable Database mode. Canceling Options afterward does not undo that address.
|
||||
### Backup, restore, and upgrade
|
||||
|
||||
SQLite runs in WAL mode, so copying `nests.db` from a running service is not a
|
||||
backup. Back up during a quiet period with no saves. Each step below is a Bash
|
||||
backup. Back up during a maintenance period with ALL client writes disabled,
|
||||
including metadata edits, delete, recover and restore, until capture finishes.
|
||||
The HTTP manifest is not a transaction across requests; writes during capture
|
||||
can invalidate it. Keep the entire stopped data directory, including any WAL/SHM
|
||||
files, original image digest and deployment configuration. Copy
|
||||
`scripts/Server.Tests/nest_manifest.py` from this source version to the deployment
|
||||
directory as `nest_manifest.py` (Python 3 standard library only). It enumerates
|
||||
bounded active AND deleted pages plus ALL revision pages, then streams every
|
||||
current/historical archive through SHA-256. Canonical JSON includes complete
|
||||
metadata, IDs, current pointers, visibility, timestamps, byte sizes and restore
|
||||
source relationships. Missing pages, legacy endpoints, failed downloads,
|
||||
inconsistent totals and invalid revision chains fail closed, not as empty history.
|
||||
Protect manifests like the database: they contain names and comments.
|
||||
Each step below is a Bash
|
||||
function that checks every command and stops at the first failure, however it is
|
||||
called; define them in the shell where `compose` is set:
|
||||
|
||||
```bash
|
||||
url=http://<bind-address>:<port>
|
||||
|
||||
# "<id> <archive SHA-256>" for every record, sorted by id. Fails on any failed request.
|
||||
# Canonical complete manifest; stdout is emitted only after all reads succeed.
|
||||
nest_manifest() (
|
||||
set -o pipefail
|
||||
list=$(curl -fsS "$1/api/nests") || exit 1
|
||||
for id in $(printf '%s' "$list" | grep -o '"id":"[0-9a-f-]*"' | cut -d'"' -f4 | sort); do
|
||||
hash=$(curl -fsS "$1/api/nests/$id/file" | sha256sum) || exit 1
|
||||
printf '%s %s\n' "$id" "${hash%% *}"
|
||||
done
|
||||
python3 ./nest_manifest.py "$1" || exit 1
|
||||
)
|
||||
|
||||
# Records the expected manifest and metadata, then archives the stopped data directory.
|
||||
@@ -274,7 +300,9 @@ nest_manifest() (
|
||||
backup_nests() ( # usage: backup_nests opennest-data-YYYY-MM-DD
|
||||
set -o pipefail
|
||||
fail() { echo "STOP: $*" >&2; exit 1; }
|
||||
[[ ! -e "$1.tar" ]] || fail "$1.tar already exists"
|
||||
for suffix in tar manifest metadata.json; do
|
||||
[[ ! -e "$1.$suffix" ]] || fail "$1.$suffix already exists"
|
||||
done
|
||||
nest_manifest "$url" > "$1.manifest" || fail "could not record the manifest"
|
||||
curl -fsS "$url/api/nests" > "$1.metadata.json" || fail "could not record the metadata"
|
||||
$compose stop || fail "could not stop the service"
|
||||
@@ -284,7 +312,7 @@ backup_nests() ( # usage: backup_nests opennest-data-YYYY-MM-DD
|
||||
curl -fs --retry 30 --retry-all-errors --retry-delay 1 "$url/healthz" > /dev/null \
|
||||
|| fail "the restarted service is not healthy"
|
||||
[[ $status == 0 ]] || fail "the archive failed; $1.tar is incomplete"
|
||||
echo "Backed up $(wc -l < "$1.manifest") records to $1.tar"
|
||||
echo "Backed up all records and revisions to $1.tar"
|
||||
)
|
||||
|
||||
backup_nests opennest-data-YYYY-MM-DD
|
||||
@@ -293,20 +321,25 @@ backup_nests opennest-data-YYYY-MM-DD
|
||||
Restore only into a volume created for that restore, never into an existing one
|
||||
(least of all the active volume). `restore_check` refuses an existing volume,
|
||||
extracts the backup into a new one, starts a temporary loopback-only container on
|
||||
it, and requires the exact metadata list and every archive hash to match. It
|
||||
it, and requires complete active/deleted revision metadata and archive hashes to match. It
|
||||
removes only the container it started; the new volume is kept either way:
|
||||
|
||||
```bash
|
||||
restore_check() ( # usage: restore_check opennest-data-YYYY-MM-DD <image> [new-volume]
|
||||
set -o pipefail
|
||||
fail() { echo "STOP: $*" >&2; exit 1; }
|
||||
[[ -r "$1.tar" && -r "$1.manifest" && -r "$1.metadata.json" ]] || fail "backup is incomplete"
|
||||
restore=${3:-opennest-data-restore-$(date +%Y%m%d-%H%M%S)}
|
||||
! docker volume inspect "$restore" > /dev/null 2>&1 || fail "volume $restore already exists"
|
||||
docker volume create "$restore" > /dev/null || fail "could not create volume $restore"
|
||||
docker volume create --label "com.opennest.restore.owner=$restore" "$restore" > /dev/null \
|
||||
|| fail "could not create volume $restore"
|
||||
[[ $(docker volume inspect --format '{{index .Labels "com.opennest.restore.owner"}}' "$restore") == "$restore" ]] \
|
||||
|| fail "restore volume ownership differs"
|
||||
echo "Restoring into new volume $restore"
|
||||
docker run --rm -i --network none --entrypoint tar -v "$restore:/app/data" "$2" \
|
||||
-C /app/data -xf - < "$1.tar" || fail "extraction into $restore failed"
|
||||
check=$(docker create -p 127.0.0.1::8090 -v "$restore:/app/data" "$2") \
|
||||
check=$(docker create --cap-drop ALL --security-opt no-new-privileges:true \
|
||||
-p 127.0.0.1::8090 -v "$restore:/app/data" "$2") \
|
||||
|| fail "could not create the check container"
|
||||
trap 'docker rm -f "$check" > /dev/null' EXIT
|
||||
docker start "$check" > /dev/null || fail "the check container did not start"
|
||||
@@ -315,7 +348,7 @@ restore_check() ( # usage: restore_check opennest-data-YYYY-MM-DD <image> [new-
|
||||
|| fail "the restored service is not healthy"
|
||||
curl -fsS "http://127.0.0.1:$port/api/nests" | cmp - "$1.metadata.json" || fail "metadata differs"
|
||||
nest_manifest "http://127.0.0.1:$port" | diff - "$1.manifest" || fail "archives differ"
|
||||
echo "Verified $restore. Set OPENNEST_DATA_VOLUME=$restore in server.env, then run: \$compose up -d"
|
||||
echo "Verified $restore. Keep it isolated until the operator explicitly approves switching volumes."
|
||||
)
|
||||
|
||||
restore_check opennest-data-YYYY-MM-DD <image>
|
||||
@@ -328,15 +361,37 @@ To upgrade, take a stopped-service backup and keep a copy of the current
|
||||
`server.env`. Prefer a digest in `OPENNEST_SERVER_IMAGE` so that file records
|
||||
exactly what ran; for a tag, record
|
||||
`docker image inspect --format '{{join .RepoDigests " "}}' <image>` first. Change
|
||||
only `OPENNEST_SERVER_IMAGE` and run `$compose up -d`. To roll back, restore the
|
||||
previous `server.env` and run `$compose up -d`; if the new version wrote data the
|
||||
old one cannot read, also restore the backup as above. Never run
|
||||
only `OPENNEST_SERVER_IMAGE` after migration/restore rehearsal on a disposable copy,
|
||||
then run `$compose up -d` only with separate deployment authorization.
|
||||
**Never run an old image against a migrated database**, even if its current-table
|
||||
GETs still work: old writers can overwrite snapshots or physically delete rows
|
||||
without updating history. Rollback requires a NEW volume restored from the verified
|
||||
PRE-migration backup and the exact old image/configuration. Test its old current
|
||||
metadata/byte manifest on that isolated copy before switching; the revision-aware
|
||||
manifest deliberately refuses old endpoints. Archive the migrated volume unchanged
|
||||
for recovery; reverting loses post-backup changes from the old server's view, so
|
||||
reconcile them deliberately, never replay them automatically. A pre-version server
|
||||
backup needs its own current-record metadata/byte oracle before migration; verify
|
||||
those records become byte-identical revision 1 on the disposable upgraded copy.
|
||||
Never run
|
||||
`docker compose down -v`: it deletes the data volume.
|
||||
|
||||
#### Capacity and retention
|
||||
|
||||
Every revision currently stores a full archive BLOB, including metadata-only
|
||||
and visibility revisions. Budget approximately the sum of ALL revision archive
|
||||
sizes plus current-table archives, SQLite indexes/pages, WAL and backups; do not
|
||||
estimate capacity from active current records alone. Measure volume/database and
|
||||
WAL growth during the disposable rehearsal, reserve space for migration and a
|
||||
separate verified restore copy, and monitor free space before rollout. There is
|
||||
no automatic retention or purge; no revision deletion is authorized. Deduplication
|
||||
requires a separately reviewed migration after measuring actual growth. An old
|
||||
backup is not a substitute for retaining the migrated volume during rollback.
|
||||
|
||||
## Isolated container smoke
|
||||
|
||||
Prerequisites: a local Linux Docker daemon on the default Unix-socket context,
|
||||
Bash, curl, GNU `timeout`, `mktemp`, `tar`, and a .NET SDK able to build `net8.0` projects
|
||||
Bash, Python 3, curl, GNU `timeout`, `mktemp`, `tar`, `cmp`, and a .NET SDK able to build `net8.0` projects
|
||||
(.NET 8 or newer). Restore needs NuGet access or cached packages. Run from the
|
||||
repository root after building the image:
|
||||
|
||||
@@ -346,6 +401,22 @@ scripts/Test-ServerContainer.sh --image opennest-server:local
|
||||
scripts/Test-ServerContainer.sh --image opennest-server:local --results /path/to/results
|
||||
```
|
||||
|
||||
Manifest refusal tests run without Docker. The optional image variables also
|
||||
exercise the Bash backup/restore blocks extracted verbatim from this page in a
|
||||
unique disposable Compose project, including existing-file/volume refusal and
|
||||
tampered-history rejection. The old image test restores ONLY a synthetic
|
||||
pre-migration backup to another fresh volume and compares every old record's
|
||||
metadata/bytes; it never mounts a migrated database into the old image.
|
||||
|
||||
```sh
|
||||
python3 -m unittest discover -s scripts/Server.Tests -p test_revision_backup.py -v
|
||||
OPENNEST_TEST_IMAGE=opennest-server:local \
|
||||
OPENNEST_TEST_OLD_IMAGE=<local-pre-version-image> \
|
||||
python3 -m unittest discover -s scripts/Server.Tests -p test_revision_backup.py -v
|
||||
```
|
||||
|
||||
Local smoke evidence does not imply registry publication or production deployment.
|
||||
|
||||
The wrapper accepts an image, not a URL or an existing volume/container. It uses
|
||||
only the local default Docker context, creates uniquely labeled disposable
|
||||
resources, and publishes to `127.0.0.1` on a Docker-allocated port. Every container
|
||||
@@ -363,16 +434,21 @@ owns `nests.db`, and can write `/app/data` but not `/app` or the server DLL.
|
||||
The test-only `scripts/Server.Tests/Server.Tests.csproj` console exercises the
|
||||
existing .NET storage client; it is not included in the server image.
|
||||
|
||||
Before seeding, the wrapper restores two synthetic legacy-schema rows into its
|
||||
fresh volume, verifies byte-identical revision-1 migration against a precomputed
|
||||
complete metadata/hash oracle, then checks repeated startup preserves history.
|
||||
The synthetic fixture is generated in code: one plate and one rectangular part,
|
||||
with no customer files. `seed` uses the actual `RemoteNestRepository` and
|
||||
`NestSaveSession` to create, update the same ID, and save a copy with a new ID.
|
||||
It also creates metadata-only, historical-restore, delete/recover/delete revisions,
|
||||
leaving a six-revision hidden record with a restore-source relationship.
|
||||
It checks read-back metadata, session binding, exact list membership, raw
|
||||
camelCase/string-enum JSON, identical downloaded bytes, and `NestReader` geometry
|
||||
and counts. `reject` snapshots every record's metadata and archive hash, then
|
||||
checks 400 responses for missing/invalid/null metadata and missing/empty files on
|
||||
both upload and file-update routes; unknown metadata/file IDs must return 404.
|
||||
Every rejected operation must leave the snapshot unchanged.
|
||||
`limits` stores, downloads byte-for-byte, and deletes a 29,000,000-byte synthetic
|
||||
`limits` stores, downloads byte-for-byte, and soft-deletes a 29,000,000-byte synthetic
|
||||
archive (buffered to disk by the non-root runtime), then requires 413 for a
|
||||
request over 30,000,000 bytes sent with `Expect: 100-continue`, and a reported
|
||||
failure from the real client's oversized upload and file update. The snapshot
|
||||
@@ -384,6 +460,10 @@ fresh named volume, starts a replacement on another allocated loopback port, and
|
||||
archive hashes, and exact list membership. It then follows the documented
|
||||
stopped-service backup: `tar` of the whole data directory through the image,
|
||||
restored into a second owned volume whose service must pass the same `verify`.
|
||||
The Python manifest is captured after all mutations and compared exactly after
|
||||
restart and volume restore, with page size 1 to exercise every pagination boundary.
|
||||
It covers active and deleted current records and ALL revisions, not just the
|
||||
ordinary `/api/nests` enumeration; the large soft-deleted archive remains included.
|
||||
The original volume mounted read-only, and a root-owned `tmpfs`, must each make
|
||||
the server exit nonzero at startup with a SQLite error and never report healthy.
|
||||
|
||||
|
||||
@@ -103,6 +103,20 @@ internal static class Program
|
||||
Check(copy.Id != Guid.Empty && copy.Id != created.Id, "Save copy must create a different id.");
|
||||
CheckBinding(session, copy.Id, options.Url);
|
||||
var copyState = await ReadSaved(repository, nest, archive, copy, ct);
|
||||
// Keep a hidden nest with metadata, restore-source and visibility history in the backup.
|
||||
var hidden = await repository.UploadAsync(archive, new NestRecord { Name = "Synthetic hidden history" }, ct);
|
||||
hidden.Comments = "Metadata-only revision";
|
||||
hidden = await repository.UpdateMetadataAsync(hidden.Id, hidden, hidden.Revision, ct);
|
||||
hidden = await repository.RestoreAsync(hidden.Id, 1, hidden.Revision, ct);
|
||||
Check(hidden.Revision == 3 && hidden.RestoredFromRevision == 1, "Restore must record its source.");
|
||||
hidden = await repository.DeleteAsync(hidden.Id, hidden.Revision, ct);
|
||||
hidden = await repository.RecoverAsync(hidden.Id, hidden.Revision, ct);
|
||||
hidden = await repository.DeleteAsync(hidden.Id, hidden.Revision, ct);
|
||||
Check(hidden.Deleted && hidden.Revision == 6, "Visibility changes must retain all revisions.");
|
||||
var hiddenHistory = await repository.GetDeletedRevisionsAsync(hidden.Id, limit: 2, cancellationToken: ct);
|
||||
Check(hiddenHistory.Total == 6 && hiddenHistory.Items.Count == 2, "Deleted history must be bounded.");
|
||||
Check((await repository.GetDeletedRevisionFileAsync(hidden.Id, 1, ct))!.SequenceEqual(archive),
|
||||
"Deleted history must retain original bytes.");
|
||||
var entries = baseline.Concat(new[] { updateState, copyState }).ToList();
|
||||
await CheckSnapshot(repository, entries, ct);
|
||||
var state = new SmokeState { Version = 1, Records = entries };
|
||||
@@ -153,6 +167,8 @@ internal static class Program
|
||||
if (test.File is not null)
|
||||
content.Add(new ByteArrayContent(test.File), "file", "synthetic.nest");
|
||||
using var request = new HttpRequestMessage(method, new Uri(new Uri(url + "/"), path)) { Content = content };
|
||||
if (method == HttpMethod.Put)
|
||||
request.Headers.TryAddWithoutValidation("If-Match", $"\"{before[0].Metadata.Revision}\"");
|
||||
using var response = await http.SendAsync(request, ct);
|
||||
Check(response.StatusCode == HttpStatusCode.BadRequest, $"{method} {test.Name} must return 400.");
|
||||
await CheckSnapshot(repository, before, ct);
|
||||
@@ -190,14 +206,15 @@ internal static class Program
|
||||
Check(downloaded is not null && downloaded.AsSpan().SequenceEqual(accepted),
|
||||
"A near-limit archive must download byte-for-byte.");
|
||||
await CheckMembership(repository, before.Select(s => s.Metadata.Id).Append(created.Id), ct);
|
||||
await repository.DeleteAsync(created.Id, ct);
|
||||
var deleted = await repository.DeleteAsync(created.Id, created.Revision, ct);
|
||||
Check(deleted.Deleted && deleted.Revision == 2, "Near-limit archive must be retained as deleted history.");
|
||||
await CheckSnapshot(repository, before, ct);
|
||||
Console.WriteLine($"Accepted, downloaded, and deleted a {accepted.Length}-byte archive; all other records unchanged.");
|
||||
|
||||
// Kestrel rejects an oversized declared length before the app reads the form. With
|
||||
// Expect: 100-continue the client waits for, and must receive, that 413 response.
|
||||
var oversized = new byte[RequestBodyLimit];
|
||||
var oversizedRecord = new NestRecord { Name = "Synthetic oversized upload" };
|
||||
var oversizedRecord = new NestRecord { Name = "Synthetic oversized upload", Revision = before[0].Metadata.Revision };
|
||||
var metadata = JsonSerializer.Serialize(oversizedRecord, JsonOptions);
|
||||
foreach (var method in new[] { HttpMethod.Post, HttpMethod.Put })
|
||||
{
|
||||
@@ -207,6 +224,8 @@ internal static class Program
|
||||
content.Add(new ByteArrayContent(oversized), "file", "synthetic.nest");
|
||||
using var request = new HttpRequestMessage(method, new Uri(new Uri(url + "/"), path)) { Content = content };
|
||||
request.Headers.ExpectContinue = true;
|
||||
if (method == HttpMethod.Put)
|
||||
request.Headers.TryAddWithoutValidation("If-Match", $"\"{before[0].Metadata.Revision}\"");
|
||||
using var response = await http.SendAsync(request, ct);
|
||||
Check(response.StatusCode == HttpStatusCode.RequestEntityTooLarge,
|
||||
$"{method} of a request over {RequestBodyLimit} bytes must return 413.");
|
||||
@@ -244,6 +263,7 @@ internal static class Program
|
||||
Check(returned.SavedAt != default, "Server must assign SavedAt.");
|
||||
var expected = NestRecordFactory.FromNest(nest, returned.Id, archive.LongLength);
|
||||
expected.SavedAt = returned.SavedAt;
|
||||
expected.Revision = returned.Revision;
|
||||
CheckMetadata(expected, returned);
|
||||
var readback = await repository.GetMetadataAsync(returned.Id, ct);
|
||||
Check(readback is not null, "Saved metadata must be readable.");
|
||||
@@ -360,6 +380,9 @@ internal static class Program
|
||||
Check(reader.Warnings.Count == 0, "NestReader must parse without warnings.");
|
||||
var parsed = NestRecordFactory.FromNest(nest, metadata.Id, archive.LongLength);
|
||||
parsed.SavedAt = metadata.SavedAt;
|
||||
parsed.Revision = metadata.Revision;
|
||||
parsed.Deleted = metadata.Deleted;
|
||||
parsed.RestoredFromRevision = metadata.RestoredFromRevision;
|
||||
CheckMetadata(metadata, parsed);
|
||||
Check(nest.Drawings.Count == 1 && nest.Plates.Count == 1 && nest.Plates[0].Parts.Count == 1,
|
||||
"Parsed archive must contain one drawing, plate, and part.");
|
||||
@@ -459,7 +482,15 @@ internal static class Program
|
||||
CheckWireRecord(record);
|
||||
}
|
||||
else
|
||||
CheckWireRecord(json.RootElement);
|
||||
{
|
||||
if (json.RootElement.TryGetProperty("items", out var items))
|
||||
{
|
||||
foreach (var record in items.EnumerateArray())
|
||||
CheckWireRecord(record);
|
||||
}
|
||||
else
|
||||
CheckWireRecord(json.RootElement);
|
||||
}
|
||||
CheckedResponses++;
|
||||
}
|
||||
return response;
|
||||
|
||||
@@ -0,0 +1,51 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Generate only synthetic legacy SQLite files inside caller-owned scratch."""
|
||||
import hashlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
import sqlite3
|
||||
import sys
|
||||
import tarfile
|
||||
|
||||
SCHEMA = """
|
||||
CREATE TABLE nests (
|
||||
id TEXT PRIMARY KEY, name TEXT NOT NULL, customer TEXT NOT NULL DEFAULT '',
|
||||
dateCreated TEXT NOT NULL, dateModified TEXT NOT NULL, material TEXT NOT NULL DEFAULT '',
|
||||
thickness REAL NOT NULL DEFAULT 0, status TEXT NOT NULL DEFAULT 'Quote',
|
||||
plateCount INTEGER NOT NULL DEFAULT 0, partCount INTEGER NOT NULL DEFAULT 0,
|
||||
comments TEXT NOT NULL DEFAULT '', madeBy TEXT NOT NULL DEFAULT '',
|
||||
fileSize INTEGER NOT NULL DEFAULT 0, savedAt TEXT NOT NULL, file BLOB NOT NULL);
|
||||
CREATE INDEX ix_nests_savedAt ON nests(savedAt DESC);
|
||||
"""
|
||||
|
||||
|
||||
def create(directory):
|
||||
db = directory / 'nests.db'
|
||||
expected = []
|
||||
with sqlite3.connect(db) as connection:
|
||||
connection.executescript(SCHEMA)
|
||||
for index in range(1, 3):
|
||||
ident = f'00000000-0000-4000-8000-{index:012d}'
|
||||
data = bytes([0, index, 255, 128, 0])
|
||||
date = '2026-01-02T03:04:05'
|
||||
fields = (ident, f'Synthetic legacy {index}', 'Synthetic fixture', date, date,
|
||||
'Synthetic alloy', 0.25, 'ToBeCut', 2, 3, 'Legacy notes', 'Fixture', len(data), date)
|
||||
connection.execute('INSERT INTO nests VALUES (' + ','.join('?' * 15) + ')', (*fields, data))
|
||||
metadata: dict[str, object] = dict(zip(('id', 'name', 'customer', 'dateCreated', 'dateModified', 'material',
|
||||
'thickness', 'status', 'plateCount', 'partCount', 'comments',
|
||||
'madeBy', 'fileSize', 'savedAt'), fields))
|
||||
metadata.update(status='toBeCut', revision=1, deleted=False, restoredFromRevision=None)
|
||||
archive = {'sha256': hashlib.sha256(data).hexdigest(), 'bytes': len(data)}
|
||||
expected.append({'current': metadata, 'archive': archive,
|
||||
'revisions': [{'metadata': metadata, 'archive': archive}]})
|
||||
(directory / 'legacy.manifest').write_text(json.dumps({'format': 1, 'records': expected}, sort_keys=True, indent=2) + '\n')
|
||||
with tarfile.open(directory / 'legacy.tar', 'w') as archive:
|
||||
info = archive.gettarinfo(str(db), arcname='./nests.db')
|
||||
info.uid = info.gid = 1654
|
||||
info.uname = info.gname = ''
|
||||
with db.open('rb') as source:
|
||||
archive.addfile(info, source)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
create(Path(sys.argv[1]))
|
||||
@@ -0,0 +1,100 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Read-only, canonical revision manifest; requires a quiescent version-aware server."""
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import sys
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
import uuid
|
||||
|
||||
|
||||
def require(condition, message):
|
||||
if not condition:
|
||||
raise ValueError(message)
|
||||
|
||||
|
||||
def collect(url, limit=200):
|
||||
require(1 <= limit <= 200, "Invalid page size")
|
||||
opener = urllib.request.build_opener(urllib.request.ProxyHandler({}))
|
||||
|
||||
def read(path, binary=False):
|
||||
try:
|
||||
response = opener.open(url.rstrip('/') + path, timeout=30)
|
||||
except urllib.error.HTTPError as error:
|
||||
error.close()
|
||||
raise ValueError("HTTP request failed") from None
|
||||
with response:
|
||||
if binary:
|
||||
digest = hashlib.sha256()
|
||||
size = 0
|
||||
while chunk := response.read(1024 * 1024):
|
||||
size += len(chunk)
|
||||
digest.update(chunk)
|
||||
return {"sha256": digest.hexdigest(), "bytes": size}
|
||||
return json.load(response)
|
||||
|
||||
def pages(path):
|
||||
items, total = [], None
|
||||
while True:
|
||||
page = read(f"{path}?offset={len(items)}&limit={limit}")
|
||||
require(page['offset'] == len(items) and page['limit'] == limit, "Page bounds differ")
|
||||
require(type(page['total']) is int and page['total'] >= 0, "Invalid total")
|
||||
if total is None:
|
||||
total = page['total']
|
||||
require(total == page['total'], "Server changed during enumeration")
|
||||
batch = page['items']
|
||||
require(isinstance(batch, list) and len(batch) <= limit, "Invalid page")
|
||||
require(len(items) + len(batch) <= total, "Page exceeds total")
|
||||
require(batch or len(items) == total, "Truncated enumeration")
|
||||
items.extend(batch)
|
||||
if len(items) == total:
|
||||
return items
|
||||
|
||||
result, ids = [], set()
|
||||
for deleted in (False, True):
|
||||
path = '/api/nests/deleted' if deleted else '/api/nests'
|
||||
records = pages(path + '/query')
|
||||
for record in records:
|
||||
ident = record['id']
|
||||
require(str(uuid.UUID(ident)) == ident and ident not in ids, "Invalid or duplicate nest ID")
|
||||
ids.add(ident)
|
||||
require(record['deleted'] is deleted, "Visibility differs")
|
||||
revisions = pages(path + '/' + ident + '/revisions')
|
||||
require([r['revision'] for r in revisions] == list(range(record['revision'], 0, -1)),
|
||||
"Missing, duplicate, or unordered revisions")
|
||||
require(revisions and revisions[0] == record, "Current projection differs from latest revision")
|
||||
history = []
|
||||
for revision in revisions:
|
||||
number = revision['revision']
|
||||
require(revision['id'] == ident, "Revision belongs to another nest")
|
||||
parent = revision['restoredFromRevision']
|
||||
require(parent is None or type(parent) is int and 1 <= parent < number,
|
||||
"Invalid restore source relationship")
|
||||
archive = read(f"{path}/{ident}/revisions/{number}/file", binary=True)
|
||||
require(archive['bytes'] == revision['fileSize'], "Revision archive size differs")
|
||||
history.append({"metadata": revision, "archive": archive})
|
||||
current = read(f"{path}/{ident}/file", binary=True)
|
||||
require(current == history[0]['archive'], "Current archive differs from latest revision")
|
||||
result.append({"current": record, "archive": current,
|
||||
"revisions": sorted(history, key=lambda r: r['metadata']['revision'])})
|
||||
return {"format": 1, "records": sorted(result, key=lambda r: r['current']['id'])}
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('url')
|
||||
parser.add_argument('--page-size', type=int, default=200)
|
||||
args = parser.parse_args()
|
||||
try:
|
||||
manifest = collect(args.url, args.page_size)
|
||||
# Emit only after every request and invariant succeeds; failure never emits a partial manifest.
|
||||
print(json.dumps(manifest, sort_keys=True, indent=2))
|
||||
except Exception as error:
|
||||
print(f"Manifest failed: {type(error).__name__}", file=sys.stderr)
|
||||
return 1
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,277 @@
|
||||
"""Manifest refusal tests and opt-in executable documentation/rollback rehearsal.
|
||||
|
||||
OPENNEST_TEST_IMAGE=<local-new-image> [OPENNEST_TEST_OLD_IMAGE=<local-old-image>]
|
||||
python3 -m unittest discover -s scripts/Server.Tests -p test_revision_backup.py -v
|
||||
Only generated loopback containers/volumes are used; no live server URL is accepted.
|
||||
"""
|
||||
import copy
|
||||
import hashlib
|
||||
import http.server
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import shutil
|
||||
import socket
|
||||
import subprocess
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
import unittest
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
import uuid
|
||||
|
||||
from nest_manifest import collect
|
||||
from legacy_fixture import create
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
|
||||
|
||||
class ManifestTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.ident = '00000000-0000-4000-8000-000000000001'
|
||||
self.record = dict(id=self.ident, revision=3, deleted=True, restoredFromRevision=1,
|
||||
fileSize=3, comments='restored', savedAt='2026-01-02T03:04:05')
|
||||
self.revisions = [self.record, dict(self.record, revision=2, deleted=False,
|
||||
restoredFromRevision=None, comments='edited'),
|
||||
dict(self.record, revision=1, deleted=False, restoredFromRevision=None)]
|
||||
self.failure = None
|
||||
owner = self
|
||||
|
||||
class Handler(http.server.BaseHTTPRequestHandler):
|
||||
def do_GET(self):
|
||||
path = urllib.parse.urlsplit(self.path).path
|
||||
query = urllib.parse.parse_qs(urllib.parse.urlsplit(self.path).query)
|
||||
if owner.failure == 'legacy' or owner.failure == 'download' and path.endswith('/file'):
|
||||
self.send_error(500 if owner.failure == 'download' else 404)
|
||||
return
|
||||
if path.endswith('/file'):
|
||||
data = b'abc'
|
||||
else:
|
||||
values = owner.revisions if path.endswith('/revisions') else ([owner.record] if '/deleted/' in path else [])
|
||||
offset, limit = int(query['offset'][0]), int(query['limit'][0])
|
||||
batch = values[offset:offset + limit]
|
||||
if owner.failure == 'truncated' and offset > 0:
|
||||
batch = []
|
||||
total = len(values) + (1 if owner.failure == 'total' and offset else 0)
|
||||
data = json.dumps(dict(items=batch, total=total, offset=offset, limit=limit)).encode()
|
||||
self.send_response(200)
|
||||
self.end_headers()
|
||||
self.wfile.write(data)
|
||||
|
||||
def log_message(self, format, *args):
|
||||
pass
|
||||
|
||||
self.server = http.server.ThreadingHTTPServer(('127.0.0.1', 0), Handler)
|
||||
self.thread = threading.Thread(target=self.server.serve_forever, daemon=True)
|
||||
self.thread.start()
|
||||
self.url = f'http://127.0.0.1:{self.server.server_port}'
|
||||
|
||||
def tearDown(self):
|
||||
self.server.shutdown()
|
||||
self.server.server_close()
|
||||
self.thread.join()
|
||||
|
||||
def test_all_deleted_revisions_and_parent_metadata_are_preserved(self):
|
||||
manifest = collect(self.url, 1)
|
||||
record = manifest['records'][0]
|
||||
self.assertEqual(self.record, record['current'])
|
||||
self.assertEqual([1, 2, 3], [r['metadata']['revision'] for r in record['revisions']])
|
||||
self.assertEqual(hashlib.sha256(b'abc').hexdigest(), record['archive']['sha256'])
|
||||
self.assertEqual(1, record['revisions'][2]['metadata']['restoredFromRevision'])
|
||||
altered = copy.deepcopy(manifest)
|
||||
altered['records'][0]['revisions'][0]['metadata']['comments'] = 'tampered'
|
||||
self.assertNotEqual(manifest, altered)
|
||||
|
||||
def test_failed_download_emits_no_partial_manifest(self):
|
||||
self.failure = 'download'
|
||||
result = subprocess.run(['python3', str(ROOT / 'scripts/Server.Tests/nest_manifest.py'), self.url], capture_output=True)
|
||||
self.assertNotEqual(0, result.returncode)
|
||||
self.assertEqual(b'', result.stdout)
|
||||
|
||||
def test_legacy_endpoint_is_not_empty_history(self):
|
||||
self.failure = 'legacy'
|
||||
with self.assertRaises(Exception):
|
||||
collect(self.url, 1)
|
||||
|
||||
def test_missing_pages_and_changed_totals_fail(self):
|
||||
for failure in ('truncated', 'total'):
|
||||
self.failure = failure
|
||||
with self.assertRaises(ValueError):
|
||||
collect(self.url, 1)
|
||||
|
||||
def test_missing_revision_and_invalid_parent_fail(self):
|
||||
self.revisions.pop(1)
|
||||
with self.assertRaises(ValueError):
|
||||
collect(self.url, 1)
|
||||
self.revisions.insert(1, dict(self.record, revision=2, restoredFromRevision=None))
|
||||
self.record['restoredFromRevision'] = 3
|
||||
with self.assertRaises(ValueError):
|
||||
collect(self.url, 1)
|
||||
|
||||
|
||||
@unittest.skipUnless(os.environ.get('OPENNEST_TEST_IMAGE'), 'Set OPENNEST_TEST_IMAGE for disposable Docker documentation rehearsal')
|
||||
class DocumentationTests(unittest.TestCase):
|
||||
def run_command(self, *args, input=None, check=True):
|
||||
result = subprocess.run(args, input=input, text=True, capture_output=True, timeout=120, cwd=self.scratch)
|
||||
if check and result.returncode:
|
||||
self.fail(f"Command failed ({result.returncode}): {args}\n{result.stdout}\n{result.stderr}")
|
||||
return result
|
||||
|
||||
def docker(self, *args, **kwargs):
|
||||
return self.run_command('docker', '--context', 'default', *args, **kwargs)
|
||||
|
||||
def setUp(self):
|
||||
self.temp = tempfile.TemporaryDirectory(prefix='opennest-backup-doc-', dir=os.environ.get('TMPDIR'))
|
||||
self.scratch = Path(self.temp.name)
|
||||
self.token = 'opennest-doc-' + uuid.uuid4().hex
|
||||
self.image = os.environ['OPENNEST_TEST_IMAGE']
|
||||
self.volumes = []
|
||||
self.containers = []
|
||||
self.compose = None
|
||||
self.addCleanup(self.cleanup_resources)
|
||||
endpoint = self.docker('context', 'inspect', 'default', '--format', '{{.Endpoints.docker.Host}}').stdout.strip()
|
||||
self.assertEqual('unix:///var/run/docker.sock', endpoint)
|
||||
self.volume = self.token + '-data'
|
||||
self.new_volume(self.volume)
|
||||
with socket.socket() as probe:
|
||||
probe.bind(('127.0.0.1', 0))
|
||||
port = probe.getsockname()[1]
|
||||
configuration = {'services': {'opennest-server': {'image': self.image,
|
||||
'ports': [f'127.0.0.1:{port}:8090'], 'volumes': [self.volume + ':/app/data'],
|
||||
'cap_drop': ['ALL'], 'security_opt': ['no-new-privileges:true']}},
|
||||
'volumes': {self.volume: {'external': True}}}
|
||||
(self.scratch / 'compose.json').write_text(json.dumps(configuration))
|
||||
self.compose = f'docker --context default compose -p {self.token} -f compose.json'
|
||||
self.run_command('bash', '-c', self.compose + ' up -d')
|
||||
self.cid = self.run_command('bash', '-c', self.compose + ' ps -q').stdout.strip()
|
||||
self.url = self.ready(self.cid)
|
||||
shutil.copy(ROOT / 'scripts/Server.Tests/nest_manifest.py', self.scratch / 'nest_manifest.py')
|
||||
doc = (ROOT / 'docs/nest-storage.md').read_text()
|
||||
blocks = re.findall(r'```bash\n(.*?)```', doc, re.S)
|
||||
selected = [block for block in blocks if 'backup_nests()' in block or 'restore_check()' in block]
|
||||
self.assertEqual(2, len(selected))
|
||||
self.functions = '\n'.join('\n'.join(line for line in block.splitlines()
|
||||
if not line.startswith(('url=', 'backup_nests opennest-', 'restore_check opennest-')))
|
||||
for block in selected)
|
||||
(self.scratch / 'procedures.sh').write_text(self.functions)
|
||||
self.run_command('bash', '-n', 'procedures.sh')
|
||||
# Opaque synthetic archives exercise history without any customer data.
|
||||
record = self.request('POST', '/api/nests', multipart=True)
|
||||
record['comments'] = 'metadata version'
|
||||
record = self.request('PUT', f"/api/nests/{record['id']}/metadata", record, record['revision'])
|
||||
record = self.request('POST', f"/api/nests/{record['id']}/revisions/1/restore", revision=record['revision'])
|
||||
self.request('DELETE', f"/api/nests/{record['id']}", revision=record['revision'])
|
||||
self.request('POST', '/api/nests', multipart=True)
|
||||
|
||||
def new_volume(self, name):
|
||||
self.assertNotEqual(0, self.docker('volume', 'inspect', name, check=False).returncode)
|
||||
self.docker('volume', 'create', '--label', 'com.opennest.doc-owner=' + self.token, name)
|
||||
self.assertEqual(self.token, self.docker('volume', 'inspect', '--format', '{{index .Labels "com.opennest.doc-owner"}}', name).stdout.strip())
|
||||
self.volumes.append(name)
|
||||
|
||||
def ready(self, cid):
|
||||
port = self.docker('port', cid, '8090/tcp').stdout.strip().split(':')[1]
|
||||
url = f'http://127.0.0.1:{port}'
|
||||
for _ in range(60):
|
||||
try:
|
||||
with urllib.request.urlopen(url + '/healthz', timeout=1) as response:
|
||||
if json.load(response) == {'status': 'ok'}:
|
||||
return url
|
||||
except OSError:
|
||||
time.sleep(1)
|
||||
self.fail('Disposable service did not become ready')
|
||||
|
||||
def request(self, method, path, data=None, revision=None, multipart=False):
|
||||
headers = {}
|
||||
if revision:
|
||||
headers['If-Match'] = f'"{revision}"'
|
||||
if multipart:
|
||||
boundary = 'synthetic-boundary'
|
||||
data = (f'--{boundary}\r\nContent-Disposition: form-data; name="metadata"\r\n\r\n'
|
||||
'{"name":"Synthetic documentation fixture"}\r\n'
|
||||
f'--{boundary}\r\nContent-Disposition: form-data; name="file"; filename="fixture.nest"\r\n'
|
||||
f'Content-Type: application/zip\r\n\r\nabc\r\n--{boundary}--\r\n').encode()
|
||||
headers['Content-Type'] = 'multipart/form-data; boundary=' + boundary
|
||||
elif data is not None:
|
||||
data = json.dumps(data).encode()
|
||||
headers['Content-Type'] = 'application/json'
|
||||
request = urllib.request.Request(self.url + path, data=data, headers=headers, method=method)
|
||||
with urllib.request.urlopen(request, timeout=10) as response:
|
||||
return json.load(response)
|
||||
|
||||
def procedure(self, command, check=True):
|
||||
prefix = f'compose={json.dumps(self.compose)}; url={self.url}; source ./procedures.sh; '
|
||||
return self.run_command('bash', '-c', prefix + command, check=check)
|
||||
|
||||
def cleanup_resources(self):
|
||||
# Compose removes only this randomly generated project's containers, never its external volumes.
|
||||
try:
|
||||
if self.compose:
|
||||
self.run_command('bash', '-c', self.compose + ' down')
|
||||
for cid in self.containers:
|
||||
self.docker('rm', '-f', cid)
|
||||
for volume in self.volumes:
|
||||
labels = json.loads(self.docker('volume', 'inspect', '--format', '{{json .Labels}}', volume).stdout)
|
||||
self.assertTrue(labels.get('com.opennest.doc-owner') == self.token
|
||||
or labels.get('com.opennest.restore.owner') == volume and volume.startswith(self.token + '-'))
|
||||
self.assertEqual('', self.docker('container', 'ls', '-aq', '--filter', 'volume=' + volume).stdout.strip(),
|
||||
'A documentation check container was not removed')
|
||||
self.docker('volume', 'rm', volume)
|
||||
self.assertNotEqual(0, self.docker('volume', 'inspect', volume, check=False).returncode)
|
||||
self.assertEqual('', self.docker('container', 'ls', '-aq', '--filter', 'label=com.docker.compose.project=' + self.token).stdout.strip())
|
||||
finally:
|
||||
self.temp.cleanup()
|
||||
|
||||
def test_verbatim_backup_restore_and_refusals(self):
|
||||
self.procedure('backup_nests backup')
|
||||
original = json.loads((self.scratch / 'backup.manifest').read_text())
|
||||
self.assertEqual(2, len(original['records']))
|
||||
self.assertEqual(5, sum(len(r['revisions']) for r in original['records']))
|
||||
started = self.docker('inspect', '--format', '{{.State.StartedAt}}', self.cid).stdout
|
||||
refused = self.procedure('if backup_nests backup; then exit 90; fi')
|
||||
self.assertIn('already exists', refused.stderr)
|
||||
self.assertEqual(started, self.docker('inspect', '--format', '{{.State.StartedAt}}', self.cid).stdout)
|
||||
restored = self.token + '-restored'
|
||||
refused = self.procedure(f'if restore_check backup {self.image} {self.volume}; then exit 90; fi')
|
||||
self.assertIn('already exists', refused.stderr)
|
||||
self.volumes.append(restored)
|
||||
self.procedure(f'restore_check backup {self.image} {restored}')
|
||||
tampered = self.token + '-tampered'
|
||||
self.volumes.append(tampered)
|
||||
original['records'][0]['revisions'][0]['archive']['sha256'] = '0' * 64
|
||||
(self.scratch / 'backup.manifest').write_text(json.dumps(original))
|
||||
refused = self.procedure(f'if restore_check backup {self.image} {tampered}; then exit 90; fi')
|
||||
self.assertIn('archives differ', refused.stderr)
|
||||
|
||||
@unittest.skipUnless(os.environ.get('OPENNEST_TEST_OLD_IMAGE'), 'Set OPENNEST_TEST_OLD_IMAGE for old-image rollback rehearsal')
|
||||
def test_old_image_only_on_pre_migration_backup_copy(self):
|
||||
create(self.scratch)
|
||||
rollback = self.token + '-rollback'
|
||||
self.new_volume(rollback)
|
||||
# Restore synthetic pre-migration tar to a NEW owned volume, never the upgraded database.
|
||||
with (self.scratch / 'legacy.tar').open('rb') as archive:
|
||||
subprocess.run(['docker', '--context', 'default', 'run', '--rm', '-i', '--network', 'none',
|
||||
'--entrypoint', 'tar', '-v', rollback + ':/app/data', self.image,
|
||||
'-C', '/app/data', '-xf', '-'], stdin=archive, check=True, timeout=30, capture_output=True)
|
||||
cid = self.docker('run', '-d', '--cap-drop', 'ALL', '--security-opt', 'no-new-privileges:true',
|
||||
'-p', '127.0.0.1::8090', '-v', rollback + ':/app/data',
|
||||
os.environ['OPENNEST_TEST_OLD_IMAGE']).stdout.strip()
|
||||
self.containers.append(cid)
|
||||
url = self.ready(cid)
|
||||
expected = json.loads((self.scratch / 'legacy.manifest').read_text())['records']
|
||||
with urllib.request.urlopen(url + '/api/nests') as response:
|
||||
actual = {r['id']: r for r in json.load(response)}
|
||||
self.assertEqual(2, len(actual))
|
||||
for entry in expected:
|
||||
record = {k: v for k, v in entry['current'].items() if k not in ('revision', 'deleted', 'restoredFromRevision')}
|
||||
self.assertEqual(record, actual[record['id']])
|
||||
with urllib.request.urlopen(url + f"/api/nests/{record['id']}/file") as response:
|
||||
self.assertEqual(entry['archive']['sha256'], hashlib.sha256(response.read()).hexdigest())
|
||||
with self.assertRaises(Exception):
|
||||
collect(url)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -23,7 +23,7 @@ while (($#)); do
|
||||
esac
|
||||
done
|
||||
[[ -n "$image" && "$image" != -* ]] || { usage; exit 2; }
|
||||
for command in docker dotnet curl timeout mktemp tar; do
|
||||
for command in docker dotnet curl timeout mktemp tar python3 cmp; do
|
||||
command -v "$command" >/dev/null || { printf 'Missing prerequisite: %s\n' "$command" >&2; exit 2; }
|
||||
done
|
||||
root=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd)
|
||||
@@ -40,7 +40,7 @@ token=${scratch##*/}
|
||||
owner_label='com.opennest.server-smoke.owner'
|
||||
# Two service generations on one data volume, a stopped-service backup and restore into
|
||||
# a second volume, its service, then two storage locations that must fail startup.
|
||||
container_roles=(first replacement backup restore restored readonly unwritable)
|
||||
container_roles=(first replacement backup restore restored readonly unwritable legacy)
|
||||
container_names=()
|
||||
cidfiles=()
|
||||
container_attempted=()
|
||||
@@ -306,16 +306,29 @@ run_tool() {
|
||||
|
||||
# A fresh named volume must be writable by the non-root image user.
|
||||
create_volume 0
|
||||
# Migrate a synthetic prior-schema database, never an existing service's data.
|
||||
python3 "$root/scripts/Server.Tests/legacy_fixture.py" "$scratch"
|
||||
create_container 7 --rm --interactive --network none --entrypoint tar \
|
||||
--mount "type=volume,src=${volumes[0]},dst=/app/data" "$image" -C /app/data -xf -
|
||||
docker_local start --attach --interactive "$cid" < "$scratch/legacy.tar" > "$run_results/legacy.log" 2>&1
|
||||
service=first
|
||||
start_service 0 "${volumes[0]}"
|
||||
run_tool seed --state "$scratch/state.json"
|
||||
python3 "$root/scripts/Server.Tests/nest_manifest.py" "$url" --page-size 1 > "$scratch/migrated.manifest"
|
||||
cmp "$scratch/legacy.manifest" "$scratch/migrated.manifest"
|
||||
run_tool seed --state "$scratch/state.json" --test-allow-nonempty
|
||||
python3 "$root/scripts/Server.Tests/nest_manifest.py" "$url" --page-size 1 > "$scratch/before-reject.manifest"
|
||||
run_tool reject
|
||||
python3 "$root/scripts/Server.Tests/nest_manifest.py" "$url" --page-size 1 > "$scratch/after-reject.manifest"
|
||||
cmp "$scratch/before-reject.manifest" "$scratch/after-reject.manifest"
|
||||
run_tool limits
|
||||
python3 "$root/scripts/Server.Tests/nest_manifest.py" "$url" --page-size 1 > "$scratch/all.manifest"
|
||||
retire_service 0
|
||||
# The replacement gets another Docker-allocated loopback port; only the volume is reused.
|
||||
service=replacement
|
||||
start_service 1 "${volumes[0]}"
|
||||
run_tool verify --state "$scratch/state.json"
|
||||
python3 "$root/scripts/Server.Tests/nest_manifest.py" "$url" --page-size 1 > "$scratch/reopened.manifest"
|
||||
cmp "$scratch/all.manifest" "$scratch/reopened.manifest"
|
||||
retire_service 1
|
||||
|
||||
# Back up the whole data directory from the stopped service (SQLite WAL makes a live
|
||||
@@ -335,4 +348,7 @@ docker_local start --attach --interactive "$cid" < "$scratch/data-backup.tar" >
|
||||
service=restored
|
||||
start_service 4 "${volumes[1]}"
|
||||
run_tool verify --state "$scratch/state.json"
|
||||
python3 "$root/scripts/Server.Tests/nest_manifest.py" "$url" --page-size 1 > "$scratch/restored.manifest"
|
||||
cmp "$scratch/all.manifest" "$scratch/restored.manifest"
|
||||
printf 'PASS: legacy migration, idempotent restart, active/deleted ALL-revision metadata, parent relationships and byte hashes match after restore.\n'
|
||||
printf 'PASS: non-root healthy runtime, real-client round trips, rejections, upload limit, persistent recreation, stopped-service backup/restore, and storage startup failures.\n'
|
||||
Reference in new issue
Block a user