Fleet changelogs · dev.ecs0.net
rdmpw3275m-changelog-20260916-2201-tyrell-build26-fleet-release-and-postdeploy-soak

Tyrell Build 26 fleet release and post-deploy soak

Tyrell 0.2.0 Build 26 was built from clean, lineage-complete source, fully tested, Developer ID signed and notarized, deployed with retained rollback to all six Macs, and independently verified online. The original tyrelld Offline incident is now closed as a stable-runtime-path defect; a fresh guarded post-deploy memory soak remains active and has not yet reached any acceptance gate.

Scope

Why tyrelld was Offline

The original outage was not an unexplained daemon crash. The per-user LaunchAgent referenced the mutable SwiftPM alias /Users/richh/dev/apps/Tyrell/.build/release/tyrelld. SwiftPM had rewritten that alias to .build/out/Products/Release, whose target later disappeared. After reboot, launchd could not spawn the executable and recorded exit 78 (EX_CONFIG); loopback ports 43117 and 43118 were therefore silent, so Tyrell.app correctly displayed Offline. Tailscale was also down during the original incident, but that was secondary because the app's local loopback health checks already failed.

The durable fix originated in commit 869503a: install a verified universal daemon into a retained hash-addressed release under ~/Library/Application Support/Tyrell/releases/, then atomically point the stable ~/Library/Application Support/Tyrell/bin/tyrelld link and LaunchAgent at it. Build 26 carries and exercises that lifecycle on all six hosts. Existing ISSUE-20260905-23 was the correct bug record, so no duplicate was filed; it was resolved after fresh six-host verification.

Build and artifact evidence

The first Aqua release wrapper correctly built, signed, and notarized both artifacts but then exited because it attempted to parse JSON with PlistBuddy. That evidence-only finalization defect was corrected to use a JSON parser. The accepted artifacts were not rebuilt or overwritten; the corrected finalizer revalidated source, dependencies, identities, signatures, notarization, and packaging. The original failed wrapper is retained as build-release-aqua.failed-v1.zsh; the corrected wrapper and finalizer remain beside it in the canonical release packet.

Deployment and verification

Minor operator-side orchestration errors were caught before causing product-state loss and are retained rather than hidden: an invalid hash validator in the first canary daemon attempt; a local quoting error after the hub daemon had already deployed; a verifier pipeline exit 141 caused by pipefail plus early awk; and a JSON-summary quoting error in the first warm-up evidence formatter. Each corrected rerun passed. No failed attempt changed the accepted artifacts or bypassed rollback.

Post-deploy soak

The exact fleet-deployed Build 26 daemon began a fresh protected soak on rdmbair15m5 at 2026-09-16 21:46:14 EDT:

No short, six-hour, or 24-hour Build 26 memory acceptance is claimed in this record. The active Codex automation finish-tyrell-build-16 was renamed to Monitor Tyrell Build 26 post-deploy soak, remains active at a 15-minute cadence, and now pins these exact identities and gate times. Its TOML SHA-256 after update is a7d744f85a10aab2d397ab268e8bf22aac699d40cafb3e02e2bd8889820c12d6.

Files and records changed

Undo and recovery

The deployment harness retained the exact previous Build 17 app and Build 15 daemon on every host, plus a pre-deploy database backup that independently passed quick_check. The canary guard automatically restores those exact identities on a guard, identity, power, lid, or product failure. For another host, use the retained host-specific rollback paths and receipts in the canonical release evidence; verify app/daemon hashes, one process each, both status planes, database integrity, and launchd autoload after rollback. Do not restore or overwrite a live database unless integrity or migration evidence specifically requires it.

A laptop power or lid event is an environmental canary interruption and does not by itself authorize fleet rollback. A reproducible product or identity failure on the exact Build 26 artifacts requires a coordinator classification and then rollback using the retained per-host payloads.

Outstanding work