Tyrell Build 26 fleet release and post-deploy soak

Tyrell 0.2.0 Build 26 was built from clean, lineage-complete source, fully tested, Developer ID signed and notarized, deployed with retained rollback to all six Macs, and independently verified online. The original tyrelld Offline incident is now closed as a stable-runtime-path defect. The first guarded post-deploy memory run ended safely before its short gate when the laptop lid closed: the canary restored exact Build 17/15, while Build 26 remains online on the other five hosts and under active monitoring.

Scope

• Coordinator and evidence host: rdmpw3275m
• Canonical source/signing host: rdmsm4x
• Designated canary: rdmbair15m5
• Additional deployment targets: rdmbair13m5, rdmpw3265m, jdmbair13m5
• Owner decision: DEC-20260916-04 — deploy the latest Build 26 to the fleet and soak the deployed build, superseding another pre-deploy 24-hour laptop-canary attempt for this release
• Source worktree: /Users/richh/dev/_worktrees/tyrell-build26-cut-20260916
• Source branch and HEAD: claude/tyrell-build26-cut-20260916 at 40145fd912afe8b32ae7ee64c24950303a49eb7a
• Source tree: fbf1bdb43396db16e070101d9db707fb17fd9fb0

Why tyrelld was Offline

The original outage was not an unexplained daemon crash. The per-user LaunchAgent referenced the mutable SwiftPM alias /Users/richh/dev/apps/Tyrell/.build/release/tyrelld. SwiftPM had rewritten that alias to .build/out/Products/Release, whose target later disappeared. After reboot, launchd could not spawn the executable and recorded exit 78 (EX_CONFIG); loopback ports 43117 and 43118 were therefore silent, so Tyrell.app correctly displayed Offline. Tailscale was also down during the original incident, but that was secondary because the app's local loopback health checks already failed.

The durable fix originated in commit 869503a: install a verified universal daemon into a retained hash-addressed release under ~/Library/Application Support/Tyrell/releases/, then atomically point the stable ~/Library/Application Support/Tyrell/bin/tyrelld link and LaunchAgent at it. Build 26 carries and exercises that lifecycle on all six hosts. Existing ISSUE-20260905-23 was the correct bug record, so no duplicate was filed; it was resolved after fresh six-host verification.

Build and artifact evidence

• Full swift test exited 0. Swift Testing summaries total 727 cases with zero failures; XCTest reported zero failures. Raw log: /Users/richh/dev/_handoff/tyrell-build26-release-20260916/swift-test.log, SHA-256 515c86665c2e90cc53bc0644b3c5a73041ae22a7d913dcecf39e371b3c213d87.
• App executable SHA-256: 75d41b417c393fb71ee9483836dcf1397236f207a2919faeabb3e74e9c1e9d2c.
• App arm64 full CodeDirectory SHA-256: 5e230723fa522635d852dccb15bfebe64833daa26f8a9c0fbb8ab95c8ebb4031.
• Designated-requirement SHA-256: 9943c03ba47653aab0eacfd8fab8ea70d887d4392fd24832b96bffaa9c502aee.
• Daemon SHA-256: a4f43d83eb69113ed05d22b8f0b8521e09f71cd3475857e21d89e2572efea6e4.
• App notarization submission: b1552b90-42c3-44ff-a352-f48480d3860f.
• Daemon notarization submission: 321c684b-41e4-48db-9f49-67d63c618e38.
• Both executables are universal2, Developer ID signed by Team ZU2882L4HT, and hardened-runtime enabled. The app is stapled and Gatekeeper accepted.
• Independent artifact verifier exited 0; log SHA-256 4c67b59e304c47d8dc1965ee562926ebbb10d451f832508da64d5913a8e98529.

The first Aqua release wrapper correctly built, signed, and notarized both artifacts but then exited because it attempted to parse JSON with PlistBuddy. That evidence-only finalization defect was corrected to use a JSON parser. The accepted artifacts were not rebuilt or overwritten; the corrected finalizer revalidated source, dependencies, identities, signatures, notarization, and packaging. The original failed wrapper is retained as build-release-aqua.failed-v1.zsh; the corrected wrapper and finalizer remain beside it in the canonical release packet.

Deployment and verification

• Deployment nonce: tyrell-b26-20260916T212700Z-40145fd.
• Canary app probe SHA-256: f87dd46ad5f5aef554c8a3fa8bf6c47f765638969d0ed11acb0c1433aa0c2105.
• Canary receipt SHA-256: ba947cb9156503f0970cdabe2beb8ef477a3f7327755decf598df6316113fa71.
• The exact app and daemon were installed on all six hosts. rdmsm4x runs the server role; the other five run clients.
• Fresh independent verification on every host found exactly one Tyrell app, one TyrellBar, and one tyrelld; exact app and daemon hashes; correct roles; both /api/status planes at HTTP 200 with valid JSON; live database quick_check=ok; retained Build 17 app and Build 15 daemon rollback; and a coherent Build 26 pre-deploy database backup.
• Stable-runtime re-verification exited 0 on all six hosts: every LaunchAgent uses /Users/richh/Library/Application Support/Tyrell/bin/tyrelld, resolving to releases/daemon-a4f43d83eb69/tyrelld; no .build runtime remains. Log SHA-256: 0afd4081f6a94b25c57f6a2905cf34299188defaf0a2557f11f3c603f90c24a2.
• Canonical release evidence: /Users/richh/dev/_handoff/tyrell-build26-release-20260916.

Minor operator-side orchestration errors were caught before causing product-state loss and are retained rather than hidden: an invalid hash validator in the first canary daemon attempt; a local quoting error after the hub daemon had already deployed; a verifier pipeline exit 141 caused by pipefail plus early awk; and a JSON-summary quoting error in the first warm-up evidence formatter. Each corrected rerun passed. No failed attempt changed the accepted artifacts or bypassed rollback.

Post-deploy soak R1

The exact fleet-deployed Build 26 daemon began a fresh protected soak on rdmbair15m5 at 2026-09-16 21:46:14 EDT:

• Guard PID 71421; allocation sidecar PID 71841; guard-owned caffeinate PID 71845; daemon PID 72820.
• Live stage: /Users/richh/.tyrell/build26-postdeploy-soak-20260916-40145fd.
• Persistent reboot/guard-loss interlock is armed. Normal daemon autoload is deliberately disabled while the already-loaded guarded process runs.
• Exact Build 17 app and Build 15 daemon rollback passed preflight and remain retained.
• The frozen launch snapshot contains 11 valid samples through 601.022871 seconds. Every sample has one PID/start/path, AC power, open clamshell, valid HTTP 200 status planes, descriptors 25–27, and exactly two TCP descriptors.
• Physical footprint began at 277,562,664 bytes, reached a diagnostic first-inventory warm-up maximum of 486,786,656 bytes, and fell to 110,495,064 bytes (105.376 MiB) at the first post-warm-up sample. This is neither a pass nor a failure; the 300 MiB ceiling is evaluated over the protected measurement windows after warm-up.
• Frozen launch/warm-up packet: /Users/richh/dev/_handoff/tyrell-build26-release-20260916/postdeploy-soak-launch-evidence-20260916-2157; manifest SHA-256 f9c8dcf84e71bc5bd0aaf4847168cecbaee50ca9673c46a1e5bcd77266e0dfc6.
• The nominal short gate would have occurred at about 2026-09-16 22:26:15 EDT, but the canary clamshell closed at 22:21:26 EDT before any short receipt existed. The continuous guard failed closed and restored the retained Build 17/15 pair.
• R1 produced 35 samples through 2,041.299 seconds. Every sampled status response on ports 43117 and 43118 was HTTP 200 with valid JSON; the same candidate PID/start/path persisted through the last sample. The final two physical footprints were 110,986,608 and 111,052,144 bytes, with 27 descriptors and two TCP descriptors.
• Independent rollback inspection at 22:24 EDT found exact Build 17 app SHA-256 8693ebce3d474fed049597d11e3579bf1cd32f7a78acfd9544c78d77592da05e, exact Build 15 daemon SHA-256 dbc440bd54b85a54dd6c4150b23b3e508bff7d8d8080687d81b9ba7ac2cbbcf0, one app/bar/daemon, the managed stable launchd path, both APIs HTTP 200, database quick_check=ok, recovery job absent, and no candidate/guard/sidecar/caffeinate residue.
• Classification is ENVIRONMENTAL_INCONCLUSIVE_BEFORE_SHORT_GATE, not a Build 26 memory rejection. No R1 clock carries forward and no duplicate product bug was filed.
• Canonical immutable abort evidence: /Users/richh/dev/_handoff/tyrell-build26-release-20260916/postdeploy-r1-environmental-abort-evidence-20260916-2224; MANIFEST.sha256 SHA-256 2cdae45ef476916b2cd1bf6705bc346407d842aa3f54517c684569dbcb7e1004.

No short, six-hour, or 24-hour Build 26 memory acceptance is claimed in this record. Build 26 remains installed and online on rdmsm4x, rdmpw3275m, rdmbair13m5, rdmpw3265m, and jdmbair13m5; only rdmbair15m5 is back on Build 17/15. The active Codex automation finish-tyrell-build-16 remains active at a 15-minute cadence. While the laptop lid is closed it monitors the five Build 26 hosts, including consecutive rdmpw3275m observations through inventory completion. It may start only a distinct R2 after a fresh fingerprint-pinned AC/open preflight, with a new stage and every gate clock at zero. Automation TOML SHA-256 after this correction: 55452836876d31cc9e62bed04a227163f303079c52d61dd80f015bedaa26fc34.

Bounded fleet footprint diagnostic

Two read-only fleet samples approximately 92 seconds apart found physical footprints of: rdmsm4x 196→196 MB; rdmbair15m5 106→106 MB; rdmpw3275m 631→631 MB; rdmbair13m5 33→33 MB; rdmpw3265m 132→131 MB; and jdmbair13m5 44→44 MB. All exact Build 26 PIDs remained continuous and their APIs remained healthy.

The 631 MB Intel observation is above the later canary ceiling, but it was flat across the bounded pair while a direct process sample placed the active thread inside ClientLoop.autoInventory / TyrellScanner.scan; the status plane still exposed an incomplete 22,348/256,402 inventory, and the mount-independent libproc probe found only 27 descriptors and two TCP descriptors. This is not evidence of the prior pipe leak and is not yet a memory-gate verdict. Existing memory ticket ISSUE-20260912-01 was updated; no duplicate bug or fleet rollback was triggered. The required follow-up is consecutive sampling through inventory completion and escalation only for sustained growth, post-inventory above-bound footprint, identity/endpoint failure, or an authoritative guard verdict.

The reusable read-only sampler is /Users/richh/dev/_handoff/tyrell-build26-release-20260916/sample-build26-fleet-footprint.zsh, SHA-256 e8fab8c8938601b48af7824a18e5dde525f50f8befc92d30d82e9f05bb48bcff. Sample logs: fleet-footprint-sample-a.jsonl SHA-256 ed32e1ef07e04a21cdd36fc3be5e84eb26ffa01984116c5fe375c84e97256c82; fleet-footprint-sample-b.jsonl SHA-256 e4542ad8050619145f9ac2d3dc4e1e661093b54b968d6745ad0e97bda9b1a2ff.

Files and records changed

• Installed /Applications/Tyrell.app on all six hosts with exact Build 26 app identity.
• Installed retained daemon release ~/Library/Application Support/Tyrell/releases/daemon-a4f43d83eb69/tyrelld on all six hosts and repointed the stable bin/tyrelld link.
• Updated per-host LaunchAgent runtime state through the guarded installer; did not weaken firewall, SIP, TCC, or Tailscale policy.
• Created coherent per-host pre-deploy database backups and retained predecessor app/daemon releases.
• Resolved canonical ticket ISSUE-20260905-23; commented precise deployment, soak, environmental-abort, and rollback boundaries on ISSUE-20260914-15 and ISSUE-20260912-01. No duplicate bug was filed for the expected guard action.
• Updated canonical /Users/richh/dev/PROJECTS.md first for deployment and then by compare-and-swap from SHA-256 ed360f9faace09003723f83e7d1d8368bff4f2871c6a6a21ad0fa92dcb440a68 to 8308c36367ed7e37cfa868eb288e1e78e1353601ce08ae47813b783faa14f9bc for the R1 environmental closeout.
• Updated fleet check-in /Users/richh/.agent-coordination/checkins/codex-rdmpw3275m-tyrell-build25-canary-20260915.json; JSON validation passed and the fleet check-in synchronizer distributed the environmental closeout.
• Updated automation /Users/richh/.codex/automations/finish-tyrell-build-16/automation.toml, SHA-256 55452836876d31cc9e62bed04a227163f303079c52d61dd80f015bedaa26fc34, to preserve R1 as immutable, monitor the five live Build 26 hosts, launch only a fresh R2 from a verified AC/open state, and record AGY completion without falsely pinning the later RTTy head.
• Coordination milestones: deployment message 20260916-220001-62E38515; R1 environmental-closeout message 20260916-223542-09009EF6.
• AGY work is complete at RTTy commit c723b4f2b74f131272c832e134354b6e0a184a0e; the original parent is gone and must not be resumed again. Fresh verification found that commit in the ancestry of the current clean canonical RTTy main at 172e06c25deeb80bb5cfc5bebac02aca1d76201d; unrelated later work legitimately advanced main.

Undo and recovery

The deployment harness retained the exact previous Build 17 app and Build 15 daemon on every host, plus a pre-deploy database backup that independently passed quick_check. The canary guard automatically restores those exact identities on a guard, identity, power, lid, or product failure. For another host, use the retained host-specific rollback paths and receipts in the canonical release evidence; verify app/daemon hashes, one process each, both status planes, database integrity, and launchd autoload after rollback. Do not restore or overwrite a live database unless integrity or migration evidence specifically requires it.

A laptop power or lid event is an environmental canary interruption and does not by itself authorize fleet rollback. A reproducible product or identity failure on the exact Build 26 artifacts requires a coordinator classification and then rollback using the retained per-host payloads.

Outstanding work

• Continue bounded monitoring of the five Build 26 hosts. In particular, sample rdmpw3275m through inventory completion before classifying its currently flat, mid-inventory footprint.
• When rdmbair15m5 is freshly fingerprint-pinned on AC with its lid open, prepare and launch a distinct protected R2 with a new stage/nonce and every acceptance clock at zero; then independently verify its short, six-hour, and fixed 24-hour gates.
• Keep ISSUE-20260912-01, ISSUE-20260909-12, and ISSUE-20260914-15 open until their actual criteria are met.
• Do not merge or cut a future release from current main while ISSUE-20260915-18 remains open; Build 26 was cut from the approved lineage-complete base.
• Delete the monitor automation only after legitimate Build 26 completion or definitive rollback closeout.

Durable-record receipt

• Canonical file-copy name: /Users/richh/dev/LLM/Claude/changelogs/rdmpw3275m-changelog-20260916-2201-tyrell-build26-release.md.
• Apple Notes folder: rdmpw3275m.
• Apple Notes title: rdmpw3275m-changelog-20260916-2201-tyrell-build26-release.
• Apple Notes record ID was captured and independently verified after the final idempotent write; it is retained in the coordinator check-in rather than embedded here, because replacing the note necessarily changes its Core Data object ID.
• Independent verification found exactly one matching note and confirmed both the Build 26 app hash and frozen-soak manifest markers in its body.
• The legacy helper initially wrote one record to llmlog, contrary to current rule 26, and two first manual copies acquired Notes-truncated titles. Those three records were deleted after the correct per-host record was independently verified; Apple Notes deletion is recoverable through Recently Deleted until Notes retention expires.