rdmpw3275m-changelog-20260904-1405-tyrell-heatmap-regression-fix-and-daemon-deploy

Resolved the Tyrell 24-hour activity heatmap regression on rdmpw3275m and canonical main (rdmsm4x), staged and started the universal tyrelld spoke daemon on rdmpw3275m, and verified portfolio stability across macOS-ramDisk-app and RTTy.

1. Scope

• Hosts: rdmpw3275m (Intel Core i9, macOS 26.7) and rdmsm4x (M4 Max, macOS 27.0, canonical hub).
• Projects: ~/dev/apps/Tyrell, ~/dev/apps/RTTy, ~/dev/apps/macOS-ramDisk-app.

2. Root Cause Analysis

1. Daemon Route Truncation (/api/activity):
   • In rdmsm4x:~/dev/apps/Tyrell/Sources/tyrelld/Daemon.swift, /api/activity was hardcoded to env.store.recentEvents(limit: 100).
   • The 24-hour heatmap (FleetHeatmapView / ObservatoryView) requires 36 columns across 86,400s (~692 to 2,200 events across 6 hosts).
   • Hardcoded limit: 100 only returned ~1 to 2 hours of events, leaving columns 0–32 (~85% of the 24h grid) blank.
   • High-frequency polling on /api/activity caused SQLite read lock contention and disk I/O.
2. Spoke Host Inactivity on rdmpw3275m:
   • Host rebooted at 04:52 EDT into login console (/dev/console root, no Aqua GUI session).
   • tyrelld was not running; launchctl GUI domain gui/501 was unavailable.
   • Host lacked newer release b15-1525565e86f6 and its config was targeting rdmsm4x.ts.dataroo.net:43117 while Tailscale routing flapped.

3. Changes Applied

A. Tyrell Canonical main (rdmsm4x:~/dev/apps/Tyrell)

• Integrated Commit af4aeaa (ISSUE-20260903-31):
  • ActivityEventBufferActor.swift: Added in-memory ring buffer (capacity 5,000 events) absorbing bursts and calculating single-pass 36-column heatmap buckets.
  • SQLiteStateStore.swift & StateStore.swift: Added recentEvents(since:limit:) with default limit 5,000, warming ActivityEventBufferActor on initialization.
  • Daemon.swift: Parameterized /api/activity to support limit, since, and host query parameters via activityBuffer.events(). Added /api/activity/heatmap endpoint for sub-millisecond bucketing.
  • ObservatoryModels.swift: Added ObservatoryFleetReceiptProjection and ObservatoryDataProcessor for precomputed heatmap rendering.
  • AppStatusFeed.swift & ObservatoryView.swift: Switched to background detached observatory precomputation with event cache merging.
  • Unit Tests: Added ObservatoryDataProcessorTests.swift and ActivityEventBufferActorTests.swift.

B. Daemon Deployment on rdmpw3275m

• Staged Mach-O universal binary (x86_64 + arm64) to /Users/richh/Library/Application Support/Tyrell/releases/b15-1525565e86f6/tyrelld.
• Updated /Users/richh/Library/LaunchAgents/com.eastcoastscience.tyrelld.plist binary path.
• Updated /Users/richh/.tyrell/config.json defaultMaster to rdmsm4x:43117 (LAN-verified 0.35ms connection).
• Launched tyrelld under managed supervisor in tmux session agy:tyrelld.

4. Verification Evidence

1. Tyrell Daemon & Heatmap API Verification (rdmsm4x):
   • swift test --filter ActivityEventBufferActorTests: 100% pass (0 failures).
   • swift test --filter ObservatoryDataProcessorTests: 100% pass (0 failures).
   • curl -s http://192.168.0.29:43117/api/activity | jq 'length': 2,197 events (previously truncated to 100).
   • curl -s http://192.168.0.29:43117/api/activity/heatmap: HTTP 200, response latency 19 ms.
   • Heatmap bucket output confirms all 6 fleet hosts populated across all 36 columns:
     • rdmpw3275m, rdmsm4x, rdmbair15m5, rdmbair13m5, jdmbair13m5, rdmpw3265m.
2. Spoke Daemon Operation (rdmpw3275m):
   • Process running: PID 14718 (/Users/richh/Library/Application Support/Tyrell/releases/b15-1525565e86f6/tyrelld).
   • Local inventory completed: 207,336 files scanned, 97.48 GB indexed.
   • Central status confirmed on hub: rdmpw3275m reporting live census (gemini: 1, codex: 28) and sync progress.
3. macOS-ramDisk-app Portfolio State:
   • 11 of 11 tests pass (100% green).
   • Clean git status at commit f361d1d.
   • Verified published across all 3 remotes on rdmsm4x.
4. RTTy Portfolio State:
   • Clean working tree at commit 10be51d.
   • swift build passes with exit code 0.
   • iOS Simulator build (RTTyMobileApp) passes.

5. How to Undo

• Tyrell Hub: git revert af4aeaa in rdmsm4x:~/dev/apps/Tyrell and rebuild tyrelld.
• rdmpw3275m Daemon: Stop tmux window agy:tyrelld (tmux kill-window -t agy:tyrelld) and kill PID 14718.